Written by Carlos Eduardo de Paula, Senior Cloud Architect at Red Hat and RISC-V Ambassador

In this article, we will create a complete environment for developing and simulating embedded applications with no need for external hardware. (The files for the complete project can be cloned from this GitHub
repository: https://github.com/carlosedp/PlatformIO-Renode-Demos.)

We will install and configure Visual Studio Code, PlatformIO and Renode, all open-source tools and then create a simple project using Zephyr RTOS to be run on a simulated RISC-V HiFive1 board showing how the integration of these projects can bring benefits for the ecosystem and your workflow.

Embedded system developers know that usually the it is required not only the developer workstation but also the target devices, hardware to flash, configure and debug these devices like JTAG debuggers or dongles and cables. Lots of cables. Having a self-contained environment can help and speed-up your development and testing process.

At the end, you will have a setup similar to the picture below, making the toolchain management less painful allowing you to focus on the application itself.

Components

PlatformIO

PlatformIO is a new generation toolset for embedded development. It leverages the integration with IDEs to provide Intelligent code completion and Smart code linter with Built-in Terminal and Serial Port Monitor.

Its cross-platform build system without external dependencies to the OS provides 700+ embedded boards, 30+ development platforms, 20+ frameworks.

It contains thousands of Popular Libraries to be used on projects that are indexed into single platform with advanced search by keywords with project dependency management and Semantic Versioning requirements.

Platforms can be embedded and desktop targets composed of pre-built toolchains, debuggers, uploaders and frameworks which work under popular systems like Atmel AVR, ST STM32, SiFive, ESP and more. These platforms can even have sub-platforms that are the toolchains used for this platform for example on Atmel, you can develop using Arduino or Simba, on SiFive you can use their FreedomSDK or Zephyr, on STM32 you can use Arduino, Zephyr, Mbed and many more.

Renode

Renode is a development framework which accelerates IoT and embedded systems development by letting you simulate physical hardware systems — including both the CPU, peripherals, sensors, environment and wired or wireless medium between nodes.

It lets you run, debug and test unmodified embedded software on your PC — from bare System-on-Chips, through complete devices to multi-node systems. Applications built and run on Renode requires no changes to be run on the physical device.

This allows development for embedded systems without having access to the physical devices at all times. Currently, it supports the simulation of platforms from SiFive, STMicroelectronics, Microchip, Silabs and others. These platforms have STM32, RISC-V, Tegra and many other cores which can be composed as simple or complex multi-node configurations.

You can check the list of supported boards, or in more detail the available CPU’s and configuration scripts.

Zephyr RTOS

The Zephyr™ Project strives to deliver the best-in-class RTOS (Real-Time Operating System) for connected resource-constrained devices, built be secure and safe.

Zephyr is a open, collaborative environment to deliver an operating system that’s open source, applicable in a wide diversity of use cases, and currently supports more than 170 hardware devices.

Zephyr has a small-foorprint Kernel focusing on embedded devices compatible with x86, ARM, RISC-V, Xtensa and others but nothing stops you from developing an application able to be run on your computer console.

The built-in library provides drivers and stacks for multiple protocols and services. Explore the documentation for all the available features.

SiFive HiFive1 Board

The HiFive1 is a low-cost, Arduino-compatible development board featuring the Freedom E310 SOC. It’s the best way to start prototyping and developing your RISC‑V applications.

The E310 SOC contains a SiFive E31 RISC-V core following the RV32IMAC instruction set providing Integer, Multiplication, Atomic and Compressed instructions.

We will simulate the complete functionality of the HiFive1 board on Renode running an application and interacting with its serial console.

Remember, no hardware is required except your own computer! 🙂

Setting up your environment

The first step is setting the development environment. We will first install the pre-requisites and then we will see how PlatformIO will handle the installation of the development libraries for us.

Installing Visual Studio Code

Visual Studio Code is a free and Open-Source IDE platform created by Microsoft and available for most operating systems (Linux, Mac, Windows).

 Head to https://code.visualstudio.com/, download the installer package and follow the process to install on your system.

Install PlatformIO IDE on VSCode

After installing VSCode, install the PlatformIO IDE. It’s the platform that manages the embedded systems toolchains, build processes, code upload to the boards, debugging and much more.

It’s presented as a plugin for multiple editors including VSCode. You can check all it’s integrations on the documentation.

To install on VSCode, click the “Extensions” icon on the left menu bar and search for “platformio”. Click it’s name and “Install”. It might ask for VSCode restart.

When you first open VSCode, PlatformIO will present it’s “Home Screen” where you can quickly view the documentation about supported platforms, boards, the library browser and create/browse projects.

Installing Renode

Head to Renode website and download the package for your system and install accordingly. You can also check their release notes on GitHub for newly added support and features. Installing is a matter of dropping the app on the Mac, installing a MSI on Windows or a rpm/deb on Linux.

To make things easier on Mac, I created a symbolic link in the PATH to launch Renode easily:

sudo ln -sf /Applications/Renode.app/Contents/MacOS/macos_run.command /usr/local/bin/renode

This way I can just launch it by typing renode and pass command line arguments if needed. Also makes things easier on PlatformIO integration. I recommend creating this link on your platform of choice too.

Creating a new project

Now lets create a new project. For this article we will create a simple application using Zephyr RTOS that will present over its serial console some messages and a prompt. Any text you type and press enter will be printed back into the console. A simple “echo” app that is able to show how to setup, build, upload, debug and interact with the application running at the simulated board.

First, open VSCode. If the PlatformIO home doesn’t show, you can either click the tiny house icon at the bottom bar or the Alien icon on the left bar and then at the quick-access menu click “Open”.

The “Quick-access” and “Project Tasks” menus have all the tools you need to interact with your project. There are some shortcuts also that will be placed on the bottom bar.

At the PlatformIO Home, click “New Project”and name it “ConsoleEcho”. Then at the Board choose “HiFive1 (SiFive)” and in the “Framework” choose “Zephyr”. This defines the hardware you have and the toolchain that will be used.

Now, PlatformIO will start downloading all requirements to develop and build applications targeting this platform. A few minutes later depending on your internet connection you have a blank project with all basic structure of folders and files created. If you ever developed for embedded systems you know this is way better than starting from scratch.

This process happens for any kind of platform you plan to develop as target. PlatformIO will download and keep all toolchains and tools organized based on your requirements and use them according to the project.

Adding code to the project

Since we created a blank Zephyr project, we will add the source code to our application:

First rename the src/empty.c file by right-clicking it at the left tree and naming it main.c.

Now open the file and add the following content:

https://gist.github.com/carlosedp/bdc0c4e41a49c68a64ab8aab11521ad9

Next right-click the “zephyr” folder on the tree and click “New File”. Name it “prj.conf”. This is the file that holds Zephyr additional Kernel parameters. We will enable the console support by adding the two lines below:

Now we need to customize the project file platformio.ini adding the integration to Renode, the simulation platform. To do this, open the file and add the following:

https://gist.github.com/carlosedp/9fb584ff08ca17247c8d9aec2e87edcc

You see that the part below the comment line is for Renode Integration. If you plan to deploy this application in a real HiFive1 board in the future, remove or comment the added lines to restore defaults that work with the hardware boards.

What the added lines do is overriding the default upload (sending to the board), debug and monitor commands to connect to Renode instead of the real board. We also set the monitor port access as a network connection via telnet to localhost port 1234.

In Renode we have the option of exposing it’s features over a local network connection. Looking deeper line by line, here is what we are doing:

The debug section is very similar to the upload except that we don’t need to load the binary (it’s done over the GDB protocol) and the start command is issued after the binary is loaded. We also disable Renode logs.

Save all files and we are good to go.

Building your project

Now, you can build your project by using either the Project Tasks “Build” option or clicking the tiny “checkmark” icon at the bottom bar.

If everything goes well, you will have a new pane open on VSCode showing the terminal with all build output. You can investigate warnings or any error that might happen here.

Running your application on Renode

Now we will run the application on Renode. You can select the “Upload” option on Project Tasks or click the tiny “Right-arrow” at the bottom bar.

After building the app, PlatformIO opens Renode, issuing the commands we defined previously. Renode opens two windows, one named “Monitor” that is the main console where you can interact with Renode.

The second window is our simulated board serial console. Renode capture here all input and output as if you had a serial connection to your board. Now type something into the console to see it echoed back. Success!

We can also use PlatformIO integrated console by clicking the “Monitor” option on Project Tasks or the small plug icon in the bottom bar. It will open the serial console (connected over telnet to Renode):

You might need to click this pane and hit enter a couple times to have the terminal to connect. I’m investigating this and the weird characters issue.

To choose between the terminals that have the build output and the serial console, use the pull-down box in the top-right of this pane.

To close the terminal and/or the Renode session, click the little Trash Can icon at the top-right of this Terminal pane for each Task. You can also type “q” in Renode Monitor window.

If you change Zephyr Kernel parameters on prj.conf click the “Clean” option at the Project Tasks to rebuild all with correct features.

Debugging your project

Debugging your application follows a similar approach to Upload.

Here I’ve set two breakpoints, at the printk statements by hovering the line and clicking the red dot at the left of the line number. Then click the “Debug” button on the left sidebar and then the small “Play” icon at the top.

And there you go, the debug stopped at my first breakpoint where I can see the state of variables, registers and all environment during execution.

Navigate thru the debugging session using the tools for “Continue”, “Step-over”, “Step-into”, “Step-Out” and “Stop”. Restarting is yet not working in the integration. To close the debugging session, click the “Stop” button where VSCode will disconnect and Renode will be closed.

Check more details on debugging in VSCode at https://code.visualstudio.com/docs/editor/debugging.

Conclusion

As we saw, by integrating three tools, VSCode, PlatformIO and Renode we can have a complete embedded systems environment allowing you to be productive and play with these technologies without the need to buy hardware and carry them around.

The process used here could be used for other SDKs like SiFive’s Freedom-E or even using a similar process to develop for other boards that are supported by PlatformIO and Renode like STM32 using Arduino framework.

Also if you want to contribute, all these tools are open-source projects that you can bring support for your favorite board or improve some platform.

Zephyr RTOS: https://github.com/zephyrproject-rtos/zephyr/

Antmicro Renode: https://github.com/renode/renode/

Visual Studio Code: https://github.com/microsoft/vscode

PlatformIO: https://github.com/platformio/platformio-core/

Follow me on Twitter, Github or Linkedin and send me your feedback about the article and improvements to the workflow.

Adafruit, who recently joined the Zephyr Project ecosystem, launched a new guide from the Adafruit Learning System – Blinking an LED with the Zephyr RTOS. This blog originally ran on Adafruit’s website. For more content from this Zephyr Project member, visit their blog.  

There are lots of ways to make programming microcontrollers really easy – CircuitPython, MicroPython, and Arduino are all options to get your project up and running, even as a beginner programmer. But sometimes you don’t just need easy – you need beefy. When it’s time to break out the big guns, you might consider using an RTOS – a Real Time Operating System, sort of a very tiny version of what runs on your desktop or laptop computer, but one that’s built for single-chip microcontrollers like those on an Arduino or Feather board.

RTOSes can be big and complex, since they’re usually marketed toward corporate teams or very experienced freelancers. But they don’t have to be hard to learn! In this guide, we’ll be sticking to the basics – getting an LED up and running in Zephyr, which has been backed by the Linux Foundation, Intel, NXP, and many other powerful microcontroller companies.

You’ll learn how to:

• Install the Zephyr core on Mac OSX or Linux computers
• Install Zephyr’s custom management tool, West
• Test your setup with the RTOS’s built-in sample projects
• Create your own application folder
• Blink an LED on the Feather STM32F405 Express
• Start learning RTOS concepts for custom projects

Overview

There are lots of ways to make programming microcontrollers really easy – CircuitPython, MicroPython, and Arduino are all options to get your project up and running, even as a beginner programmer. But sometimes you don’t just need easy – you need beefy. When it’s time to break out the big guns, you might consider using an RTOS – a Real Time Operating System, sort of a very tiny version of what runs on your desktop or laptop computer, but one that’s built for single-chip microcontrollers like those on an Arduino or Feather board.

An RTOS is built to handle chips with lots of features automatically, juggling sensors, buses, screens and buttons without huge messes of custom code to manage them all. Unlike Arduino’s startup/loop, or Circuitpython’s while True:, an RTOS can run many different operations (called Tasks) in parallel, never allowing any one task to fall too far behind. This means an RTOS is great for big, sprawling projects that have a lot of things running at once, or for projects like sensor data collection where one task is so critical that it needs to be constantly serviced. 

However, all this capability comes with a cost – RTOSes can be big and complex, since they’re usually marketed toward corporate teams or very experienced freelancers. But they don’t have to be hard to learn! In this guide, we’ll be sticking to the basics – getting an LED up and running in an up-and-coming RTOS, Zephyr, which has been backed by the Linux Foundation, Intel, NXP, and many other powerful microcontroller companies.

You’ll learn how to:

• Install the Zephyr core on Mac OSX or Linux computers
• Install Zephyr’s custom management tool, West
• Test your setup with the RTOS’s built-in sample projects
• Create your own application folder
• Blink an LED on the Feather STM32F405 Express
• Start learning RTOS concepts for custom projects

Parts

The heart of this project is the Feather STM32F405 Express:

Adafruit Feather STM32F405 Express

Depending on whether you want to work exclusively off of USB, or use a JLink programmer, you may also need one or more of the following parts:

For more information about recommended parts, please visit the Adafruit website.

If you have feedback or questions about Zephyr, please join our Slack channel or Mailing List.

Additionally, we invite you to try out Zephyr 2.1. You can find our Getting started Guide here.  If you are interested in contributing to the Zephyr Project please see our Contributor Guide.

Last week at Embedded World 2020 more than 900 exhibitors and 1,500 speakers and participants from 46 countries made it to Nuremberg, Germany. There were around 13,800 visitors this year and most of them had positive feedback about the show. However, this number was significantly lower than previous years as many companies decided to pull their participation at the last minute as a result of new travel restrictions.

The Zephyr Project canceled our participation in order to keep our member company volunteers and spokespersons healthy and safe. But the show must go on, so we’ve teamed up with some our members to showcase the demos that would have been showcased at Embedded World. Check them out below!

Linaro and its members are developing a set of software components and tools to help with a system approach to multiprocessing. This short video shows a multiprocessing demo running on the STM32MP1 from STMicroelectronics.

Featured Linaro work in this demo includes:

• Devicetree – an on-going area of work in Linaro
• OpenAMP – a Linaro Community Project
• 96Boards specification – Avenger96 Community Board and mezzanine expansion
• Zephyr – Linaro is a contributor and member of the Zephyr project

For more details of these technologies, please see our previous blog on heterogeneous multicore systems.

In this video, Nordic Semiconductor demonstrates how to build the Zephyr RTOS twice in order to generate two images, one for each of the two cores inside Nordic’s flagship nRF5340 Bluetooth Low Energy SoC. It describes how the two cores communicate using OpenAMP (using the RPMsg protocol) to connect the Bluetooth LE Host running in the Application core with the Bluetooth Controller running in the Network core. Finally, it also shows how to analyze and decode Bluetooth traffic generated by the open source Bluetooth Low Energy stack built-in in Zephyr. Read about the nRF5340 here: https://www.nordicsemi.com/Products/L…

This NXP video demonstrates Bluetooth mesh functionality – specifically controlling either the RGB lens and/or color intensity on multiple development boards while highlighting the capabilities of i.MX RT1050 crossover MCU.

If you have feedback or questions on our Slack channel or Mailing List.

Additionally, we invite you to try out Zephyr 2.1. You can find our Getting started Guide here.  If you are interested in contributing to the Zephyr Project please see our Contributor Guide.

On February 25-27, more than 32,000 people will be in Nuremburg, Germany for Embedded World 2020. The event offers the global community the opportunity to learn about new developments in all aspects of embedded system technologies. Last year, 1,117 companies from 42 countries, almost 100 more than in the previous year, showcased their solutions to the around 31,000 trade visitors from 84 countries at this leading fair for the embedded community.

The Zephyr Project is back at the show again this year – this time with a booth filled with innovative demos, speaking engagements and products/solutions at several member and community booths including: Antmicro (4A-621), Foundries.io (5-440), Nordic Semiconductor (4A-310), PHYTEC (1-438) and RISC-V (3A-536). Here’s a sneak peek at some of the Zephyr demos at their booths:

Antmicro (4A-621): Antmicro will showcase three live demos featuring the Zephyr Real Time Operating System.

Zynq Video Board: a Zephyr-capable FPGA SoC open camera board

The Zynq Video Board is Antmicro’s open hardware platform for high-speed SoC FPGA video processing. Designed to accommodate the Xilinx Zynq (or other FPGA modules), this fully open source board allows interfacing up to two 2-lane MIPI CSI-2 cameras and offers practical HDMI, Ethernet, and SD card support. The EW 2020 demo will feature camera sensors controlled by Zephyr running on a VexRiscV RISC-V softcore – another example of how Antmicro is using and contributing to open source IP, enabling entire FPGA processing pipelines built on open source.

GEM: a rapid turnaround chiplet-based ASIC with Zephyr

Antmicro’s GEM chiplet-based ASIC series was introduced at the 2019 RISC-V Summit to prove that rapid turnaround chips based on open ISAs are enabling engineers to create custom solutions within a practical, quick-prototyping / quick-tape-out process. The GEM demo will show a real-time analysis of video footage from a camera running a Lattice iCE40 FPGA inside the GEM chip. The chip itself features a RISC-V core in a soft SoC (LiteX) and runs Zephyr RTOS for simple control tasks or re-programming the processing FPGA with different bitstreams.

QuickLogic EOS S3 with Zephyr port
Antmicro has recently added a Zephyr port to QuickLogic’s Quick Feather development board for the EOS S3 and the soon-to-be-released addition to the open source Tomu tiny USB family of devices, nicknamed Qomu – both of which will be showcased at Antmicro’s booth. The EOS S3 is also supported in Antmicro’s Renode open source simulation framework for rapid prototyping, development and testing of multi-node systems, offering a more efficient hardware/software co-design approach to Zephyr developers.

PHYTEC (1-438): PHYTEC will feature advanced technologies in the fields of test & measurement, healthcare, agriculture, fitness, energy, transportation, industrial IoT and security applications. They will also be giving out 50 reelboards. The key demonstrations in the system solutions area will showcase: 

Connect 4 demo (Human vs Machine): The PHYTEC object detection, inference, and real-time robotics demo (aka Connect4 Demo)  is a classic showdown of Machine vs Human. The Machine, in this case, is a robotic arm, a ball return system, a camera, and a PHYTEC phyCORE-AM5729 embedded system that sees the human’s move, plans its own move, places the game piece, and cleans up after near certain victory. 

Low Power Solutions for IoT: 
        a) different display technologies (passive/EPD, OLED, LCD) using Zephyr OS 
        b) reel board with link boards as a rapid prototyping system for Low Power Applications 
        c) presenting a new link board NOTM.2 module with Nordic nRF5340 
        d) presenting a new link board CAN (Communication via CAN-bus between Linux and Zephyr OS) 

Not only will Zephyr be on display in these hands-on demos, but it will also be featured in several presentations. For more details about the Zephyr presentations, click here: https://www.zephyrproject.org/embedded-world-february-25-2020-february-27-2020/

We hope to see you at Embedded World! Tag @ZephyrIoT in your tweets and we’ll retweet your photos and messages!

This blog originally ran on the Antmicro website. For more Zephyr development tips and articles, please visit their blog.

The flexibility of FPGAs is an excellent match to any format conversion tasks, and at Antmicro we’ve been using FPGAs from various vendors to convert fast data streams between interfaces like MIPI CSI-2, HDMI, SDI, USB, PCIe and others. The increasing capabilities and decreasing cost of larger FPGAs means that new applications are becoming feasible in markets which previously could not afford to use an FPGA, and use cases that previously seemed to be reserved to large server rooms or heavyweight PCI-based industrial systems can be converted to compact, low-power and low-cost embedded devices.

The Internet is full of devices which at least nominally convert almost any input to any output, a majority of such data conversion and streaming solutions are however proprietary and come with their own closed bitstreams (often called firmware), binary drivers (even if there is a concept of Linux support, this is done with binary kernel modules) and little documentation. It’s extremely hard – or downright impossible – to integrate such devices in any product that does not exactly match whatever application the original vendor had in mind, let alone guarantee long-term availability and upgrade paths for other use cases. That is why at Antmicro we take an entirely different approach and help our customers create freely programmable and well-documented hardware, firmware and software for all types of data conversion.

Concept: smart conversion/streaming device

This document presents a concept of a universal conversion device like many similar ones we had built for our own and our customers’ use cases.

A smart SDI and HDMI to HDMI, MIPI or Ethernet converter / streamer, this device uses the freely programmable on-board FPGA to enable video format conversion, live processing, encoding and streaming. The video stream coming from an SDI or HDMI source can be processed with minimal latency and then output as HDMI – directly onto a display, or MIPI CSI-2 – for further processing on an embedded device.

Linux and Zephyr capabilities

The processing FPGA runs a compact, open source soft SoC system – LiteX – with a RISC-V CPU that is both Linux and RTOS capable, offering incredible flexibility. By default, the device uses a compact Zephyr RTOS configuration, a natural choice since Antmicro is a member of the Zephyr project and maintains the Zephyr LiteX port.

In the Linux configuration, it is very simple to build on top of standard Linux software for an embedded web control server, live analysis and reporting software. The built-in RISC-V core can be easily extended with custom accelerators, and the FPGA can be used for entirely custom processing blocks, such as codecs. For software development purposes, the processing system is possible to simulate in Antmicro’s open source Renode framework, which allows for software/FPGA co-simulation.

The device makes it possible to perform live processing on the video signal, including overlays, color processing and AI tasks such as object detection/classification. It provides an Ethernet port for control, debugging and – potentially – encoding and streaming data over the network.

Integration with more complex systems

The converter can be used in standalone form, but its MIPI CSI-2 also allows to connect it to a variety of Antmicro’s MIPI-capable hardware, such as our open source Jetson Nano / Jetson Xavier NX board, or the Apalis iMX8/TK1 Smart Vision kit, creating a complete edge AI video processing platform with SDI and HDMI camera input capabilities.

Antmicro provides both FPGA and OS/driver development services, host software development and customized hardware services to help you build your custom project similar to the one described above. If you are interested in building your next project with Antmicro, give a shout at contact@antmicro.com.

Last week, the Internet of Things Helsinki hosted another meetup on Thursday, January 23 from 5-8 pm at the Central Library Oodi. Almost 50 folks registered for the event, which focused on Industrial IoT.

Jefim Borissov with Nortal Oy kicked off the meetup and presented IIoT as part of DDD solutions. He shared why and when DDD matters, common challenges of IIoT and how DDD principles can be applied to the IIoT industry.

Then, Andrei Laperie, Zephyr Project TSC member and Engineer Manager for Intel Open Source Technology Center, discussed industrial networking support in Zephyr RTOS. In this talk, he offered a brief introduction of Zephyr as well as Time Sensitive Networking and CAN support in an industrial environment.

If you couldn’t make it to the meetup, below is the presentation Andrei went through. Look it over and let us know if you have feedback or questions on our Slack channel or Mailing List.

Additionally, we invite you to try out Zephyr 2.1. You can find our Getting started Guide here.  If you are interested in contributing to the Zephyr Project please see our Contributor Guide.

Written by David Leach, Software Architect at NXP Semiconductor and member of the Zephyr Technical Steering Committee

Last month, the Zephyr Project announced the release of Zephyr RTOS 2.1. A long list of enhancements and bug fixes contained in Zephyr 2.1 can be found in the release notes. 

Major Enhancements

·         Normalized APIs across all architectures.

·         Expanded support for ARMv6-M architecture.

·         Added support for numerous new boards and shields.

·         Added numerous new drivers and sensors.

·         Added new TCP stack implementation (experimental).

·         Added BLE support on Vega platform (experimental).

·         Memory size improvements to Bluetooth host stack.

The Numbers

This release is the result of the hard work and skill of over 350 individuals engaged with the project over the last 3 months with over 1500 PRs merged and 532 issues closed. We would like to thank all those who engaged with the project both in front and behind the scenes to help improve the Zephyr Project for this release.

Sample boards that now have support

• 

• 

• 

• 

• 

• 

What’s Next

Improvements to Zephyr Project never stops. Work continues on the new TCP stack implementation, many different activities with Bluetooth, converting GPIO drivers to the new GPIO API,  and many other enhancements and bug fixes.

Join Us

We invite you to try out Zephyr 2.1. You can find our Getting started Guide here.  If you are interested in contributing to the Zephyr Project please see our Contributor Guide. Join the conversation or ask questions on our Slack channel or Mailing List.

This blog originally ran on the Antmicro website. For more Zephyr development tips and articles, please visit their blog.

While much of the focus for the recent developments in AI has been on cloud-centric implementations, there are many use cases where AI algorithms have to be run on small and resource constrained devices. Google’s TensorFlow Lite, a smaller brother of one of the world’s most popular Machine Learning frameworks, is focused on exactly that – running neural network inference on resource constrained devices. A more recent but very exciting effort, led by Pete Warden’s team at Google in collaboration with partners like ARM, Ambiq Micro, Sparkfun and Antmicro, aims to bring TF Lite to microcontrollers, with sub-milliwatt power envelopes and sub-dollar unit costs for an AI-enabled node.

Antmicro’s Renode and Google’s TF Lite teams have been collaborating to use Renode for demonstration and testing of the ML framework, and bring it to new frontiers with real industrial use cases. Last year, we helped port the framework to RISC-V, the open ISA which encourages collaborative software-driven hardware innovation. For this year’s RISC-V Summit, we are announcing a next step in bringing TF Lite to play well with existing standards: TF Lite micro for the Zephyr RTOS on LiteX/VexRiscv.

Why is LiteX and FPGA a good platform for TF Lite?

The recently released TensorFlow Lite port to Zephyr for LiteX/VexRiscv presents a proof of concept implementation of TF Lite running on a small soft CPU-based system in FPGA.

FPGA devices are often used to accelerate parallel data processing, including deep neural network inference. Their dynamic reconfiguration capability is a good match to the changing needs of ML, where new advancements in the software space happen too fast for traditional hardware approaches to adapt. With the availability of small FPGAs that are freely programmable with open source tooling, also extremely low-power applications like the ones encouraged by TF Lite for MCUs can be targeted. The reconfigurability of FPGA devices is an interesting aspect, as new hardware configurations can be deployed to existing devices to get the best of the latest ever-evolving ML applications.

On the other hand, real neural network inference applications do require some sequential processing of data and top-level code controlling the flow. In the FPGA world, the way to solve this problem would be to implement a soft CPU core, and that’s what we have done for dozens of applications, with a standardized soft SoC solution called LiteX and an excellent RISC-V CPU implementation that plays well with LiteX, VexRiscv. LiteX comes with wide FPGA platform support that we actively help develop, and a variety of I/O options, starting with the UART, SPI or I2C, but also covering Ethernet, PCIe, USB and SATA for larger systems. It has been our soft SoC of choice, and we have implemented support for it in our open source simulation framework, Renode, as well as both Zephyr and Linux.

In the TF Lite context, LiteX combines the best of both worlds – as it enables a system designer to easily create a soft CPU based SoC in FPGA and focus on the capability to extend it with custom, ML-oriented blocks.

Why Zephyr?

Zephyr is a small but powerful RTOS capable of running on resource constrained devices. POSIX compatibility allows easy integration of existing code bases, and baked-in vendor neutrality ensures the RTOS will work well RISC-V’s open and collaborative nature. Throw in clear licensing, powerful tooling and the most dynamic growth in the RTOS space, and you have a natural winner.

Antmicro is actively developing Zephyr support for RISC-V in general and LiteX/VexRiscv in particular, so porting Tensorflow Lite to Zephyr and running it on a Litex/VexRiscv system was a natural choice.

Demos

Two demos have been prepared to present the functionality of the system:

• A simple “Hello World” Tensorflow lite application which prints sine function values on the serial terminal,
• A “Magic Wand” application which recognizes different shapes “drawn” in the air with the board; the demo collects data from the accelerometer and feeds it to a neural network performing gesture recognition.

The system is composed of three main components. All the parts have been released to public repositories on Antmicro’s GitHub:

• Tensorflow,
• Zephyr,
• Litex-buildenv.

To make the build process easier, a master repository has been created linking the other repositories as submodules.

To set up the build environment and build the demos, follow the instructions found in the README file located in the master repository.

Running the demos

The demos can be run either on hardware or in the Renode simulator. Running on hardware requires a setup with the Digilent Arty A7-35T board with PmodACL connected to the JD port.

The onboard FPGA chip has to be programmed with gateware bitstream. After the board has been programmed, the LiteX System will boot. On a serial console you should see a Litex Bios prompt. In order to run the demo, the compiled application binary has to be uploaded to the device. The binary can be uploaded to the device using the serial connection with the litex_term tool.

The application starts as soon as it is uploaded. The “Magic Wand” application will print a welcome message while waiting for the accelerometer data. As soon as a gesture is detected, the application will print the results. See the listing below for an example of the output of the application:

Got id: 0xe5
*****
Booting Zephyr OS build v1.7.99-22021-ga6d97078a3e2
*****
Got accelerometer
RING:
          *
       *     *
     *         *
    *           *
     *         *
       *     *
          *
SLOPE:

        *
       *
      *
     *
    *
   *
  *
 * * * * * * * *

Running the demo in Renode does not require the physical board or fiddling with gateware. Detailed instructions on how to run the demo applications in Renode can be found in the master repository README file.

Near future plans

The demos present the functionality of the system and prove that TensorFlow Lite can be successfully run in Zephyr on a LiteX system with a VexRiscv CPU. However, additional work needs to follow for the code to be merged upstream. While the FPGA platform definition code – done as part of our earlier efforts – has been merged some time ago, Zephyr support for Tensorflow Lite and the Zephyr driver for the accelerometer used in the demo have not yet been added upstream. If you have an application which would benefit from running TensorFlow Lite with Zephyr, make sure you visit our booth at the RISC-V Summit or contact us at contact@antmicro.com.

By Maureen Helm, Chair of the Zephyr Project Technical Steering Committee

The Zephyr community converges every year at the Embedded Linux Conference Europe, and 2019 was no exception. This year we traveled to Lyon, France for an engaging week full of technical talks, spontaneous hallway conversations and hacking sessions, team dinners, and perhaps a nice glass of wine or two. It was a wonderful opportunity to get to know some of our newer members in person and finally put faces to familiar names and voices.

After the main conference, the Zephyr technical steering committee stayed on for two days of face-to-face meetings, including a few dial-ins from those who couldn’t make the trip. Compared to our weekly calls, the longer format F2F meeting allowed us to discuss and debate issues in greater depth, and make decisions about the technical direction of the project.

#1 – Mainline releases: Historically we have aimed for quarterly releases, but will shift to a 4-month cycle in 2020. More details, including dates, are in the Program Management wiki page.

#2 – LTS releases: We clarified that LTS releases will be maintained for two years, and LTS2 will be released approximately two years after LTS1. We did not decide on a cadence beyond LTS2.

#3 – Toolchains: We agreed that multiple members have an interest in supporting commercial toolchains and will kickoff a new toolchain-focused working group.

#4 – User Experience: We brainstormed possible solutions to common problems encountered by new developers.

#5 – Roles and Responsibilities: We debated a contributor ladder towards maintainership, how to distribute merge rights, and how to fill the release manager role for future releases. This conversation has continued in subsequent process working group meetings.

Thank you to everyone who joined, and I look forward to seeing you next time at the Embedded Linux Conference in Austin, Texas!