Posts Tagged ‘programming’

A Look at Three Options to Develop Real-Time Linux Systems on Application Processors – HMP, Real-Time Linux and Xenomai

October 15th, 2016 6 comments

This is a guest post by written by Guilherme Fernandes, Raul Muñoz, Leonardo Veiga, Brandon Shibley, all working for Toradex.


Application processor usage continues to broaden. System-on-Chips, usually powered by ARM Cortex-A cores, are taking over several spaces where small ARM Cortex-M, and other microcontroller devices, have traditionally dominated. This trend is driven by several facts, such as:

  • The strong requirements for connectivity, often related to IoT and not only from a hardware point of view, but also related to software, protocols and security
  • The need for highly interactive interfaces such as multi-touch, high resolution screens and elaborate graphical user interfaces;
  • The decreasing price of SoCs, as consequence of its volume gain and new production capabilities.

Typical cases exemplifying the statement above are the customers we see every day starting a product redesign upgrading from a microcontroller to a microprocessor. This move offers new challenges as the design is more complicated and the operating system abstraction layer is much re complex. The difficulty of hardware design using an application processor is overcome by the use of reference designs and off-the-shelf alternatives like computer-on-modules or single board computers. On the operating system layer, the use of embedded Linux distributions is widespread in the industry. An immense world of open source tools is available simplifying the development of complex and feature rich embedded systems. Such development would be very complicated and time consuming if using microcontrollers. Despite all the benefits, the use of an operating system like Linux still raises a lot of questions and distrust when determinism and real-time control application topics are addressed.

A common approach adopted by developers is the strategy of separating time-critical tasks and regular tasks onto different processors. Hence, a Cortex-A processor, or similar, is typically selected for multimedia and connectivity features while a microcontroller is still employed to handle real-time, determinism-critical tasks. The aim of this article is to present some options developers may consider when developing real-time systems with application processors. We present three possible solutions to provide real-time capability to application processor based designs.

Heterogeneous Multicore Processing

The Heterogeneous Multicore Processing (HMP) approach is a hardware solution. Application processors like the NXP i.MX7 series, the NXP i.MX6SoloX and the upcoming NXP i.MX8 series present a variety of cores with different purposes. If we consider the i.MX7S you will see a dual core processor composed of a Cortex-A7 core @ 800MHz side-by-side with a Cortex-M4 core @ 200MHz. The basic idea is that user interface and high-speed connectivity are implemented on an abstracted OS like Linux with the Cortex-A core while, independently and in parallel, executing control tasks on a Real-Time OS, like FreeRTOS, with the Cortex-M core. Both cores are able to share access to memory and peripherals allowing flexibility and freedom when defining which tasks are allocated to each core/OS. Refer to Figure 1.

NXP i.MX7 Block Diagram (Click to Enlarge)

Figure 1 – NXP i.MX7 Block Diagram (Click to Enlarge)

Some of the advantages of using the HMP approach are:

  • Legacy software from microcontrollers can be more easily reused;
  • Firmware update (M4 core) is simplified as the firmware may be a file at the filesystem of the Cortex-A OS;
  • Increased flexibility of choosing which peripherals will be handled by each core. Since it is software defined, future changes can be made without changing hardware design.

More information on developing applications for HMP-based processors are available at these two articles:

Toradex, Antimicro and The Qt Company collaboratively built a robot showcasing this concept. The robot – named TAQ – is an inverted pendulum balancing robot designed with the Toradex Computer on Module Colibri iMX7. The user interface is built upon Linux with the QT framework running on the Cortex-A7 and the balancing/motor control is deployed on the Cortex-M4. Inter-core communication is used to remote control the robot and animate its face as seen in the short video below.

Real-Time Linux

The second approach we present in this article is software related. Linux is not a real-time operating system, but there are some initiatives which have greatly improved the determinism and timeliness of Linux. One of these efforts is the Real-Time Linux project. Real-Time Linux is a series of patches (PREEMPT_RT) aimed at adding new preemption options to the Linux Kernel along with other features and tools to improve its suitability for real-time tasks. You can find documentation on applying the PREEMPT_RT patch to the Linux kernel and developing applications for it at the official Real-Time Linux Wiki (formerly here).

We did some tests using the PREEMPT_RT patches on a Colibri iMX6DL to exemplify the improvement in real-time performance. The documentation on preparing the Toradex Linux image to deploy the PREEMPT_RT patch is available at this link. We developed a simple application which toggles a GPIO at a 2.5KHz (200µs High / 200µs Low). The GPIO output is connected to a scope where we measure the resulting square wave and evaluate the real output timings. The histograms below show the comparison between the tests on a standard Linux kernel configured for Voluntary Preemption (top) and a PREEMPT_RT patched Linux kernel configured for Real-time Preemption (bottom). The x-axis represents the period of the square wave sample and the y-axis represents the number of samples which measured with such a period. The table below the chart presents the worst and average data.

Click to Enlarge

Figure 2: Histogram of the square wave generated using the standard Kernel (top) and Preempt-RT kernel (bottom) – Click to Enlarge



Smallest (µs)

Worst Case for 99% of  Samples (µs)

Worst Case (µs)

Median (µs)

Average (µs)

Default Kernel














Table 1: Comparison between Default Kernel and real-time Kernel when generating a square wave.

An example software system using the PREEMP_RT patch is provided by Codesys Solutions. They rely on the Real-Time Linux kernel, together with the OSADL (Open Source Automation Development Lab), to deploy their software PLC solution which is already widespread throughout the automation industry across thousands of devices. The video below presents the solution running on a Apalis iMX6Q.


Xenomai is another popular framework to make Linux a real-time system. Xenomai achieves this by adding a co-kernel to the Linux kernel. The co-kernel will handle time-critical operations and will have higher priority than the standard kernel. To use the real-time capabilities of Xenomai the real-time APIs (aka libcobalt) must be used to interface user-space applications with the Cobalt core, which is responsible for ensuring real-time performance.


Figure 3: Dual Core Xenomai Configuration

Documentation on how to install Xenomai on your target device can be found at the Xenomai website. Additionally, there is a variety of Embedded Hardware which is known to work as indicated in the hardware reference list, which includes the whole NXP i.MX SoC series.

To validate the use of Xenomai on the i.MX6 SoC we also developed a simple experiment. The target device was the Colibri iMX6DL by Toradex. We ran the same test approach as described above for the Real-Time Linux extension. Some parts of the application code used to implement the test are presented below to highlight the use of Xenomai APIs.

The results comparing Xenomai against a standard Linux kernel are presented in the chart below. Once again, the real-time solution provides a clear advantage – this time with even greater distinction – over the time-response of the standard Linux kernel.

Click to Enlarge

Figure 3: Histogram of the square wave generated using the standard Kernel (top) and Xenomai (bottom) – Click to Enlarge



Smaller (µs)

Worst Case for 99% of Samples (µs)

Worst Case (µs)

Median (µs)

Average (µs)

Default Kernel







Xenomai Implementation







Table 2: Comparison between Default Kernel and Xenomai implementation when generating a square wave.


This article presented a brief overview of some solutions available to develop real-time systems on application processors running Linux as the target operating system. This is a starting point for developers who are aiming to use microprocessors and are concerned about real-time control and determinism.

We presented one hardware-based approach, using Heterogeneous Multicore Processing SoCs and two software based approaches namely: Linux-RT Patch and Xenomai. The results presented do not intend to compare operating systems or real-time techniques. Each of them has strong and weak points and may be more or less suitable depending on the use case.

The primary takeaway is that several feasible solutions exist for utilizing Linux with application processors in reliable real-time applications.

Top 10 Programming Languages in 2016 for Embedded Software Development

July 27th, 2016 6 comments

IEEE Spectrum has published a list of the top programming languages in 2016 for Web, Mobile, Enterprise, and Embedded sectors with rankings created by weighting and combining 12 metrics from 10 sources. So I thought it would be fun to have a look at the top 10 of languages used for embedded software, and the results are:

Top_10_Embedded_Programming_Languages_2016As expected, C and C++ are at the top, but I’m quite surprised that “Arduino” is now considered a programming language, as it is simply based on C/C++.  When I worked as an embedded software engineer a few years ago, I personally used C, and Assembly, and to a lesser extend C++ and VHDL. I only recently started to play with Arduino code, and while I’ve heard of most other languages in the list, it’s the first time I’ve ever seen Ladder Logic, probably because it’s designed to program PLCs in industrial control applications.

The methodology used is interesting, as the company did not survey actual engineers or developers, but instead used data from technical, social and job search websites, such as Github, Twitter, or CareerBuilder, to generate the rankings.

Google Summer of Code 2015 is Now Open for Student Applications

March 19th, 2015 2 comments

Google has now announced that students applications for Google Summer of Code (GSoC) are now open. Students can get paid up to $5,500 to work on various open source projects selected for the event.

GSoC_2015Fewer companies have been accepted this year, and even big names like the Linux Foundation and Mozilla got their application rejected. There are still over 137 open source projects to work on including:

  • MinnowBoard  project – Potential software projects for the Intel Atom embedded board include making low speed I/O buses more accessible via intermediate open source libraries (e.g. SMBus/PMBus/Wiring libraries), and improving the open source firmware.
  • lowRISC SoC project – Potential projects: Schematic Viewer for Netlists (SVG/JavaScript), open source FPGA compilation flow using Yosys, accessing the OpenCores ecosystem, etc…
  • – Lots of project ideas relying on the BeagleBone Black board, dealing with Linux kernel support for embedded devices and interfaces, ARM processor support in open source operating systems and libraries, Heterogeneous co-processor (PRU) support in open source operating systems and libraries, and more.

Interested students can browse the projects, and submit their own proposals based on the “idea pages” or not, before Friday, March 27 at 19:00 UTC on the application page.

Students who are accepted will work on an actual open source software project over the summer, be paired with a mentor, and get paid for their work.  You need to be at least 18 years old, and be enrolled in an accredited academic institution anywhere in the world. You don’t necessarily need to be follow a Computer Science or Electronics Engineering program to apply, as past students Shave also come from disciplines such as Ecology, Medicine and Music. Getting accepted to GSoC and having worked on an open source project for several weeks is certainly something nice to have on your CV. Good luck!

Thanks to Alex (lowRISC) for the tip.

Learn How to Write a Driver for Linux 3.x With The Linux Driver Template

November 14th, 2012 No comments

A Linux Driver Template (LDT) has been published to help new Linux kernel developers writing hardware device drivers.

Constantine Shulyupin posted the Linux Driver Template (LDT) on the Linux mailing list in order to merge it into the mainline Linux kernel. The code can be used as as a starting point for new drivers, and shows how to use several Linux facilities such as  module, platform driver, file operations (read/write, mmap, ioctl, blocking and nonblocking mode, polling), kfifo, completion, interrupt, tasklet, work, kthread, timer, simple misc device,
multiple char devices, Device Model, configfs, UART, hardware loopback, software loopback and  ftracer.

This sample has been added to other device drivers samples in And if you want to learn further there’s always the Linux driver bible: “Linux Device Drivers, Third Edition” which can be downloaded for free as PDF, although it’s for 2.6.10 kernel and many parts may not be up-to date.

Via: Phoronix

Debian is Worth a Lot (Yet it’s Free) and C/C++ Language Still Rules

February 16th, 2012 No comments

James E. Bromberger (JEB) , a contributor to Perl CPAN and Debian, has estimated the cost of developing Debian Wheezy (7.0) from scratch based on the the number of lines of code (LOC) counted with SLOCCount tool, the Constructive Cost Model (COCOMO) and the average wage of a developer of 72,533 USD (using median estimates from and for 2011).

He found 419,776,604 lines of code in 31 programming languages giving an estimated cost of producing Debian Wheezy in February 2012 of 19 billion US dollar (14.4 Billion Euros), making each package source code (out of the 17,141 packages) worth an average of 1,112,547.56 USD to produce.

He also estimated the cost of Linux 3.1.8 Kernel with almost 10 millions lines of source code would be worth 540 million USD at standard complexity, or 1.877 billions USD when rated as ‘complex’.

I don’t know which tool he used for the calculation (maybe his own), but there are two simple COCOMO calculators on the internet in order to estimate the number of man hours required for a particular project based on the LOC and the code complexity:

Bromberger also estimated the cost of several individual projects used in Debian and found that developing Apache 2.2.9 would cost 33.5 million USD and MySQL 64.2 million USD with today’s salary.

Of course, there are some caveats to such cost estimation. First, those costs assume software engineers based in the US. Taking into account outsourcing would significantly reduce those cost estimations. Second, he also tried another source code analysis tool (Ohcount) which found much less lines of code in Debian Wheezy. Finally, you also need to estimate the code complexity in the COCOMO model which may lead to great variation in the final costs.

In the last section of his post, Bromberger also analyzed the 31 programming languages that were used to develop the Debian software (See his chart below). It shows C and C++ still rule the (programming) world with respectively 40% and 20% of code written in those 2 programming languages. Java comes a distant third at 8%.

C/C++ Rules the World !!!

Top 11 Programming Languages of Debian Wheezy

Android Training & Tutorials

December 16th, 2011 No comments

The Android team just announced the launch of Android Training a collection of classes aimed at helping developers to build better Android applications.

Each class explains the steps required to solve a problem, or implement a feature, with code snippets and sample code.

They’ll add more materials over time, but right now developers can already learn (more) about the following Android Graduatetopics:

  • Designing for Multiple Screens
  • Improving Layout Performance
  • Managing Audio Playback
  • Optimizing Battery Life
  • Remembering Users
  • Sharing Content
  • Capturing Photos
  • Maintaining Multiple APKs
  • Developing for Enterprise
  • Monetizing Your App
  • Designing Effective Navigation

There are currently 34 lessons on all those topics, and most include code samples such as the News Reader  (Designing for Multiple Screens).

You can get started immediately by visiting Android Training page.

Qt Quick QML Digital Signage Demo Part 2

December 12th, 2011 4 comments

Following up on Qt Quick QML Digital Signage Demo Part 1, I’ve updated the digital signage demo to support the following:

  • Play 5 videos in a loop
  • Display 5 pictures in a loop where the picture is changed every 5 seconds
  • Use a (twitter) RSS feed for the scrolling text
Digital Signage Programmed with QtQuick/QML

QML Digital Signage Demo Screenshot

I initially planned to use QML to list the media files, but it is apparently not possible without using C/C++ and I may do it later on. So instead, I hard-coded the video and picture playlists in the QML files with the ListModel element. Videos are located in the video directory and pictures in the pic folder.

An index is needed to scroll thru the playlist, but QML does not support global variables, so I created a JavaScript file (globals.js) to store the video and picture index:

// Global variables in JavaScript file
var iVideoIndex = 0
var iPicIndex = 0

Then imported the script at the top of main.qml:

Now the code to play 5 videos in a loop looks like:

I only used Windows Media (wmv), WebM and Mpeg video, as it seems the AVI and MP4 video I used made the emulator crash.

The picture zone uses a 5 files list (pic/pic1.jpg to pic/pic5.jpg) with a 5 second timer:

In the code above, I had to switch parent.visible between false and true or the picture would not be updated.

In the scrolling text zone, I used cnxsoft twitter feed, retrieved it, parsed it and displayed it with QML XmlListModel and ListView:

Here’s what the Qt QML Digital Signage demo looks like when running in Qt Simulator (Nokia N900):

You can also download the source code (main.qml, ScrollingText.qml and global.js). You’ll need to create the video and pic directories in your project and add 5 videos and 5 pictures to test it.
The full QtQuick project is also available on Gitorious.

Qt Quick QML Digital Signage Demo Part 1

December 10th, 2011 No comments

I’ve recently started to play around with Qt and since I’d like to do a digital signage player running on Raspberry Pi, I’ve decided to try to make a simple digital signage demo application to evaluate the development platform.

Raspberry Pi Digital Signage

Screenshot of Digital Signage Demo Written with Qt Creator/QML

In Part 1, my goal was to make a 3 zones layout with a video zone, a picture zone and a scrolling text zone. I would just play one hard-coded media in each zone and the video and scrolling text would have to continuously loop.

I used Qt Creator to create  a “Pigital Signage” application (or should it be Πgital Signage ?).

To create the 3 zones I used the Gridview Element with 3 rectangles:

  • Video zone: 600×432
  • Picture zone: 200×432
  • Text zone: 800×48

Displaying the image is very easy with the Image Element:

The video playback was also supposed to be easy with the Video Element but it can not work on Desktop, so I had to revert to used the Nokia N900 emulator to be able to play a short video (hence the 800×480 resolution of the demo).  It works fine but there is a transparency issue with the emulator (in my PC), so if I want to clearly see the video I have to open a black picture and move the emulator on top. Here’s the video playback code:

The scrolling text should also have been easy since we can use  Webview  Element and html marquee, but I could not solve a problem with left and top margins that always left white border to the text zone.  Finally, I wrote the scrolling text code based on sample code provided by Kunal. I added a new file called ScrollingText.qml to the project:

The code is modified to start scrolling the text from the right until it fully disappears on the left and continuously repeat it.

Here’s the full main.qml with the code for the 3 zones:

In order for QtMultimediaKit to work properly, you also need to modify PigitalSignage.Pro since it is part of QtMobility:

Developing with Qt Creator and QML language seems relatively straightforward as this short demo could be written in one afternoon.

I’m not sure Qt 5 Video Element will be supported for the Raspberry Pi so QML Binding Element may have to be used to call an external video player that support R-Pi GPU.

In Part 2, I plan to list video and pictures file in a directory and play them continuously. For the scrolling text, the plan is to use a (twitter) RSS feed, scroll it and reload it each time the scroll is finished. I may also try to play online video (e.g. YouTube) and parse a config file to configure things like picture display duration, text color and scrolling speed etc…