CNXSoft – Embedded Software Development

UBM releases a study of the embedded market every year, by surveying over 1,000 embedded professional every year. They’ve just published their 2013 Embedded Market Study (85 pages report), after surveying over 2,000 engineers and managers, so let’s see whether anything has evolved in the software development and processor space compared to 2012.

Again this year, most respondents are based in the US (62%), followed by Europe (20%), and Asia (12%).

C/C++ languages still rule the embedded world with 81% market share, although a little less than last year (85%), assembler is a distant third (5%).

Interestingly, the average size of development teams seems to have shrunk from 15.9 in 2012 to 14.6 in 2013, the average project being composed of 4 software engineers, 2.9 hardware engineers, 2.7 firmware engineers, 2 QA/Test engineers, 1.5 system integrators, and 1.5 with other functions.

About a third of project last less than 6 months, another between 6-month and a year, and the last third over a year, with project lasting 12.5 months on average.

Embedded Projects Duration in Month

Projects life cycle breakdown shows much more time is spent detailed design stage, and a bit less on testing and debugging. Another interesting point is that only 1% of the time is now spent on documentation,  so either people gives even less of a damn about documentation than the previous years, or everybody has become highly efficient at writing docs.

Percentage of Time Spent on Different Stages of Development

The trends continues, as less and less Embedded projects use commercial operating systems, and companies opt more and more for open source operating systems, and in 2013, the survey shows that most projects are likely to use an open-source OS without commercial support. The top 3 reasons for choosing a commercial OS is real-time capability, good software tools, and processor & hardware compatibility.

Type of Operating System used in Current (left) and Next (right) Project

AS with previous years, the operating system choice is mainly influenced by software engineering staff & managers (62%), but it’s still sometimes decided by corporate management (17%). The top three reasons for OS selection are a little different from last year, as real-time performance, although still important (29%),  has been kicked out:

• Full source code availability (42% in 2013 vs 41% in 2012)
• Availability of tech support (34% in 2013 vs 30% in 2012)
• No royalties (30% in 2013 vs 31% in 2012).

54% of respondents are currently using or consider using embedded Linux in their next project because it is low cost (66%) and easily adaptable and extensible (52%), but 47%, a bit more than last year, would not consider embedded Linux mainly because they don’t need it (72%), but also because of software/app/driver incompatibility (18%)  and performance/real-time capabilities (15%). Legal reasons seem to have become less important, as only 6% consider it an issue with  embedded Linux, compared to 14% last year.

The enthusiasm for Android in embedded systems seems to have faded a bit compared to last year (28% vs 34%), FreeRTOS is still very much used, and 19% companies still rely only their own OS. Micrium and QNX have declined the most compared to last year.

Overall there has been very little changes and Linux share, including Android, is still strong with about 60%, compared to last year 70%. I’m a bit confused with the chart above, as most OS seem to be stable, or have lost market share, but only In-house OS and Keil have gained slightly.

Most embedded projects still use 32-bit processors (63%), followed by 14% using 16-bit version, 12% 8-bit and 8% for 64-bit. Both 8-bit and and 16-bit usage is declining, 32-bit usage is stable compared to last year, and 64-bit usage is on the rise. Most embedded projects still have a main processor clock rate below 250 MHz, the average frequency is creeping up (485 MHz in 2013 vs 425 Mhz in 2012), and 1 and 2+ GHz processors are on the rise.

Frequency of Main Processor in Current Embedded Project

When asked to cite one and only one vendor with the best ecosystem for the needs of the respondent, Texas Instruments still gets the lead (19%), followed by Freescale (12%) and Microchip (10%) in just the same order as last year.

According to the points above in UBM survey, there hasn’t been drastic changes between 2012 and 2013, but there are still some long and slow trends over several years, such as the increased share of open source operating systems, and the predominance of 32-bit processors.

Yesterday, Linaro announced the released of the IKS (In-kernel switcher) implementation for big.LITTLE processor which allows the SoC to switch between individual Cortex A7 or Cortex A15 cores to optimize power consumption. Currently, the only consumer device using supporting big.LITTLE the Samsung Galaxy S4 thanks to Samsung Exynos 5 Octa featuring 4 ARM Cortex A7 and 4 ARM Cortex A15 cores. IKS implementation can only make use of 4 cores at a time in this processor, since it must choose between A7 or A15 depending on the load. HMP (Heteregenous Multi-Processing) implementation is currently worked on in order to be able to use all 8 cores, and distributes tasks to the right core for the job. You can read my previous post for differences between IKS and HMP.

Linaro explains the current source still needs a few more modifications before being upstream to mainline. The code was developed for and tested on the VExpress TC2 development platform, so if you want to try it on another big.LITTLE processor, an MCPM backend (Multi-cluster Power Management) and possibly a special cpufreq clock driver are required.

The switcher is comprised of 4 parts:

1. Low-level power management – Responsible for powering up and down individual CPUs, enabling and disabling caches, power planes, etc.
2. Switcher core – Handles the switching process itself, including the saving of the execution context for the outbound CPU, migration of interrupts from the outbound CPU to the inbound CPU, and restoration of the execution state on the inbound CPU.
3. The core cpufreq layer – Standard Linux cpufreq subsystem augmented with a special driver that provides an adaptation layer between that subsystem and the switcher core code described above.
4. The cpufreq policy governors -  Those are made of kernel modules or user space daemons monitoring the system activity and requesting CPU frequency changes to the cpufreq core.

You can get the IKS kernel with git:

git clone git://git.linaro.org/landing-teams/working/arm/kernel.git

It seems the most important part of the source code can be found in drivers/cpufreq/arm_big_little.c, arch/arm/common/bL_switcher.c, arch/arm/common/bL_entry.c, and arch/arm/common/bL_head.S.

b.L Code specific to TC2, more exactly Dual Cluster System Control Block (DCSCB),  can be found in arch/arm/mach-vexpress/dcscb.c and arch/arm/mach-vexpress/dcscb_setup.S. [Update: Based on the comment below,  TC2 code is actually located in arch/arm/mach-vexpress/tc2_pm.c, and arch/arm/mach-vexpress/tc2_pm_setup.S.]

You may want to refer to the following resources for further details about the implementation:

• Porting documentation
• Switcher code description (slightly out of date)

Linus Torvalds has announced the release of Linux Kernel 3.9:

So the last week was much quieter than the preceding ones, which makes me suspect that one reason -rc7 was bigger than I liked was that people were gaming the system and had timed some of their pull requests for just before the release, explaining why -rc7 was big enough that I didn’t  actually want to do a final release last week. Please don’t do that.

Anyway. Whatever the reason, this week has been very quiet, which makes me much more comfortable doing the final 3.9 release, so I guess the last -rc8 ended up working. Because not only aren’t there very many commits here, even the ones that made it really are tiny and not pretty obscure and not very interesting.

Also, this obviously means that the merge window is open. I won’t be merging anything today, but if you start sending me your pull requests (Konrad already sent in his Xen pull request for the 3.10 merge window a week ago), tomorrow the flood gates start opening..

Linux 3.8 brought file systems enhancement for Btrfs, XFS and ext-4, introduce F2FS file system, memory management improvement, and the removal for i386 support.

Linux 3.9 brings the following key changes (Sources H-Online and Phoronix):

• File-system improvements:
  • Btrfs has experimental RAID5/6 support and fsync performance improvements.
  • EXT-4 bug fixes for corruption issue, and JBD2 journaling layer issue which affected performance.
  • Various improvements for F2FS file-system.
• Added ability to use SSDs as hard-disk cache.
• Update the latest LZO compression implementation within the kernel. Decompression and compression performance has been massively improved and x86 and ARM targets.
• Improved power management, including “lightweight suspend” (aka “suspend freeze”) mode.
• Improved ARM SoC support
  • Added Nvidia Tegra 4 support including support for Dalmore and Pluto development boards.
  • Added Nvidia Tegra 3 Beaver Board support
  • Kernel-based Virtual Machine (KVM) support for ARMv7 (Cortex A15 required)
• Mainlining of Google’s Goldfish virtual CPU.
• Initial ARC Linux support. See commit.
• Added support for Imagination Meta ATP (Meta 1) and HTP (Meta 2)
• Graphics drivers updates – Nouveau, the open-source reverse-engineered NVIDIA Linux graphics driver, is faster for some Linux OpenGL games. There’s also some Intel OpenGL performance changes.
• Support for Intel 7000 Wi-Fi components supporting 802.11ac.
• CONFIG_EXPERIMENTAL kernel configuration option has been removed

More details about Linux 3.9 will be available on Kernelnewbies.org (which is currently down).

MK802-II is an Android 4.0 mini PC powered by AllWinner A10 (ARM Cortex A8) with 1GB RAM and 4GB flash. Instructions are also available to run Ubuntu, or other Linux distributions. ARM Development Studio 5 (ARM DS-5) is software development tool suite for ARM processors that can be used for both Linux and Android debugging, and available in 2 versions: professional edition and community edition, the latter being free of charge. I’m writing about both today, because Bob Peng, Technical Marking Engineer for ARM China, recently wrote a blog post in Chinese [Update: An English version is now available] showing how to use MK802-II, preloaded with the required drivers and daemon, with DS-5 Streamline Performance Analyzer with is part of both versions. The community edition may be missing some features of Streamline however.

Streamline Performance Analyzer allows you to:

• Find out which modules or functions to take up most of the CPU, in order for you to optimize the affected code to speed up code execution speed.
• Monitor power, current and voltage in real-time, so that you can optimize the CPU tasks to achieve the best performance at the lowest power consumption.
• Analyze and optimize ARM Mali GPU in real-time, monitor CPU & GPU cache usage, and system memory usage.

Before using DS-5 Streamline Performance Analyzer, you normally need to setup your hardware development environment, and manually compile DS-5 gator driver for Streamline daemon. If you’re just an application developer, it can be a pain because you don’t usually care about low level stuff like compiling the kernel and modules, and your application is (mostly) hardware independent. That’s why, the ARM team in China, as chosen MK802-II as as low cost (about $35) development platform, and pre-built all you need to get going with DS-5 Streamline.

The trick seems you need to buy Rikomagic MK802II mini PC from a specific Aliexpress shop, and ask them to install DS-5 gator driver during the purchase, and you’ll also find ARM DS5 Streamline starting guide and USB driver in /home/ARM-DS-5 directory according to the shop. Sorry, I could not find a link to download the binaries and USB driver (USBDriver_MK802ii.zip) if you already have such device… Let me know if it’s available somewhere. I assume they decided to proceed this way to avoid having to handle different firmware versions, and if you want to use an existing device you should be able to build this yourself.

The instructions to configure the TV Dongle and your computer (Windows, Linux or Mac), install ARM DS-5 (PE or CE) are explained in details in the post, but I’ll skip that, and write directly about the things you can do with Streamline and MK802-II.

Once all is configured properly, you can start ARM DS-5 to analyze your application:

1. Timeline view – CPU performance analysis
   The part you want for the CPU usage analysis in the screenshot above is the linux Scheduler, branch, and clock section, but this view also include GPU vertex and fragment and Disk IO, and other GPU parts shown in the figure below.
2. Timeline View – Mali GPU Performance Analysis
   
   Scrolling down the Timeline View, you’ll find more details about GPU usage such as geometry statistics and GPU vertex processor activity, but you’ll also get information about memory usage, interrupts and more.
3. Call Paths View
   The call paths view is supposed to show how functions are called, but since they just used the Android Browser for testing it mainly shows the libraries which have been called and the CPU usage.
4. Functions view
   Again, the example above is not optimal because they used a release binary, but it still shows you which libraries takes the most CPU usage.

If some cases you may want to know if a library or binary built for the ARM architecture is using hard-float (armhf) or soft-float (armel). You can analyze ELF binary using readefl utility, so let’s have a try.

First let’s install some armel and armhf files on a computer running Ubuntu by install gcc/g++ toolchain for armel and armhf:

apt-get install g++-arm-linux-gnueabi  g++-arm-linux-gnueabihf

We now have armhf and armel libraries installed in /usr/arm-linux-gnueabihf/lib and /usr/arm-linux-gnueabi/lib respectively. Let’s check the output of readelf filtered with “FP” string for libm.so.6 for armel:

readelf -a /usr/arm-linux-gnueabi/lib/libm.so.6  | grep FP

Tag_FP_arch: VFPv3-D16
Tag_ABI_FP_denormal: Needed
Tag_ABI_FP_exceptions: Needed
Tag_ABI_FP_number_model: IEEE 754
Tag_ABI_HardFP_use: SP and DP

and armhf:

readelf -a /usr/arm-linux-gnueabihf/lib/libm.so.6 | grep FP

Tag_FP_arch: VFPv3-D16
Tag_ABI_FP_denormal: Needed
Tag_ABI_FP_exceptions: Needed
Tag_ABI_FP_number_model: IEEE 754
Tag_ABI_HardFP_use: SP and DP
Tag_ABI_VFP_args: VFP registers

Great, so there’s an extra line for armhf (Tag_ABI_VFP_args) that seems to confirm the library is hard-float.

With readelf compiled from elftoolchain-0.6.1 (source code), the extra line will be a bit different: “Tag_ABI_VFP_args: AAPCS (VFP variant)

AAPCS stands for ARM Architecture Procedure Call Standard. You can read more details on ARM website. There are also two other possible values for Tag_ABI_VFP_args: “AAPCS (base variant)” and “toolchain-specific”, but I’m not sure in which case they may be returned.

“-a” will just dump everything, but readelf also allows you to only select architecture depend information with “-A”. So beside floating point type, you can also find out about the architecture, thumb, whether it’s build with NEON support and more:

readelf -A /media/usb1/system/lib/libEGL.so
Attribute Section: aeabi
File Attributes
Tag_CPU_name: "ARM v7"
Tag_CPU_arch: v7
Tag_CPU_arch_profile: Application
Tag_ARM_ISA_use: Yes
Tag_THUMB_ISA_use: Thumb-2
Tag_FP_arch: VFPv3
Tag_Advanced_SIMD_arch: NEONv1
Tag_ABI_PCS_wchar_t: 4
Tag_ABI_FP_denormal: Needed
Tag_ABI_FP_exceptions: Needed
Tag_ABI_FP_number_model: IEEE 754
Tag_ABI_align_needed: 8-byte
Tag_ABI_enum_size: int
Tag_ABI_HardFP_use: SP and DP
Tag_ABI_optimization_goals: Aggressive Speed

Energy Micro has recently announced the availability of the Keil RTX real-time operating system (RTOS) as part of its Simplicity Studio tool suite. The Keil RTX has been optimized for ARM Cortex-M processors to provide flexible scheduling and high-speed operation, and has been improved to enable an ultra-low power, deep-sleep mode between tasks.

Energy Micro has implemented a tickless mode in Keil RTX. This mode allows the EFM32 Cortex M3 MCUs to wake up only when needed, either at a scheduled time or on an interrupt event. This results in much lower power consumption in sleep mode compared to other SysTick implementations.

The video below shows the difference between SYSTICK mode where the system consumes about 2.7mA in active mode, and 830 uA in sleep mode to the new tickless mode where the MCU still consume the same amount of energy in Active mode, but only 1uA in sleep mode, or in other words, an 830x improvement in power consumption.

ARM Keil RTX RTOS is available for free (as in beer), and is also provided with full source code under a BSD license that is free of charge even for commercial usage. The Keil RTX implements the RTOS API of the ARM Cortex Microcontroller Software Interface Standard (CMSIS). Windows users can download this RTOS via Simplicity Studio software suite, however it can’t be retrieved from Simplicity Studio in Linux (or I missed it), but you can still download the source code, documentation and samples from Energy Micro in a zip file.

Via Embedded.com

One of the main issues with Linux on ARM is the lack of proper GPU drivers for the platform, as most silicon manufacturers now only focus on Android drivers which are not compatible with the Linux kernel, because Android is based on Bionic C library, whereas Linux is based on glibc or its variants. There are two ways to solve this issue:

• Open Source GPU drivers. This would be the ideal solution, as you would just be able to cross-compile the drivers for the proper, as well as fix bugs without having to ask the silicon manufacturer to fix the driver for you.
• Bionic to Glibc library. Such library acts like libdl, and allows to load Bionic library and overrides some symbols from bionic with glibc based ones. This is exactly what libhybris does.

Ubuntu Touch is capable of using Android GPU drivers to run Ubuntu thanks to this library, and Libhybris will provides the interface between the HAL and Mir display server for Ubuntu 14.04.

Libhybris is released under an Apache 2.0 license, and the source code, written in C, is available in github. This library is based on the initial work of Carsten Munk.

There are 3 main directories in the source tree:

• compat
  • ui – Some wrapper for Android UI.
• hybris
  • common -  Common parts, including parsing /system/build.prop and matching common C functions (e.g. pthread*, str*…) between Bionic and glibc
  • egl – EGL support
  • gles2 – OpenGL ES 2 support
  • hardware – Loads /system/lib/libhardware.so
  • tests – Test programs for egl, gles2, lights, sensors, offscreen rendering, and ui
  • ui – Loads /system/lib/libui_compat_layer.so and creates bindings for the related functions.
• utils -  Contains load_sym_files.py utility that loads symbol files for debugging.

Libhybris not only let you access the GPU, but also other hardware such as lights and sensors. I assume, but I may be wrong, that it could also be used with Wi-Fi and Bluetooth closed source drivers. Hopefully, work can also be done for VPU (Video Processing Unit) support in order to take advantage of hardware video decoding and encoding in Linux with Android libs, but I’m not knowledgeable enough to know if it’s feasible or not.

Using Libhybris is not an optimal solution, as there may be a small performance degradation, and you’ll be using the Android kernel, so many of the things you’d got used to with the Linux kernel won’t be available, and you’ll have to use the “Android” way. The implementation is not straightforward either, as Canonical’s Ubuntu Touch preview makes use of SurfaceFlinger which means applications relying on X11, GTK… will not work at the beginning, and you’ll have to wait until 2014 and and implementation with Mir display server to get a full solution based on libhybris with X11 compatibility. Nevertheless, even though libhybris is not a perfect solution, it’s a practical one, as it’s a technical solution, and not one which relies on convincing management and lawyers to release open source drivers.

There’s also some work on reverse-engineered open source GPU drivers such as the lima project, and some companies, e.g. Freescale / Vivante, release proper (closed-source) drivers for ARM Linux, but the company with the highest market share, namely Imagination Technologies, does not participate in any of those initiatives, and libhybris could become the way of the future, so most people may end-up using Android kernels on their ARM “Linux” distribution. Comments are welcome!