Trying out Edge Impulse machine learning platform on XIAO BLE Sense board

I had seen the Edge Impulse development platform for machine learning on edge devices being used by several boards, but I hadn’t had an opportunity to try it out so far. So when Seeed Studio asked me whether I’d be interested to test the nRF52840-powered XIAO BLE Sense board, I thought it might be a good idea to review it with Edge Impulse as I had seen a motion/gesture recognition demo on the board.

It was quite a challenge as it took me four months to complete the review from the time Seeed Studio first contacted me, mostly due to poor communications from DHL causing the first boards to go to customs’ heaven, then wasting time with some of the worse instructions I had seen in a long time (now fixed), and other reviews getting in the way. But I finally managed to get it working (sort of), so let’s have a look.

XIAO BLE (Sense) and OLED display unboxing

Since the gesture recognition demo used an OLED display, I also asked for it and I received the XIAO BLE board (without sensor), the XIAO BLE Sense board, and the Grove OLED Display 0.66″.

Seeed XIAO BLE OLED display

Both boards are extremely tiny, and exactly the same except the XIAO BLE lacks the LSM6DS3TR onboard 6-axis IMU (bottom left side).

Xiao BLE Sense vs Xiao BLE
Xiao BLE Sense (Top) vs Xiao BLE (Bottom)

Some soldering…

Before loading firmware on the board, I had to solder the display to the board. I just cut the Grove cable and soldered the black and red wires to the power, and the white and yellow wires to I2C.

Solder OLED display to Xiao BLE Sense
I do not have a 3D printer (another customs story, but I digress), so instead I used several layers of dual-sided sticky tape to attach the two boards together, but that’s really optional.double-sided sticky tape boards

Arduino sketches for XIAO BLE Sense’s OLED display and accelerometer

It took me a while to find the instructions for the gesture recognition demo, as those were not listed in the video description, and I could not find anything about it in Seeed Studio wiki. Eventually I was given the link to the instructions, and the company modified the website to make it easier to find.

Before playing with Edge Impulse, we’ll run two Arduino sketches to check the OLED display and the accelerometer work as expected.

The first step is to add the board manager URL for Seeed Studio boards: https://files.seeedstudio.com/arduino/package_seeeduino_boards_index.json

arduino board manager seeeduino

Now we can install the package to support  Seeed nRF52 Boards…
Seeed nRF52 boards Arduino IDE
… and once it’s done, let’s connect the board with a USB-C cable to our computer, and select “Seeed XIAO BLE Sense – nRF52840” board with default settings.

Seeed XIAO BLE Sense Arduino

Let’s try a “Hello World” program to make sure our board works and the connection to the OLED display is fine.


But I did not work as expected, and I got an error at compile time:


The answer is on Adafruit website where we learn two important details:

  • nRF52 requires Arduino 1.8.15 or greater, so you may have to upgrade to the latest version
  • Linux requires the installation of adafruit-nrfutil

Since I’m using Ubuntu 20.04, I had to run:


Note this is NOT needed if you are using the Arduino IDE in Windows or MacOS. The utility was installed in $HOME/.local/bin, so you’ll need to add it to your path, and restart the Arduino IDE. This can be done temporarily in the command line:


Or you can change /etc/environment or ~/.bashrc file for permanently adding the folder to your PATH. The sketch could build fine, and the binary was flashed to the board without issue.

XIAO BLE Sense flash Arduino firmware

But nothing showed on the display. I added a serial.println debug message in the main loop to check it was indeed running, double-checked the connections with a multimeter, and I could not find any obvious solution. Seeed Studio told me to downgrade to version 1.0.0 of the Seeed nRF52 Boards package.

Seeed nRF52 boards 1.0.0And it indeed did the trick!

XIAO BLE Sense OLED Hello World

Note that you won’t need to downgrade to version 1.0.0 anymore in the rest of the review, and it is recommended you use 2.6.1 and greater. The new “Hello World” sample looks like this:


Time to switch to testing the accelerometer demo. First, we’ll need to install LSM6DS3 Arduino library from Seeed Studio.

Seeed Arduino LSM6DS3

Note there’s also an official Arduino_LSM6DS3 which you may need to uninstall to avoid conflicts. Here’s the code:


Note that when I used Seeed nRF52 Boards v1.0.0 with that sample, I would get “decode error” messages, which went away with version 2.6.1. We’ll need to open the Serial monitor to check X, Y, Z values are showing up.

XIAO BLE Sense accelerometer

Important: we’ll still need the accelerator demo running for the next step. So if you play with other samples first, make sure the accelerator demo is running before switching to Edge Impulse.

Edge Impulse on XIAO BLE Sense

Now that we know our hardware is working as expected, let’s go to Edge Impulse Studio to get started. Let’s register, and build our first project.

Edge Impulse Getting Started

We’ll select Accelerometer data.

Edge Impulse Project

We’ll also need to install Edge Impulse CLI in Linux (Ubuntu 20.04 for this review). This first requires installing NodeJS 14.x:


The default directory will be in /usr accessible by root, let’s change that to a user’s directory which we’ll also add to our path:


Now we can install Edge Impulse CLI:


You may have to exit the terminal and restart it for the new PATH to apply.  Now we can launch the edge-impulse-data-forwarder used for boards that are not officially supported by Edge Impulse (like XIAO BLE Sense):


The very first time the command is running, you’ll need to enter your username and password (not shown above). The utility scans serial devices, connects to edge impulse, tries to detect data from the serial port, and once done, asks us to name data fields (Ax, Ay, Az), the device (XIAO BLE SENSE), and will automatically attach it to the project we have just created. If there is more than one project in Edge Impulse, you’ll be asked to select the project first. That means it’s basically hardware agnostic, and as long as your board output accelerometer data to the serial interface it should work.

Edge Impulse Data Acquisition

Let’s go back to Edge Impulse, click on Data acquisition, and we’ll see our device together with sensor parameters and data frequency settings.
up-and-down 20s

Let’s set the sample size to 20,000 ms, define a label, click on Start Sampling, and move the board up and down with about 1 second internal for 20 seconds to acquire the data.

Edge Impulse data collection training

We’ll then need to split the data by clicking on the three dots in the raw section and choosing “Split Sample”. Click “+Add Segment” to add more section. We should repeat it until we’ve got around 20 segments representing the up and down movement.  If you moved slower or faster than 1s adjust the time in “Set segment length (ms.)”.

Split training segments 1s

I’m using Firefox, and I had a weird bug where I could add a segment, but when I selected it to move it, it would jump by an offset to the right, sometimes outside of the screen. But if I kept pressing on the mouse button, and moving it left, it could bring it back in view. It was not exactly the most convenient, and I can’t zoom in too much or the box will get too much out of the display. Using Chrome or Microsoft Edge with Edge Impulse might be better.

Once we click on Split and we’ll see the 1-second data sample we’ve selected. edge impulse split segments motion

We can repeat data acquisition and splitting for other gestures such as left and right, clockwise circle, and anti-clockwise circle. But I’d recommend keeping it simple first, as we’ll see below.

poor train/test split

At this point, you may see what looks like a problem with the data with the warning:

One or more of the labels in your dataset have a poor train/test split

To fix this, you can either capture test data for a shorter period (e.g. 2 seconds), or rebalance the dataset, by clicking on “Dashboard” in the left menu, and scroll down to find the “Perform train / test split” button.

Edge Impulse perform train/test split

We are now ready to create an impulse. Click on Create impulse -> Add a processing block -> Choose Spectral Analysis -> Add a learning block -> Choose Classification (Keras) -> Save Impulse

Create Impulse spectral analysis classification keras

Click on Spectral features in the Spectral Analysis, then Save parameters and Generate features.

Edge Impulse Spectral Features Generate features

You’d expect the data to be cleanly separated, but there’s clearly some overlap, so the trained data is not ideal. I’ll try to go ahead nevertheless.

Click NN Classifier, then “Start training” which will take about one minute to process. We can then choose Unoptimized (float32)

Edge Impulse NN Classifier Training

There’s indeed low accuracy and the model is basically unusable with only the “anti-clockwise” sample being detected properly.

edge impulse model testing

Let’s try again, but only with up and down, left and right, and circle (clockwise) and trying to keep each motion within one second.

Edge Impulse Features Explorer

The Features explorer chart looks much better with Blue, Orange, and Green dots in their own areas. It’s also possible to delete some samples that may cause issues. After training the results are not perfect, but we may still give it a try.

Edge Impulse training performance

Let’s go through “Model testing” in the left menu.

Edge Impulse model testing

That’s disappointing as only Circle clockwise works well, while “left and right” can be detected about half the time, and “up and down” being wrongly detected as left and right. So besides training the data, the operator must also be trained.

The important part is the method to create an Impulse, and over time we should e able to create better data. Let’s build an Arduino library by clicking on Deployment in the left menu, then Arduino LibraryBuild, and finally download the .ZIP file.

Edge Impulse Arduino Library

Back to the Arduino IDE. Download the Arduino sample provided by Seeed Studio, which has gone through many revisions over the last few months.  Change the edge impulse header file (line 24 in the sample below) to match your own, and I also had to comment out the line with U8X8lib.h library. I also slightly modified the code, as I had not trained for “idle” as in their demo:


We need to add the ZIP library we’ve just downloaded from Edge Impulse to the Arduino Library, and we can now build and flash the code to the board. This will take around 5 minutes the first time, and about 2 minutes for subsequent builds.

Here’s the output from the serial console.


So each time a value is over 50 it will show the corresponding text (e.g. Circle), and if none of the results goes above 50, the program simply displays “Idle”.

Here’s what it looks like on video.

The circle is recognized (even if I go in the wrong way as in the video below), but I only got “left and right” a few times, and never “up and down”. So it should take some time to have a proper demo with the important part being the data acquisition and accurate splitting to make sure all samples for a particular gesture look about the same.

I’d like to thank Seeed Studio for sending XIAO BLE (Sense) boards and Grove OLED display for testing Edge Impulse. I’d just wish they got their documentation right the first time. If you are interested in reproducing the demo above the XIAO BLE Sense board is sold for $15.99, while the OLED display goes for $5.50, and it’s optional because we can see the results in the serial terminal.

Share this:

Support CNX Software! Donate via cryptocurrencies or become a Patron on Patreon

ROCK Pi 4C Plus
Subscribe
Notify of
guest
The comment form collects your name, email and content to allow us keep track of the comments placed on the website. Please read and accept our website Terms and Privacy Policy to post a comment.
3 Comments
oldest
newest
David
David
6 months ago

FYI the the ten (10) Thai Baht coin shown for size comparison is 26 mm (1.02 inch) in diameter. See here:

https://en.wikipedia.org/wiki/Ten-baht_coin

And just for fun, at my writing time ten (10) Thai Baht is worth $0.29 USD:

https://duckduckgo.com/?q=10+Baht+in+USD&hps=1&ia=currency

David
David
6 months ago

“So it should take some time to have a proper demo with the important part being the data acquisition and accurate splitting to make sure all samples for a particular gesture look about the same.” I’ve read/watched a few of these simple Edge Impulse accelerometer “demos” now, and most of them produced finicky if not outright unreliable results. I think it’s an understatement when they say training ML is hard and time consuming. I get the feeling you need lots of training data and all of that requires manual editing to provide examples that reinforce the desired outcomes. Plus I… Read more »

Louis (Edge Impulse)
5 months ago

Hello David,

Indeed that is a fact with machine learning solutions in general, not only embedded machine learning. Collecting a good dataset is very tedious and time-consuming. We are trying to offer tools that make this process (and in general all the steps along the machine learning pipeline) smoother. Do not hesitate to reach me on Edge Impulse forum is you have any questions on how to perform better with your project, I’d be happy to help.

Regards,

Louis

Weller PCB manufacturer