Getting Started with ReSpeaker WiFi IoT Board’s Audio Capabilities, Voice Recognition and Synthesis

ReSpeaker is a development board combining an Atmel AVR MCU, a MediaTek MT7688 WiFi module running OpenWrt, a built-in microphone, an audio jack, and I/O headers to allow for voice control and output for IoT applications. That means you could make your own Amazon Echo like device with the board and add-ons, use it as a voice controlled home automation gateway and more. The board was launched on Kickstarter a few days ago, and already raised $100,000 from about 100 backers, but I’ve received an early sample, so I’ll provide some more information about the firmware, and shows how to use with some Python scripts leveraging Microsoft Bing Speech API.

You’ll need a micro USB to USB cable to connect your to computer (Linux, Windows, Mac OS…), and a speaker to connect to the board. Linux (OpenWrt) boots in a few seconds, and once it’s done all RGB LED will continuously blink.

I’m using a computer running Ubuntu 16.04, and ReSpeaker is detected by the system as an Arduino Leonardo board:

That’s optional, but if you want you can access the serial console, with programs like Minicom, screen, putty or hyperterminal and set the connection to 57600 8N1 to access the command. Here’s the full boot log:

If you think something is odd here… That’s because the serial connection will miss some characters. This happens with two computers and different USB cables. Hopefully this is either a specific issue with my sample, or if it is an issue it will be fixed by the time boards ship to Kickstarter backers [Update: The company explained me that it’s because the Atmel 32u4 and Mediatek MT7688 share the same USB port]. So instead of using the serial console, I’ll use SSH instead which means I have to connect to ReSpeaker WiFi access point first, and configure it.

ReSpeaker will show as LinkIt_Smart_7688_XXXXX, because the WiFi module is exactly the same as LinkIt Smart 7688 IoT board, and unsurprisingly the configuration interface is exactly the same.First set the root password, and login with that password.

Then go to Network tab, select station mode, and connect to your access point by entering your password. Click Configure, and you’re done. As you can see on the right above, you can also use OpenWrt’s LUCI interface to configure networking.

Now find ReSpeaker IP address via your Router DHCP client list, arp-scan, or other method:

You can now connect to the board via SSH:

and use the password you set in the web interface.

Now let’s check some CPU information:

We’ve got Mediatek NT7688 MIPS24K processor as advertised, so let’s check a few more details:

The board runs Linux 3.18.23, has 7.6MB available storage, and 128MB RAM in total.

I’m not going to test the audio features with command tools, and python script, and also include a video demo at the end of this review.Since I don’t have ReSpeaker Microphone array add-on, I have to be fairly close to the microphone for it to work well, maybe one meter at most, or the volume would be really low.

I’ll start by checking audio recording and playback with any API or internet access requirements.
We can record audio with 16000 sample rate, 16 bit width, 1 channel using the following command

and play it back with aplay:

It worked OK for me, although the volume seemed quite low.

Now we can do something a little more interested as Seeed Studio develop a few Text-to-speech and Speech-to-text Python scripts. You can retrieve the scripts from ReSpeaker github account, and install one dependencies to setup the board:

The script are using Microsoft Speech API, but in theory you could use any other speech API. Since Seeed Studio has already done all the hard work, I simply applied for a Microsoft peech API key in order to be able to use the demo.

That’s free for testing / evaluation, but if you intend to use it in commercial products, or for your own case, if you use more 5,000 transactions per month, you’d need to purchase a subscription.

You’ll find three Python scripts in the directory namely: bing_voice.py, bing_stt_with_vad.py,  tts.py. Look for BING_KEY inside each script, and paste your own key.

Time to have some fun, starting with the speech to text script:

It’s pretty slow to start (about 15 seconds), and then there are a few error message, before you can see the “* recording” message, and you can talk, with Bing returning the results: “Bing:你好”. Chinese? Yep, as currently the default is Chinese, but if it is not your strongest language, you can edit bing_stt_with_vad.py, and change the language replacing zh-CN by en-US, or other language strings:

An English works too (sort of):

In the first sentence, I said “Hello World! Welcome to CNX Software today”, but it came out as “hello world next software”, maybe because of my accent, but I doubt it…

Then I wanted to try Thai language, but I got an API failure simply because the number of supported languages by Microsoft Speeach API is limited as shown in the table below.

If your language is not listed here, then you could Google Speech API instead, and it’s likely Seeed Studio or the community will have written compatible scripts by the time ReSpeaker boards ship to backers.

So you now know how to convert your voice to text, and you can use that text to send a web search, or toggle GPIOs, but you may also want to get an audio answer to your action, and tts.py script is there for your, and very easy to use:

It did not really feel realistic, but at least I could understand the female voice in the speakers. Looks in the script I did not see any language settings, so I assume the API will automatically detect the language, and inputted a string in French instead, and all I heard was gibberish. Finally I found that you can change the voice language in bing_voice.py script with contains most of the code:

I replaced the US female voice, but a French male voice, added a “famous French saying”:

At least it was understandable, but Microsoft has still some work to do the audio output was more like “Salut mon gars. commencer a va?”. The reason could also be that the correct writing is “Comment ça va”, but the terminal (set to UTF-8), did not let me input “ç”.

You can watch all those demo in the video below to get a better feel about the audio quality, delays, and capabilities of Microsoft Bing Speech API.

Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress