Code Android Bluetooth and speech-recognition in Python
Darren Yates explores Python on Android to create a speech-controlled light using Arduino hardware, beginning with coding your Bluetooth-ready phone or tablet.
Artificial Intelligence or ‘AI’ is pretty big business at the moment. You have the tech giants promoting their speech-recognition standalone assistants, particularly the new Google Home, not to mention Amazon’s Echo, Apple’s soon-to-arrive HomePod and a raft of other alternatives we’re yet to see in Australia. Typically, you can ask these devices questions, get them to turn on and off network-connected lights — think of yourself as Tony Stark communicating with JARVIS to create a ‘smart’ home.
But if you have a late-model Android phone or tablet, about $15 worth of eBay-sourced electronics (which we cover in this month’s Arduino Masterclass, page 104) and some Python code, you can create your own speech-controlled wireless LED light, courtesy of Google’s existing speechrecognition tech inside Android.
WHAT YOU NEED
This project comes in two main parts — the Python code we’ll look at here that you run on your Android smartphone or tablet acting as your remote control, plus the Arduino-powered LED light project that you can read about elsewhere in this issue. But when these two come together, your voice becomes your light switch — OK, that sounds a bit try-hard, but it’s still pretty cool.
So to begin, you’ll need an Android device with at least Android 4.4 (KitKat), working built-in Bluetooth, a decent built-in microphone and either Wi-Fi or mobile network data plan (we’ll explain why as we go).
For software, you’ll need QPython3 ( tinyurl.com/y94pyp6p) and Bluetooth Finder ( tinyurl.com/nrufr4w), both free apps available on Google Play, along with our ‘voicelight.py’ source code available from the APC website ( apcmag.com/magstuff).
PYTHON ON ANDROID
We looked at QPython3 last month, a nicely-done Python 3 integrated development environment (IDE) for Android. What makes it worth
the download is its ability to not only run Python code on your Android device, but also tap into exotic Android features such as text-to-speech (TTS), speech recognition and Bluetooth wireless connectivity.
We’ve tested QPython3 on a range of Android releases from 4.0.3 through to 7.0 and it works really well. The only place you might strike trouble is getting it to run on x86 hardware — we tried it on the latest Android-x86 7.1 RC1 release with no success.
SCRIPTING LAYER FOR ANDROID
This added ability to tap into Android goodies comes from a programming layer inside QPython3 called ‘Scripting Layer for Android’ or ‘SL4A’. Originally developed by Google, SL4A has been added to QPython3. SL4A’s original goal was to allow scripting languages like Python and Ruby to use Android features without needing to fight your way around the Java programming language to make them work. It potentially makes using these features easy, although, as we found, not everything works exactly as expected. That said, our project uses three key Android features via SL4A — speech recognition, text-to-speech (TTS) and Bluetooth.
To use SL4A in Python, you begin by just importing the SL4A library into your code and creating an ‘Android’ object using the commands: import sl4a droid = sl4a.Android()
The first line imports the sl4a code library, the second creates an SL4A Android object called ‘droid’.
ANDROID SPEECHRECOGNIZER
Speech recognition has been baked into Android for a long time now — ever since Android 2.2/Froyo. The functionality itself is available to Java developers through an Android class called ‘SpeechRecognizer’. Thankfully, you don’t need to know anything about how it works, other than one brilliantly simple SL4A command: answer = droid. recognizeSpeech().result
This one line initiates the Android speech recognition function by launching the phone dialog interface, records your speech and returns the speech as plain text to the ‘answer’ variable. If SpeechRecognizer doesn’t hear you saying anything, it’ll even pause and notify you to tap the screen when you’re ready to voice your command. For one line of code, t hat’s about as easy as speech recognition gets.
However, in the background, there’s even more going on — once the SpeechRecognizer class has captured your speech, it must send that audio back to Google to turn into plain text. Once that’s done, the plain text is returned back to your Android device. We’ve tested this with Wi-Fi and mobile network data and it works either way. Without this internet connection, though, this form of speech recognition just doesn’t work. Phones don’t have the necessary RAM to handle the algorithms required for detecting speech and the language libraries, so sending the recorded speech to Google is a necessity.
TEXT-TO-SPEECH
But if you’re going to have speech recognition in your app, you can’t not have text-to-speech going back the other way. Android has supported TTS since the very beginning (well, at least since Donut/1.6), but in more recent Android releases, it’s received plenty of love from Google with improvements such as variation in expressiveness
“Phones don’t have the necessary RAM to handle the algorithms required for detecting speech and the language libraries, so sending the recorded speech to Google is a necessity.”
and a greater variety of voices — even an Australian voice.
As we mentioned earlier, TTS comes to our Python code through SL4A and again, it’s about as simple to use as you can get: droid.ttsSpeak(“hello, my name is python”)
TTS is initiated by calling the ‘ttsSpeak()’ function as part of the ‘droid’ object and the only parameter you need to add is the text string you want your device to say. Simple. The one thing we’ve found is that you don’t quite get enough of a delay from a comma during TTS, so for a more dramatic pause, try a full-stop (period) instead. You also don’t have to worry about capitalising your text — if you don’t say it normally, the TTS engine won’t. Spelling, however, is important — TTS will say exactly what’s in your string, but if it doesn’t recognise the word, you may get some unexpected results.
BLUETOOTH
The real key to this project is being able to tap into your Android device’s Bluetooth transceiver. Now the various online SL4A reference guides suggest you should be able to code the command: returnValue = droid. bluetoothConnect()
Do this and Android should ask you to select from the paired Bluetooth devices you already have available. The problem is it didn’t do that for us. In fact, it didn’t do anything. The only way we could make this work is to use the more-complex command:
retVal = droid. bluetoothConnect(“00001101 -0000-1000-8000-00805F9B34FB ”,”04:12:00:14:00:53”)
The first string of characters looks like nonsense but it’s actually a 128-bit universally unique identifier (UUID) for serial port profile (SPP) service used by Bluetooth modules like the HC-06 in our Arduino project. In other words, think of it as a universal phone number for dialling up Bluetooth serial connectivity functions. The second string is the 48-bit media access control (MAC) address of the HC-06 module as it’s paired to your Android device. The first string is always the same, however, the second string has to match the HC-06’s MAC address and it’s not always easy to find.
That’s where Bluetooth Finder on Google Play ( tinyurl.com/nrufr4w) fits the bill. This free app locates nearby Bluetooth devices, gives you their signal strength and, importantly for us, their MAC address. Grab that MAC address for the HC-06 Bluetooth module and type it into the ‘mac_address’ variable in the voicelight.py source code.
There’s one other thing this command does and that’s to wait or ‘ block’ further code execution until either you get a connection success or fail condition. With a successful connection, your code can now continue with a connected Bluetooth module.
CONNECTING VS PAIRING
In terms of coding, it’s important to remember that connecting is not the same as pairing — just because a Bluetooth device is paired with your Android phone doesn’t mean it has an active connection ready to go. To throw in a fairly dodgy analogy, pairing is like giving your phone number to your mate — that alone doesn’t give you a phone connection. Your mate still has to dial the number and you accept the call before you can have a conversation.
What’s even more impressive about the bluetoothConnect() command is that if your Android device isn’t first paired with the Bluetooth module before that command is executed, it’ll automatically bring up the pairing/ password window and have you fill in the password (typically ‘0000’ or ‘1234’ in this case) before continuing —
“The real key to this project is being able to tap into your Android device’s Bluetooth transceiver.”
kudos to the developers who figured that all out.
GETTING AND USING THE CODE
You’ll find the Python source code for this project on the APC website ( apcmag.com/magstuff). Unzip it, copy the ‘voicelight.py’ file to your phone and into the ‘/qpython/scripts3’ subfolder. Make sure your Arduino project is powered up and the HC-06 module’s red LED is flashing away. Pair your phone with the HC-06 module, then launch QPython3 on your phone, select ‘Programs’ from the main screen and load ‘voicelight.py’ from the list. On the little pop-up window, select ‘open’, change the ‘mac_address’ string variable to match the MAC address of your HC-06 Bluetooth module you recorded from Bluetooth Finder.
Once you’ve done that, save (press the disk icon) and press the triangle ‘run’ icon at the bottom of the screen and the code should launch. Once a connection is made, the HC-06’s LED should stay lit. Press the back arrow on your phone at any time to end the code.
THROWING YOUR VOICE
As soon as everything is set, you’ll get the Google speech recognition dialog user interface appear on your Android device. We’ll assume you’ve read this month’s Arduino Masterclass, built the Bluetooth LED light project and have it up and running — otherwise, nothing much else will happen.
Now you’re ready for your big moment — speak the command into your phone to turn the LED light on or off. The way we’ve designed the Python code is that it doesn’t matter really what you say into your phone at this point — it’s just looking for two keywords: ‘on’ and ‘off’. If you say either one in your speech, the code picks up on this and then sends the appropriate command wirelessly to the HC-06 Bluetooth module in the Arduino project. So you could try being cute and say: “Jarvis, turn the lights on, thanks, mate!” and show off to your friends, or you could go for word-economy and just say: “On!” and get the same result. You could even code in your own secret keywords — ‘apple’ for ‘on’ and ‘cats’ for ‘off’ — whatever, that’s up to you.
The key thing is that Google sends back the plain text of whatever you said and it’s stored in the ‘answer’ variable in our Python code. We then use the ‘in’ string search function to look for the keywords and execute the appropriate action from there. Once that’s been done, we loop around again over the while-loop, waiting for the next audible input. If that input is ‘stop’, the code will disengage the Bluetooth connection and give you audible cue to that effect.
DATA CHARGES AND LIMITATIONS
The one thing to be aware of is that speech recognition requires the Android ‘SpeechRecognizer’ class and according to Android documentation, “The implementation of this API is likely to stream audio to remote servers to perform speech recognition. As such this API is not intended to be used for continuous recognition, which would consume a significant amount of battery and bandwidth.” ( tinyurl.com/zpn78bn). So we’ve got two implications here — first, the more you say, the more it will cost you in bandwidth, and if using your phone network data plan, watch that usage carefully. But second, the service is in some way time-limited. Google doesn’t say how it’s limited; however, some users have reported a 60-second time-limit exists for each speech ‘session’. After that time, the speech recognition dialog interface closes.
One thing we’ve observed in practice is that, if the speech interface doesn’t hear any speech, it’ll revert to the ‘tap and speak’ dialog and just wait until you’re ready, which is handy as it provides its own code-blocking, rather than you having to code it.
CREATING SHORTCUTS
A cool little extra is the ability to create shortcuts for your Python code. While you’re in the ‘scripts’ window of the QPython3 app, press and hold one of the Python script files and the ‘create shortcut’ dialog box should appear, allowing you to create an app launcher icon for your apps list. Drag the shortcut to your home screen and now you can run your Python app code without having to pre-launch QPython3.
YOUR IMAGINATION
Turning on an LED by voice using Bluetooth might not float everyone’s boat, but once you have the mechanism for switching digital I/O wirelessly by voice, you can use it to switch on or off just about anything with the right electronics and a good dose of imagination. Python wasn’t designed to create standalone Android apps, but it’s still a fun way to learn coding and learn just how versatile the Python language can be.