Native Code Overivew

This is designed to give you an overivew of the Native Code and integrations that are used throughout the accesibility extensions, and explain how, and why, we use them (Although, I’d like to thing some of it its fairly self explanatory!)

What platforms utilise native code?

Currently, due to Unity platform limitations, most, if not all of the current platforms that we support and work with (Android, iOS, MacOS etc.) use some degree of native code to enable certain elements of functionality that are simply not possible within Unity. The key bit of fucntionality enabled by these elements of native code is Text-to-Speech and speech synthesis, something with Unity is not aware of, yet capable of interfacing with, itself. Alongside this, we also use native code to detect things like the Locale of the users device, and whether they’ve got accessibility services enabled. I have plans to use native code in the future to provide things such as haptic feedback for devices that support it.

Structure and Layout

We utilise and follow Unity’s guidelines for creating plugins where possible. Following the /Plugins/ folder structure where required. Currently, the plugins we use are spread across multiple different scripts and files, however, there is the potential for this to be consolidated and tidied up in the future (However, not now, as having things slightly spread and disconnected like this makes it easier to debug).

A note on macOS

Whilst it was initially planned that I’d use native code, to tie into the native NSSpeechSynthesizer - However, this has been slightly difficult to achieve, and has preseented more problems than the TTS implementation on iOS did.

Instead, we use the C# System.Diagnostics.Process functionality, to trigger off system processes. In this case, this works well, as macOS has a built in say command, which can be triggered via the terminal.

There are also additional plans to implement a picker for the various voices availible on a users device, to enable developers to test out how their strings sound in different locales, currently, this has been a p.i.t.a. due to serialization fun.

A further note on Windows

Initially, I had planned to use System.Speech.Synthesis in C#, however, I found that Untiy and Mono don’t include this functionality, and importing and including the DLLs seems to be a massive pain, with inconsistent results.

Looking for alternatives, I found a wrapper for the native C++ Speech API, created by Chad Weisshaar (this can be found here). I did look to build upon this, by exposing some additional functionality that native functionality has on other platforms (e.g. rate/speed of speech, pitch control), however found that I in the time frame I set myself, I couldn’t get this to work. Also, limited knowledge of SAPI, C++ and C, seemed to get in the way.

It’s also worth noting that the Windows native code is currently 64-Bit only. Chad does provide source for rebuilding to 32-Bit though, meaning it’s possible to build and recompile this if required.

Native Plugin Overview

As mentioned above, both iOS and Android utilise native plugins to provide text-to-speech functionality. We also use tiny bits of native code to detect locale of the device, as well as the status of accessibility services (e.g. whether the user has accessibility services enabled, or not).

Windows platforms currently use native code, but to a limited extent, with simple TTS relying on the system default settings being the only supported functionality.

Each bit of native code is hooked up and contained within a couple of scripts, that expose simple APIs that can be used to do things such as speak out a string, or return the devices location value, or the status of the native text-to-speech engine.

Native code can be found in the following areas:

  • TTS.cs
    • All native code is contained within one string, which uses ifdefs to call upon the appropriate code, and return a ICU locale value, which gets passed on where needed.
  • TextToSpeech.cs
    • Native code is spread across a couple of different functions within this script, to handle setting up the text-to-speech engine where needed, actually speaking out the appropriate strings, and stopping the speech where needed.
  • WindowsVoice.cs
    • This provides native TTS under windows, native functions from the C++ wrapper/plugin for the speech API, are passed through, and exposed in here.
  • OSXSaySpeech.cs
    • This handles the say command, and command execution and lifespan on macOS.