Mixspace Lexicon is a developer toolkit for creating natural language command interfaces in Unity. It is specifically tailored for use in augmented reality, virtual reality, and mixed reality.
Lexicon leverages two services from IBM Watson: Speech to Text and Assistant. Together these services provide real-time natural language processing. They work on nearly any platform that has a microphone and an internet connection, making Lexicon a cross-platform XR input solution.
A lexicon is “the vocabulary of a person, language, or branch of knowledge.” To build the vocabulary of your application you use intents and entities. By providing sample phrases Watson can extrapolate and understand a wide range of similar phrases. This allows for natural, flexible input from users.
With Mixspace Lexicon, you can design your intents and entities directly in the Unity editor. You can push your design to the Watson cloud with a couple clicks. You can bind meaning to your entities in the Inspector. You can easily parse a user’s intent, even multi-part intents, with a few lines of code.
Lexicon also gives you the ability to train custom language models for speech to text. Watson custom language models are a powerful tool to improve speech transcription for a particular domain. All the data used to build your intents and entities can also be used to build a custom language model, with a single click. So go ahead and use those domain-specific words, Lexicon will know what you mean.
Most importantly for XR, Mixspace Lexicon lines up each spoken word with the user’s focus in the world (real or virtual). This adds a layer of context to voice commands that hasn’t been available up until now. Lexicon keeps a history of focus data, allowing you to glance at objects and fire off commands in real time.
The intents that make up your lexicon are meant to be reusable. Mix and match them in a workspace, share them with others, or grab one from our growing collection of experiments. Start building your own collection of intents and you’ll have a flexible toolbox that can be dropped into any application. To use a tool, simply say the words.
We are so excited about starting this journey with you. Let’s create the future together.