With real-time translation of text common on the web and instantaneous speech-to-text gaining popularity, it seems that transliteration is cool again. But less obvious, and more difficult, methods of input are yet to be implemented. Case in point: sign language. The complicated and often contextual gestures form a vast visual vocabulary that isn’t easily captured or interpreted.
A team of British researchers, however, is making the attempt, creating a tool that translates a set of standard signs into readable text, in real time. It’s called the Portable Sign language Translator, and it should be out next year.
The signer would gesture as normal towards a camera on a phone or PC, and it would instantly translate based on a database of signs. Right now they are planning to support British Sign Language, but the system is perfectly capable of handling ASL, Makaton, and international languages and alphabets.
It is possible, however, that the static set of known symbols may still be limiting to signers, so the app will also allow the user to create their own signs for more complicated or personal objects.
The obvious application is for day-to-day communication between someone who cannot speak and someone who cannot understand sign language. But a visual, gestural language could be useful in other situations as well, and not just to people with disabilities. Multimodal communication is becoming the standard for interacting with our technology, and while heretofore we have communicated largely with inorganic tools, so to speak, such as the mouse and keyboard. Directly interacting with a machine that understands our voice, gestures, and position is going to produce extremely rich interaction methods in the future.
In the meantime, the app is being developed by Technabling, a company spun off from the University of Aberdeen. They plan to release it as a product next year, though there is no word of platforms or price. It is being funded by the UK’s Department for Business, Innovation and Skills and the Small Business Research Initiative.