Learning Your Next Language In Virtual Reality
ATi Studios recently released the first virtual reality app for language education to combine the artificial intelligence technology behind chatbots with speech recognition in virtual reality. Learn Languages VR by Mondly allows people to experience life like conversations with virtual characters, in 28 different languages, from the comfort of home.
The new VR application is the first to realize virtual reality’s promise of rich, immersive and automated educational experiences. The app offers instant feedback on pronunciation, suggestions that enrich learner’s vocabulary and surprises that transform learning a language into a fun experience.
Bridging The Technologies Behind Chatbots, Speech Recognition, and VR
Mondly’s conversational voice chatbot for language learning was first released by the company in August 2016. The technology uses advanced speech recognition that understands millions of phrases and words in 28 different languages. Since its release, it has been used by many users and has learned millions of new phrases and responses, thanks to the chatbot’s self learning technology.
[Tweet “”The toughest challenge was to make the speech interaction feel natural in VR.””]
To bridge the technologies behind its chatbot, speech recognition, and VR, the company had to develop a new automatic voice detection system. Alexandru Iliescu, the CEO and co-founder of ATi Studios, spoke about the challenges involved with creating Mondly VR:
“The toughest challenge was to make the speech interaction feel natural in VR. Because VR is so immersive, we quickly realized that traditional speech interaction models that require a tap or a voice command to enable speech recognition would kill the flow of the experience. So we developed our own automatic voice detection system. It calibrates to the room’s background noise, and it ignores unintentional sounds. The result is amazing, the conversations with the virtual characters happen as naturally as real life conversations. They just flow.”
Virtual Reality Language Training In 28 Different Languages
Learn Languages VR by Mondly allows users to practice conversing in a new language with virtual characters; all within realistic situations that teach language fundamentals and help build their confidence.
[Tweet “”The conversations with the virtual characters happen as naturally as real life conversations.””]
The first version offers three scenes and 28 languages to explore:
- The Restaurant – Practice ordering food and drinks in a small bistro.
- The Eurotrain – Have a conversation and make new friends on a train in Europe.
- The Hotel – Check into a boutique hotel and score a room with a view.
Having learned from the usage patterns of over 15 million Mondly users to date, ATi studios has created the VR application based on what users actually need to effectively learn a new language: Instant feedback and an immersive, real world experience to build their confidence.
“Fluency in a language is not how many words you know, but how well you communicate with the words you do” said Alexandru Iliescu. “Our goal with Mondly is to push the envelope of language learning with new technologies that drive our users to become better communicators and have richer life experiences. And the Oculus team and the virtual reality technology they’ve created offered us the perfect environment for achieving this goal. ”
Full list of languages: British English, American English, German, French, Italian, Spanish, Russian, Romanian, Greek, Portuguese, Arabic, Croatian, Czech, Danish, Dutch, Finish, Hebrew, Hindi, Hungarian, Japanese, Korean, Norwegian, Polish, Swedish, Turkish, Ukrainian, Vietnamese, and Thai.
