FAST COMPANY: Are you ready for bots to read your face?

Story by Sean Captain | Robot Revolution | Fast Company | 11.15.17

 [Screenshot: Autodesk/SoulMachines] [Screenshot: Autodesk/SoulMachines™]

Would you turn on your webcam so that a customer service robot can get to know you better?

Soul Machines™, a New Zealand startup, thinks so. It builds a customer service bot with an amazingly human face and a simulated nervous system that interprets how customers feel and reacts accordingly—in part by watching them over a webcam. As I reported today, design software maker Autodesk will be the first big client to try out the technology next year—what Soul Machines™ calls a “digital human”—with a remake of its AVA customer service bot. (Below, a video about the creation of AVA.)

While many companies boast about personalized service but can’t afford to hire enough people (and while most existing customer service bots haven’t exactly managed to fill in the gap), Soul Machines™ sees an opportunity for new CGI and AI-driven techniques. AVA’s photorealistic appearance—based on scans and recordings of actress Shushila Takao—is an outgrowth of the work the company’s founder, Mark Sagar, has done as a CGI engineer for Hollywood films like Avatar; her “emotional intelligence,” guiding how she responds to human cues, comes from his AI research simulating a human nervous system in software.

AVA, says Sagar, can read cues in people’s expressions, such as a smile or a furrowed brow, to get a sense of a customer’s disposition. Are they happy and open for pleasant chat, or impatient and wanting to get to the point? Or are they just freaked out by a robot staring at them, reading their expressions and possibly classifying their emotions?

“I think that is going to be the biggest challenge of it,” Gregg Spratto, Autodesk’s VP of operations, told me about the concerns that Autodesk’s privacy-conscious users might have about a face-scanning chatbot. “And I think a lot of our success will be dependent on how we convince people to give that a try.”

AVA also provides a fallback: she’s designed to pick up nuances in people’s voices too. So even with the webcam off, she can still get a better feel, so to speak, for the human she’s talking to.

NATIONAL BUSINESS REVIEW: Soul Machines™ partners with Autodesk to create digital employee

Excerpt from story by Rebecca Howard | NBR |November 16, 2017

Soul machines™, the Auckland-based developer of intelligent, emotionally responsive avatars, has joined forces with 3D design, engineering and entertainment software developer Autodesk to create the US company’s first digital employee.

AVA (Autodesk virtual agent), a virtual customer service agent, will interact with customers 24/7 to resolve any issues. However, unlike Autodesk’s current virtual assistant that is text-based, its virtual nervous system will give AVA the ability to see, hear and respond emotionally, according to Soul Machines™.

“This is where our technology is particularly unique. These digital characters do have a virtual nervous system, using AI technology and models of different parts of the human brain. That’s how we bring them to life in a very human-like way,” said Greg Cross, chief business officer for Soul Machines™ in an interview.

To read the full story click here.

COMPUTERWORLD: Soul Machines™ gives Autodesk chatbot a human face

Soul Machines™ says the humanised AVA will enable customers to get answers to questions direct them to content and enable them to complete transactions

Story by Stuart Corner (Computerworld New Zealand) 16 November 2017

Soul Machines™, a spinout from the University of Auckland Bioengineering Institute, is developing digital human interface to Autodesk’s customer assistance chatbot, the Autodesk Virtual Agent, AVA.

Soul Machines™ says the humanised AVA will enable customers to get answers to questions direct them to content and enable them to complete transactions.

“Soul Machines™ is advancing AVA’s capabilities, with a digital human face and persona that it literally brings AVA to life using [our] world leading Human Computing Engine (HCE),” the company said.

However to take full advantage of humanoid AVA, Autodesk customers wil need to turn on the video on the phone or computer so AVA can see them,

HCE is described as a virtual nervous system that combines neural networks and biologically inspired models of the human brain that will give AVA “the ability to see and hear as well as sensory systems that enable to recognise and respond emotionally in an incredibly human like way.”

Soul Machines™’ CBO Greg Cross said: “Talking to one of our digital humans means you will get the same sort of social responses and non-verbal communication cues as if you were sitting face to face across a table from a real person. It means our customers can deliver highly personalised brand accretive experiences in a way they have not been able to afford to do up till now.”

CEO of Soul Machines™ Dr Mark Sagar said “[Ava] has a virtual nervous system and all kinds of sensory capabilities so she can respond to the user’s behaviour in real time to facilitate the communication.”

For the full story click here

HOT OFF THE PRESS: Soul Machines™ Partners with Autodesk to launch AVA at Autodesk University 2017

Soul Machines™ today announced it has partnered with Autodesk, a builder of software that helps people imagine, design and create a better world, to create a digital human of Autodesk’s Virtual Agent, AVA. Announced at Autodesk University, Autodesk’s annual conference in Las Vegas, Ava will provide customers access to a service agent 24/7—answering questions, directing them to content and completing transactions.

Soul Machines™ is advancing AVA’s capabilities, with a digital human face and persona that it literally brings AVA “to life” using its world leading Human Computing Engine™ (HCE). The Soul Machines™ HCE  is a Virtual Nervous System™ that combines neural networks and biologically inspired models of the human brain. AVA’s Virtual Nervous System gives her the ability to see and hear as well as sensory systems that enable to recognize and respond emotionally in an incredibly human like way.

“The future of highly personalized customer engagement is a blend of digital and human support, and AVA our first digital employee is just that. She will understand human consciousness and interactions by reading signals such as body language and facial reactions—in turn learning more about customers to better serve them. The addition of emotional intelligence to AVA takes our customer service beyond purely transactional to relational.” – Rachael Rekart, senior manager for, machine assisted service engagement at Autodesk

By putting a face on Artificial Intelligence and extending the potential for customer interaction beyond text based chatbots and voice assistants we enable large corporations and organizations all over the world to enter a whole new era of personalized service and democratized knowledge transfer.

“As one of the world’s leading global software companies we are excited that Autodesk will be among the first to deploy our technology on top of their IBM Watson cognitive platform. If the future of software is AI and 3D graphics this is an extremely exciting partnership for both Autodesk and Soul Machines™” – Greg Cross, Chief Business Officer, Soul Machines™

As we work with Autodesk to launch their first digital employee AVA we are very excited  to be exploring the future of human machine interaction at the very beginning of the AI and robot era. Learning about how we can make these “machines” more human like  and more useful to each and everyone of us as individuals will define the future of creating compelling customer experiences.