Google Assistant Is The Smartest AI And Siri Trails All According To Recent Study
If you have ever used the AI personal assistants baked into smartphones, you know they aren't always that helpful. Research firm Stone Temple has issued an update for 2018 on its Digital Personal Assistants accuracy study. The study tested 4,952 queries on five different devices including Alexa on Amazon Echo, Cortana on the Invoke speaker, Google Assistant on Google Home, Google Assistant on an Android smartphone, and Siri.
The test structure asked the 4,952 questions to each personal assistant. In testing, the team recorded if the assistant answered verbally, if the answer was from a database like Knowledge Graph, if the answer was from a third-party like Wikipedia, how often the assistant didn’t understand the question, and when the device tried to answer but got the answer wrong.
As you can imagine, with so many queries tossed at five devices, the study is long and complex to read. Thankfully, Stone Temple gave us some short answers that are easy to absorb. The chart below shows the breakdown of scores for each device. In short, the smartest of the assistants was Google Assistant on Smartphone and the worst was Siri.
Compared to the results from 2017, the study found that all four personal assistants have increased the number of attempted answers and the biggest change was from Alexa. Comparing queries that were fully and correctly answered between 2017 and 2018 was interesting. Most voice assistants were worse this year than last with the only increase in accuracy between years being from Cortana Invoke.
Alexa attempted to answer more queries in 2018 than it did in 2017, but it got more of them wrong. The study does note that many of the errors seen from Alexa and Siri came from poorly structured or obscure queries like "What movies
