When I asked Alexa earlier this week who was playing in the Super Bowl, she responded, somewhat monotonously, âSuper Bowl 49âs winner is New England Patriots.â
âCome on, thatâs last yearâs Super Bowl,â I said. âEven I can do better than that.â
Siri and Google Now, Meet Cortana
At the time, I was actually alone in my living room. I was talking to the virtual companion inside Amazonâs wireless speaker, Echo, which was released last June. Known as Alexa, she has gained raves from Silicon Valleyâs tech-obsessed digerati and has become one of the newest members of the virtual assistants club.
All the so-called Frightful Five tech behemoths â Apple, Microsoft, Amazon, Facebook and Alphabetâs Google â now offer virtual assistants, which handle tedious tasks in response to voice commands or keystrokes, on various devices. Appleâs Siri is the best known, having been available since 2011, but Microsoft now has Cortana, Facebook is testing one called M, and Google builds its voice assistant into its search apps.
These companies are presenting scorecards of their progress with quarterly earnings reports in the next few weeks, so what better time to hand out report cards to their artificially intelligent assistants? With that in mind, I set up tests for the assistants and graded their abilities to accomplish 16 tasks in categories that most consumers generally enjoy: music, productivity, travel and commuting, dining, entertainment and interests like sports.
In the end, none of the voice assistants earned a report card that would make a strict parent proud. Hereâs how they stacked up in terms of grade-point averages out of 4.0.
â¢ Googleâs Google â 3.1
â¢ Appleâs Siri â 2.9
â¢ Microsoftâs Cortana â 2.3
â¢ Amazonâs Alexa â 1.7
Apple was the strongest at productivity tasks like calendar appointments and email; Google was the best at travel and commute-related tasks. Alexa excelled at music, and Cortana was mediocre across the board. Facebook was left out of the grading system because the company denied access to M, though I did hang out with her for two hours on a friendâs account. More on that later.
Apple said that Siri had âbecome faster and smarterâ and was available in more languages than other assistants. Microsoft said that it was âjust scratching the surfaceâ with how Cortana could help people. Google said that it wanted smartphones to do more of the heavy lifting, and that users could do a host of things just by speaking to Google. Amazon did not respond to requests for comment.
On the productivity front, Appleâs Siri, which is summoned by pressing the home button on the iPhone or by saying, âHey, Siri,â was best able to schedule a calendar meeting with a friend in Hawaii, check what was on my calendar tomorrow, send an email and dictate my most recent email. Others could complete only some of those tasks â Google, for instance, could not read your last email out loud, and Alexa could not compose an email or create a calendar event.
Siri also fared well in music-related tasks, but was bested by Amazonâs Alexa. Both assistants could play the song âHeyâ by the Pixies, put on the latest episode of the Radiolab podcast and play music in the instrumentals genre. But Alexa, which can be summoned simply by saying âAlexa,â could play a specific music station on Pandora, whereas Siri could only open the Pandora app.
Google, which builds its voice-controlled assistant into the Google mobile app, achieved the highest marks for completing travel and commuting-related tasks. It responded perfectly to the question âWhat is the traffic like to 221 Main Street?â by showing me the amount of time it would take to drive there.
When I said, âTake me to the Dogpatch Boulders gym,â it showed me a map and gave voice directions. When I said, âFind me plane tickets to New York next week,â it offered an impressive response: Flights from San Francisco to New York next week start at $ 435, and the shortest flight is five hours and 10 minutes long.
On travel and commuting, Microsoftâs Cortana could offer solutions for the questions about traffic and directions, but not the one about flights. Siri earned a C-minus in the category: She could not give traffic estimates, and in response to the question about flights to New York, she spat out an unhelpful list of web search results related to traveling to New York. And instead of taking me to a bouldering gym where I could hone my physique, she took me to a place that could destroy my body: a brewery.
Alexa got a D â she could offer traffic estimates for only one fixed location that was set up inside the app, like your office, and she added the task of finding a flight to New York to my to-do list. (I gave Alexa a pass on failing to map me to the gym because it seemed like too much to ask from a home audio speaker.)
For food-related tasks, Google and Apple were even. Each of the assistants was able to find a list of nearby Indian restaurants. Only Googleâs voice assistant could order delivery food, but with an unintuitive process that required naming a specific restaurant that delivers food through one of the apps that Google has teamed up with. Siri was the only one capable of booking a restaurant table.
As for special interests, I asked each assistant two fairly obvious questions: Who won this past Sundayâs football games, and who will be playing at the Super Bowl?
Google, Cortana and Siri loaded scores for Sundayâs National Football League games. But only Google and Cortana could say the Carolina Panthers would face the Denver Broncos in the Super Bowl, whereas Siri could only say that the big game would take place on Feb. 7 at Leviâs Stadium in Santa Clara, Calif. Alexa, on the other hand, was as clueless about sports as I am: She couldnât answer either question.
That brings me to Facebookâs elusive M assistant. The social network denied my request to meet her â it has granted access to only a small number of testers â so I used a privileged friendâs Facebook Messenger account to meet M. According to the company, M is controlled partly by artificial intelligence and partly by humans; you talk to the assistant by sending messages to M through Facebookâs Messenger service, just as you would send messages to a friend.
In my limited time with M, I asked her to handle some of the most mundane tasks: Call the water company to ask about my utility bill, find out what meats are on sale at the local Whole Foods store and research when would be the cheapest time for my editor to fly to Hong Kong (not that I was trying to get rid of my editor).
M pondered for a few minutes before answering each question, which made me suspect that a person handled most of the tasks. So I asked M to schedule a photo shoot with a studio owned by a friend of mine. Within minutes, the photo studioâs phone rang, and my friend picked up.
âHi, Iâm calling on behalf of my boss,â said M, who sounded like a young woman. âHe wanted to find out if you guys have the ability for a photo shoot at 2 p.m. tomorrow.â
M left a contact phone number with a 650 area code, which includes Menlo Park, Calif., where Facebook is headquartered.
âI didnât catch your name,â my friend at the photo studio said.
âFirst name is M,â the not-so-virtual assistant replied. âLast name is Messenger.â
âIs that Greek?â my friend at the photo studio asked. M laughed nervously.
In other words, M probably is more capable than all the virtual assistants, but largely because humans are on the other end of the puppet strings handling tasks that artificial intelligence cannot. That makes me doubtful that many consumers will get to meet M, at least in its current state in which the service is free of charge.
âM is still in its very, very early stages,â a Facebook spokeswoman said in a statement. âWeâre not yet focused on scaling M to a large number of people.â