Microsoft's 'talking camera' app for the blind isn't as magical as it sounds
Microsoft is trying to prove its artificial intelligence can do more than guess your age or win at Pac-Man.
On Wednesday, the company released Seeing AI, a "talking camera" app for iOS meant to help the visually impaired navigate their surroundings.
SEE ALSO:Microsoft squeezed AI onto a Raspberry PiIts list of features sounds almost too good to be true: it will scan documents and read you their contents, tell you which denomination of bills you have in your hands, scan barcodes to let you know exactly what you're holding, learn the faces of people you know, read their expressions, and even attempt to describe whatever's in front of you at any given moment.
In practice though, it's clear that Microsoft still has a lot of work to do before Seeing AI will actually be a useful tool for the visually impaired. (The company is well aware of this, too: the first thing that appears when you launch the app is a warning that "Seeing AI is not always accurate.") Here's how it fared in our testing.
Text and barcodes
There are already plenty of apps out there that can competently analyze text and barcodes so it's not surprising that's what the app did best. It was able to read just about every snippet of text we threw at it and identified most barcodes easily.
While these capabilities are fairly common in other apps, Seeing AI's implementation is notable in that it assists you in lining up the in-app camera with whatever it is you're trying to scan. In the case of barcodes, it will beep as you get closer to the barcode and guide you with text to line up the camera with the edges of the document.
It's readouts of printed text were a bit awkward -- but accurate -- but it scanned nearly every barcode with ease (a jar of powdered peanut butter tripped it up for some reason).
People and "scenes"
This is where things get more interesting. Text and barcodes aside, Seeing AI uses a ton of Microsoft's AI algorithms to identify objects and people in your surroundings. These were the most impressive parts of Microsoft's early demos of the app, but things didn't go as smoothly in my testing.
I started off easy: my desk and a couch. The app easily recognized both, which maybe set my expectations too high because it only got worse from there.
Credit: seeinG AiCredit: Seeing AINext, I tried a stack of books, which the app misidentified as "a stack of flyers [sic]," and a mini fridge with some games on top. Seeing AI came back with "it seems to be floor, indoor, desk," which, though not technically wrong since all those items were technically present, didn't seem helpful had I actually been trying to figure out what was in front of me.
Next up, I tried the "people" feature. It purports to tell you not only how many people are around you but also estimate their age and emotional state (based on their facial expression).
It seems Microsoft has improved its age-guessing tech quite a bit since it first came out, but it still has some work to do. While the ages came up reasonably close (my colleague whose age was overestimated by five years may disagree), the expression detection seemed less reliable.
The feature is also unable to detect people who are not looking straight into the camera, which seems like a pretty serious limitation.
There were other fails as well: a cardboard cutout of Chewbacca was identified as "person standing in front of a mirror posing for the camera," which is both hilariously wrong and weirdly specific, while one of Kylo Ren came back as "it seems to be wall, indoor [sic]."
Of course, none of this is entirely unexpected. Apps like this require a huge amount of training data and algorithm tweaking before they can be anything close to reliable.
And while it's easy to laugh at more #AIfails, the reality is this type of technology really could be life changing for someone who's visually impaired if it indeed gives them the ability to navigate their surroundings more confidently.
But after spending some time trying out Seeing AI, it's clear Microsoft still has some work to do.
Featured Video For You
This app helps the blind "see" by identifying objects around them
TopicsArtificial IntelligenceMicrosoft
-
Courting DisasterKoreas hold daily liaison phone call after restoring communication linesAjax star Haller takes to Champions League in stylePrincess Eugenie's wedding dress sheds light on scoliosisWhy Kamala Harris triggers Donald Trump so intensely.[Newsmaker] ExNK paper calls grain production 'life2023广东农技“轻骑兵”十大优秀 “短视频+农技达人”名单发布评论丨农事运动会:一场农民的盛会、新农人风采展现的盛会、城乡双向奔赴的盛会快来参与 市花市树手绘活动火热征稿中
下一篇:First officially approved Gen IV nuclear reactor in the US breaks ground
- ·Abrar Ahmed returns as Pakistan names squad for second Test against Bangladesh
- ·Own an Echo Look? It stops working on July 24.
- ·Relatives, spouses of employers punishable for workplace harassment from October: Labor Ministry
- ·Govt. goes all out to ensure stable power supply amid heat wave
- ·South Korean lawmakers brace for US election as Harris, Trump diverge on North Korea
- ·Boeing's new VR simulator immerses astronauts in space training
- ·[Newsmaker] Ex
- ·务工期间放弃购买养老保险 解除合同后她将老板告上法庭
- ·[LLG] When compassion meets law: Lawyer defends goats, dogs, other helpless animals
- ·Snapchat stops promoting Trump's account in Discover
- ·我市将开展系列志愿服务活动
- ·Chelsea 'easiest game' Juve set to face, says Chiellini
- ·Best Labor Day headphones deals: Apple, Bose, Beats, and more on sale
- ·Military reports 3 more COVID
- ·退伍军人不褪色 搬迁一线讲政策
- ·North Korea fires 2 suspected ballistic missiles eastward: South Korea's military
- ·23 Peculiar Places of 2023
- ·North Korea fires 2 suspected ballistic missiles eastward: South Korea's military
- ·North Korea remains unresponsive to Seoul's message about border
- ·Trump wants a Putin summit. What could go wrong?
- ·让法治建设成效更加可感可及
- ·UEFA drops case against European Super League rebels
- ·Weekend border crosser thought to be North Korean defector: military
- ·New cases break record for 2nd day; toughest curbs likely to be extended
- ·Campbell highlights Washington Declaration amid security concerns over Putin
- ·Put these keywords in your YouTube video title if you want more views
- ·Amazon Android Days sale 2024: Save on unlocked phones, tablets, and more
- ·拓销路、亮品牌,佛冈魔芋走进广州北京路
- ·Barack Obama’s statement on child separation missed the moment.
- ·North Korea responds to US sanctions with two more missiles
- ·雨城区疾控中心提示:科学补碘 防治碘缺乏病
- ·Military reports 3 more COVID
- ·US, 5 others urge NK to end missile tests, return to dialogue: statement
- ·Govt. goes all out to ensure stable power supply amid heat wave
- ·Travel Back in Time and Uncover Old
- ·忽视新车磨合小心落下“病根”