Akiko Ishii was recommended to attend an employment training program, in which she hoped to learn how to use a computer and work in an office. Instead, they advised her to study for therapeutic massage, because this would be a more suitable profession for someone like her … who is blind. But Ishii didn’t agree at all. After going blind as a result of a surgical operation in her 30s, she was determined to achieve her goals. This endeavor paid off when he landed a job in the HR and labor relations department of an IT firm in Tokyo. Now she and her husband have a company that teaches classes in different subjects, from yoga to practical knowledge of technology, to people who are blind or have low vision.
Ishii leads a full life and in recent years everything has been easier with his “assistant”, which is what he calls Seeing AI, Microsoft’s free application for iOS aimed at people who are blind or have low vision. This camera app uses “channels,” or modes, to read printed text, recognize the value of banknotes, as well as describe physical objects, product labels, and colors, among other things.
Be able to read, without seeing
“Now I will be able to read in Japanese. And I will also be able to read the mail and other communications and correspondence on paper; it’s going to be exciting, ”says Ishii. She is keen to use Seeing AI’s short text channel, which reads text aloud as soon as it appears on camera, as well as the document channel for longer printed text articles. As she herself explains, before, “if a communication was on paper, I had to ask someone to read it to me. Instead, now I can read it myself. “
Seeing AI launched in English in 2017, and is available in 70 countries. The app is based on artificial intelligence (AI) technology and has so far helped people complete more than 20 million tasks. It has been developed based on an inclusive design and tests are carried out by people with disabilities through non-profit organizations around the world. Seeing AI arrived in Spain last December. Microsoft announced its availability, free, for iOS mobiles.
Document recognition
Seeing AI includes functionalities such as reading short texts through the mobile phone camera; recognition of documents and identification of its different parts; emission of small beeps to facilitate the location of the bar codes and their subsequent scanning to identify the products in packages, or the information on expiration dates and preparation recommendations; recognition of people from the user’s usual environment (including estimated age and mood), as well as identification of family, friends or co-workers; determination of the value of banknotes and currencies, including euros, dollars and pounds, among others; o Recognition of the environment, such as objects, people, animals …
Cristian Sainz uses Seeing AI at home to scan the barcode on a jar of peaches in syrup from the fridge. Sainz is 50 years old and lost his sight in a car accident at 18. Currently, he is responsible for ONCE’s digital workplace unit. A cook and former restaurant owner, Sainz notes that one of the features he likes most about Seeing AI is that you can use it to find out the exact measurements you need when cooking. It has a scale that calls out the weights of food, but only in 5-gram intervals. “There are recipes where I need one gram, not five. Now, I can get that information in a few seconds. I just have to point the phone with the Seeing AI short text channel at the scale screen and the app reads it out loud. I’ve been able to start using recipes that I couldn’t make before ”.
Browse Photos
At the beginning of last year, a Seeing AI function developed at the request of users was added: the possibility to explore family photos or social networks by touch and hear descriptions of the images. With this function, “it is very easy to know what information the photos contain,” confirms Sainz. “I slide my finger across the screen and the app describes where the objects are in a scene.”
Also Florian Beijers from the Netherlands agrees with the usefulness of Seeing AI. To date, the photographs held little meaning for this 27-year-old user, who was born blind. “Now, in a way, I can ‘see’ where each thing is in the image, what else appears in it, the relationships between the different elements,” he corroborates. “The geek in me appreciates having that information.” Although most people understand that “blind people no longer stay at home doing nothing,” explains Beijers, “I keep coming across companies that still think that blind people don’t use computers.”
Facilitate everyday tasks
As he himself indicates, having Seeing AI in Dutch is going to drastically change your life, because it will make your daily tasks easier in a country where accessibility is not as adequate as it should be. According to him, in restaurants the menus are not always accessible. “There is not always a web version that can be consulted on the phone.” For Beijers, the Seeing AI document channel, which offers audio guidance to capture a printed page and recognize the original text and format, “is extremely useful, especially for reading the menu in a restaurant.”
Ishii, our first protagonist, regularly uses the app’s people and scene channels to recognize and describe the people around her, including her three-year-old daughter, Ami. “I use this feature to tell me what you’re doing,” confirms Ishii. Or if Ami asks her mom for chocolate, Ishii can go to the kitchen and use the app to “read the barcode on the chocolate to find out exactly which one the little girl wants.” This kind of independence is priceless, he observes. “One of the things I love about Seeing AI is that once you have this feature-rich app, you can use it for anything,” says Ishii. “Seeing AI is my assistant and provides me with the support I need so that I can focus on taking care of my daughter.”
