We spend more hours alone with the phone and the computer than with any other human being. At this stage of events, very few will be in a position to discuss this fact. We sleep with the mobile next to the bed: checking it is the last thing we do before going to sleep and the first thing we do when we open our eyes. The American technology research company Unisys says that it takes an average person 26 hours to report the theft of a wallet. But if the lost object is a telephone, the complaint occurs after 68 minutes.
Laptops and smartphones are our constant companions. Before them we take off the mask, we do not pretend, we are what we are. It’s a shame they can’t measure up (for now). Because, no matter how many hours you spend with your laptop, he is unable to guess if you are sad and cannot react accordingly. His cognitive intelligence is staggering, but his emotional capacity is nil. Euphoric, melancholy or nervous, you will always have the same cold and intelligent machine in front of you. Despite this, this device is capable of keeping you absorbed and isolated from your friends and relatives: in fact, a good number of your social connections take place through your networks.
Experts, fed up with warning of the relative dehumanization that underlies digital friends and lovers, have sought an unexpected twist to the matter: returning emotions to the world of digital interactions. And the only way to do that is to teach machines to interpret emotions and react in a way similar to how a human being would .
The definitive signal is the destination that some capitalists on Sand Hill Road have recently given their money, that hill near Stanford University where those who invest in the future that you cannot even imagine are concentrated. Well, these great men have decided to invest for the first time in startups with cute names. Namely: Affectiva, eMotion, Realeyes, Sension or Emotient.
The subject was introduced in 1995 by Rosalind Picard, a professor at the Massachusetts Institute of Technology (MIT), and was then treated with sarcasm for suggesting that, to make a machine truly intelligent, you had to give it an emotional response. “Programmers must consider the perspective of affection when creating software intended to interact with humans,” he wrote. He then expanded on his theory in the book Affective Computing . And, unintentionally, he christened a new field in computer science. Picard has been the inspiration for many researchers, such as the Egyptian Rana el Kaliouby, who, with his collaboration, has created the Affdex algorithm, capable of identifying the expression of different emotions from facial features.
Affectiva, which came out of the MIT project, is the origin of Affdex and, like almost all companies focused on emotions, uses the research of psychologist Paul Ekman, who by the way has signed the startup Emotient, created by Marian, from the competition. Bartlett, a professor at the University of California.
Identical expressions all over the world
Ekman began his investigations of human emotions in the 1960s, and succeeded in creating a solid body of evidence showing that it is possible to identify six human emotions from facial expression , because their features are almost identical, regardless of gender, age or age. the cultural context. Thus he created a kind of 500-page encyclopedia that compiles all possible facial movements and is known as the Facial Action Coding System.
Based on this information, with the use of a specific algorithm and machine learning techniques, it has been possible for a machine to be able to differentiate an authentic smile from a social one –designed to please– or feigned pain from the real one. A line, an excessively furrowed brow, or marked tension in a muscle can serve to distinguish one emotion from another. The creators of Affdex ensure that its accuracy to interpret a frown is close to 90%.
“Our starting point is the human face,” explains its founder. “In the science of human emotions,” he continues, “each facial movement is a unit of action. For example, the pull of the corner of the lip of a big smile is the unit of action number twelve, the frown lines are number four ”. Humans are incredibly consistent and repetitive in the way we wrinkle our nose or eyebrows to express feelings. They are universal movements, and it is all a matter of training computers to detect the small changes in facial muscles that differentiate a frown frowning in disgust from one frowning in amazement.
According to El Kaliouby, getting a machine to read these changes is difficult, because they are fast and combined with each other. Tens of thousands of real smile faces of different ages, ethnicities and genders, and the same number of social smiles, are entered into the program to teach a computer to differentiate a genuine smile from a fake one. Using machine learning, the algorithm identifies the lines, folds and facial muscle changes typical of both ways of smiling.
“The machines are very good at collecting details from images. An almost imperceptible change is immediately detected ; even a variation in a single pixel. The challenge is to get the machine to identify, for example, the sadness in the wide variety of faces that exist in the human species, “explains Jay Turcot, a researcher at Affectiva. And in order for him to be able to learn it, he has to ignore the little individualities on each face that are not substantial. “For the machine to approach or even exceed our species’ ability to detect emotions, it has to have processed thousands upon thousands of different faces,” explains Turcot.
Database with 2.9 million faces
The researcher assures that training computers is as difficult and exciting as teaching a class full of students: some surprise you, others disappoint you, new ones arrive, new methodologies are experimented, the best are subjected to more refined tests … Affectiva has managed to compile 12,000 million emotional indicators from the study of 2.9 million faces of subjects from 75 countries. It is already the largest emotional database in the world.
Emotion analytics software does not have an easy task ahead of it. Until now nothing has surpassed the human eye to detect falsehood and imposture in the faces of our fellow men. Computers outperform us in logic, but not in perception tasks, such as voice recognition or object identification. As Bartlett confirms, “the processes of perception are easy for humans and difficult for machines.” However, in her San Diego lab, this machine learning expert has made computers outperform humans when it comes to detecting false expressions of pain. It is true that it has been achieved in the controlled environment of a laboratory and not in the confusing real world, but it is a first step.
Bartlett’s system is also based on Ekman’s Facial Action Coding System, which subdivides facial expressions into movements – wrinkling of the nose, twitching of the eyelid, etc. – that involve one or more muscles of the face.
The Emotient team has worked for years to create a visual system that automates this kind of encyclopedia of facial movements and has developed another algorithm capable of recognizing the facial patterns that correspond to each emotion .
Twenty-five volunteers were recruited in the experiment in question, who were recorded in two videos. In one of them their face was filmed while they endured real pain (they had an arm submerged in an ice cube for a minute). In the second, their face was captured pretending that something was hurting them (this time their arm was floating in a pleasant warm water). Both versions were shown to 170 people, who after seeing them several times made many mistakes, and only got a little more than half correct. Instead, the computer system detected the feigned pain with an accuracy of 85%. The results were published in the journal Current Biology .
Paul Saffo, a prestigious analyst and futurologist, predicts from his Stanford University classes that face-reading technology could be combined with programs that have other complementary modes of emotion recognition, such as software that analyzes human voices. “If computers reach the point where they can combine facial codes, voice sensors, and glancing and gesture tracking, we will interact with our machines in a more natural and less forced way, ” says Saffo. Can you imagine that one afternoon when you asked Siri – the personal assistant application – to look for the movie billboard, she would detect by your voice that you are sad and only read you the movies that are good for you? By Siri’s criteria, of course. Well, advertising creatives try to do something similar with what they call smart advertising.
A key advance for the blind and autistic
Much is said and little has been seen of what are supposed to be advertisements that will appear in your life at the right time, when you need them most. Advertising agencies are very interested in providing excitement to our digital interactions. It is no coincidence that Affectiva’s clients include Unilever, Mars and Coca-Cola. The company also contributed the know-how of its algorithm to design the order of advertising in the latest edition of the Super Bowl.
“Our technology records individual facial expressions while users view an advertisement. We analyze them moment by moment. We are currently in a position to measure what these expressions reveal to provide the client with a comprehensive analysis of the user’s emotional reactions to their ad ”, explains Gabi Zijderveld, Vice President of Affectiva. What several studies have already shown is that, as people get more excited, they also become more engaged and remember more clearly. This is of great value to advertising creatives.
For El Kaliouby the most surprising thing is that we are expressive all the time, even when we are sitting alone in front of our current devices, insensitive and unable to understand each other. It would be ideal that that mute witness could interpret and react to our moods, or know that if we have a bad day it is better that he does not go around with nonsense by hanging over and over again. In five years, El Kaliouby predicts, all devices will have a built-in emotional chip and will definitely become our best friends.
However, the merit does not lie in taming the home computer, but in that, when you go to do a management, let’s say to the Treasury, you come across an unknown computer and that, despite seeing it for the first time, can identify by your facial features a white lie or a feeling of distress, or reveal any emotion that you are interested in hiding.
Technology endowed with emotions could be very useful for blind people. Glasses with an emotional chip would emit information to the user about the faces in front of them. They could also help those with autism spectrum syndrome interpret emotional clues and thus improve their social life.
Educational apps could detect when the user is bored, and speed up their learning pace. In case they interpret that the student is confused, the most likely reaction would be to slow down the release of the content.
A car capable of identifying that you are tired and sleepy would never allow you to drive , and a refrigerator that sees you somewhat anxious would be blocked so that you do not have the typical midnight binge that the next day will produce heartburn and tons of feelings of guilt .
What does not seem fair to El Kaliouby is that we spend all day hanging from devices that pass from us. She experienced it in the first person when she got a scholarship at Cambridge and had to separate from her family. “I was every night on Skype, on email , sending messages from a phone or a computer that I had no idea how lonely I felt,” she says. Hence the idea of endowing technology with at least a dash of emotion. “Every time you pick up the phone, there is an emotional pulse, especially if you are waiting for something. In our research we have seen that people check their cell phones between ten and twelve times an hour. It is a shame that we cannot fully understand each other with them. “.
Perhaps the bright day will come when your phone is so smart that it prevents you from sending that night owl and feral wasap to your ex before you regret it forever. The phone will detect from your face that you are not having a good day. You will see it in your brow, or perhaps in the trembling of the right eyelid that appears when you’re stressed. It will read on the line of the corner of your upper lip that you feel confused. And period. That wasap will self-destruct in 3, 2, 1, 0.
Images: Aldebaran Robotics
