Hunting for the perfect selfie. A new series of algorithms developed by engineers from Google and the Massachusetts Institute of Technology (MIT) will revolutionize the world of mobile photography, as these algorithms allow photos to be processed and retouched in real time, even before pressing the button. shooting. Before processing a poor quality image, the photograph may be edited and retouched before going to our image archive.
This project leverages machine learning and neural networks trained on 5,000 images created by Adobe and MIT and retouched by five professional photographers, teaching software how to adjust an image to get its best look.
The real innovation in this case is to make the resulting algorithms efficient and fast enough to apply retouching (applying improvements in brightness, contrast or saturation) while we are still framing the photograph.
“This technology has the potential to be very useful for real-time image enhancement on mobile platforms,” says John Barron of Google.
Machine learning had been used before to teach computers to process images before, both by Google and other companies, but the sheer size of today’s smartphone images, plus the limited power of built-in hardware , makes real-time edits very difficult.
The key to real-time retouching
To avoid this, engineers developed algorithms that could perform image processing on a low-resolution version of the image – which enters through the camera’s viewfinder – and then magnify the photo results without spoiling the quality. by the way.
When converting low-resolution images to high-resolution, we typically see pixelated shapes, among other things, and the software has to do a lot of guesswork on how to fill in the missing details. To avoid this, the experts augmented the low-resolution images, leaving them not as actual images, but as formulas that could be applied to the high-resolution versions, expressing the changes through mathematics rather than as actual pixels.
Thus, each pixel in the final high resolution image has four formulas that combine to establish which color / contrast / saturation … is best suited for that pixel.
Compared to a machine learning system that uses full-resolution versions of photos throughout the process, this new approach uses only one-hundredth of memory.
Which means that the image you see on your camera screen while framing the photo can be processed very quickly, even while moving. Engineers say they can tweak the system to create different shooting styles that will be used for different purposes beyond our next Instagram photo. We can create really professional photos.
In addition to lightening dark spots and balancing contrast, for example, algorithms could even mimic the style of a particular photographer. And it is that these new algorithms adapt intelligently as we modify where our phone looks.
Barron says we can all look forward to “new, compelling, real-time photography experiences without draining the battery or experiencing latency.”
When will it be available on Android?
Referencia: Automatic image retouching on your phone. MIT News. Massachusetts Institute of Technology 2017.
Here’s how the photo algorithm works:
