In his book The Republic , Plato presents the famous allegory of the cave. According to this allegory (which is not a myth as is often claimed), a group of prisoners have lived inside a cave since their birth. The chains that hold them prisoner are placed in such a way that they can only look at the back wall of the cavern, without any possibility of looking elsewhere. There are other men, who walk behind and carry objects, but cannot see them. A strategically placed bonfire casts the shadows of the walking men on the only wall that the prisoners can see. Therefore, these prisoners think that the shadows are the truth, since they have no more information. But without a prisoner being released and seeing the external world, they would see a new, deeper and expanded reality. When this prisoner returned to the cave to tell what happened, they would take him for crazy and could even kill him. Unless they used a camera with a low latency time of flight at five frames per second, to generate images without line of sight.
The term “Non Line Sight” (NLOS) or “No Line of Sight” is used to designate the reception of a radio signal that does not follow the direct path from the object to the receiver. This occurs when there is an object blocking the signal, such as trees, buildings, or mountains. Line of sight, by definition, is determined by the ability of the average human eye to resolve a distant object. While optical wavelengths are around 400-700nm, radio wavelengths range from 1mm to 30km. The shortest radio wavelength is about 2000 times longer than the largest optical wavelength. Therefore, visual and radio obstructions are not comparable. However, the scientists behind the article have managed to see behind objects, in the same way that radio waves have been doing for some time.
The idea is appealing, but it seems like something from science fiction: being able to see a scene that is hidden behind an object. How? Projecting its shadow in a certain way. Green light is reflected on a wall and the image is reconstructed using the time-of-flight technique on a computer. However, until now it took several minutes to reconstruct a static scene. The breakthrough for Nam and the rest of the team has been to achieve real-time scenes with five frames per second. Although the idea seems simple, this technique requires many ultra-fast light sensors, with very high sensitivity and not so easy video reconstruction algorithms that can work in the time lapse between two frames. The algorithm further improved by one of the co-authors, PhD student Brandt. The slowness of the previous algorithms used in NLOS was partly due to the fact that each sensor only had one pixel.
To achieve real-time video the team needed single photon avalanche diodes (SPAD), something that is now more common and even found in the latest iPhones. They have the ability to detect individual photons, that is, they are ideal for picking up very weak signals. Scientists from Wisconsin-Madison worked together with colleagues from Italy to refine SPAD far beyond the commercial ones, 50 times slower. They have reached such a point that they have been able to detect depth, which translates into reconstruction of 3D images. When we project on the wall different shades of light and shadow, although it may be very subtle. This sensitivity in the amount of light is what they have achieved with their improved 28-pixel SPADs.
The videos that have turned out so far are monochrome and fuzzy, but reflect movement and three-dimensional geometry. It may even disappoint, but let’s think about the first few photos for a moment. The first that we keep is View from the Gras window , made by Niepce in 1826. It could also disappoint at the time, unless those who held it knew they were making history and it was only the beginning. The exposure time was eight hours. The Wisconsin-Milan team has been able to verify with the recorded videos that letters can be resolved or human limbs in motion can be distinguished. In one of the videos distributed and that you can see on this page, Nam is hidden behind a cloth, like a screen. He has a stuffed animal in his hand and he tosses it up and down. SPAD detects movement, shape and depth in real time, even though the image is fuzzy.
Beyond looking like a Star Trek movie or a fashion series, the applications can be very varied. They could even save lives. For example, disaster relief. Imagine places where people are trapped and cannot be accessed with the eye, it would be great to be able to reconstruct the image using this technique. Or for autonomous vehicles, which would know what is behind a corner before reaching it. Professor Andreas Valten is the last author of the article, but also the main promoter of the technique. He has been working for a decade with NLOS, the aforementioned technique that consists of recording the echoes of the bounces of light on a surface. In Valten’s words: “It’s basically echolocation, but using additional echoes, like reverb.” These investigations, of course, the fruit of many looks. The work has been funded by the United States Department of Defense Advanced Research Projects Agency (DARPA) and the National Science Foundation. From now on, the idea is to improve the technique and the multiple applications that it will have in the future will remain in the imagination and creativity.
Artículo original: Nam, J.H., Brandt, E., Bauer, S. et al. Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second, Nature Communincations 12, 6526 (2021).
