Tech UPTechnologyThis is what the first intelligent vision sensors that...

This is what the first intelligent vision sensors that process with AI are like

For those in the field of sensors, Sony is like a ‘God’ who provides many others with this ability to see with their eyes. And a few days ago it announced with great fanfare two models of intelligent vision sensors, the first in the world to be equipped with artificial intelligence (AI) processing capacity. By including this functionality in the image sensor itself, high-speed AI processing and extraction of only the necessary data is possible, which reduces latency in data transmission when using cloud services, addresses privacy problems and reduces energy consumption and communication costs. The new smart sensors expand the opportunities to develop cameras equipped with artificial intelligence, open the field for a wide range of applications in both industrial equipment and retail companies, and help build optimal systems connected to the cloud.

The generalization of the internet of things (IoT) has made it possible for any type of device to connect to the cloud, making the use of information processing systems common where it is obtained in said devices and processed in the cloud using AI . On the other hand, the increasing volume of information handled in the cloud raises several problems: the increase in latency in data transmission makes it difficult to process information in real time , as well as security problems for users related to storage in the cloud of personal identification data; and other problems such as increased energy consumption and communication costs associated with cloud services.

Technical characteristics

From a technical point of view, the new sensors have a layered configuration consisting of a pixel chip and another logic chip. They are the world’s first image sensors equipped with AI image processing and analysis capabilities on the logic chip. The signal acquired by the pixel chip is processed using AI in the sensor itself, which eliminates the need for high-performance processors or external memory, and enables the development of artificial intelligence systems on the device itself. The sensor generates metadata (semantic information that pertains to the image data) instead of the image information, reducing data volume and privacy concerns. In addition, the AI capability makes it possible to offer various functionalities such as real-time object tracking with high-speed AI processing. Different AI models can also be chosen, rewriting the internal memory according to user requirements or conditions at the location where the system is used.

The pixel chip is backlit and has approximately 12.3 effective megapixels for capturing information at a wide viewing angle. In addition to the usual image sensor logic circuitry, the logic chip is equipped with an original Sony digital signal processor (DSP) dedicated to artificial intelligence signal processing and memory for the IA model. This configuration eliminates the need for high-performance processors or external memory, making it ideal for Edge AI systems.

Metadata output

The signals acquired by the pixel chip are run through an image signal processor (ISP) and artificial intelligence processing is done at the on-chip processing stage of logic. The information extracted is generated in the form of metadata, which reduces the amount of data that must be handled. Since image information is not displayed, this system helps reduce security risks and resolve privacy concerns. In addition to the image recorded by the conventional image sensor, users can select the data output format according to their needs and uses, such as ISP format images (YUV / RGB) or images of specific areas or ROI (Regions of Interest). ).

When recording video with a conventional image sensor, data from each individual frame needs to be sent for AI processing, which causes high data throughput and hinders proper real-time performance. Sony’s new sensors perform ISP processing as well as AI at high speed on the logic chip, so they can complete the entire process on each frame of video . This design makes it possible to offer high-precision real-time object tracking during video recording.

Chip war rages on: TSMC suspends Biren chip manufacturing

The semiconductor company is seeking to follow US rules, which prevent companies from developing certain technologies.

Do you want your company to be more optimal? use digital twins

According to one study, organizations that use them have seen a 15% improvement in operational metrics.

Not everything is DALL-E: The best websites to create images with Artificial Intelligence

The use of Artificial Intelligence in creative work is becoming more and more common and these are some of the platforms that you can use.

Electric cars with charge in 5 minutes: a new NASA technology will make it...

A new technology funded by NASA for future space missions can charge an electric car in just five minutes.

This is Optimus, Tesla's humanoid robot

Impressed? Tesla founder Elon Musk wants to build millions of robots like these and sell them for 20,000 euros a unit.

More