爱情连连看

Chinese team develops intelligent vision sensor that turns light signals into AI-ready tokens, reducing use of energy_我的网站

回魂夜

一 |

    The new ultra-low-power intelligent vision sensor chip "LightTok" developed by a research team from Nanjing University. Photo: from Science and Technology Daily
A Chinese research team from Nanjing University has developed a new ultra-low-power intelligent vision sensor chip, dubbed "LightTok," that can convert light signals into tokens within the sensor, significantly reducing the high energy consumption caused by frequent transfers of massive amounts of redundant data, the principal investigator told the Global Times.
According to a release from the Institute of Brain-Inspired Intelligence of Nanjing University, tokens generated by the LightTok chip can be directly fed into a Transformer encoder for image recognition.
"Our design idea was to move token generation onto the sensor itself, allowing the chip to directly produce tokens that AI models can process once light reaches the sensor," Miao Feng, director of the Institute of Brain-Inspired Intelligence at Nanjing University, told the Global Times on Thursday. "These tokens contain complete image information."
Physical AI refers to intelligent systems capable of autonomously perceiving, reasoning, acting and receiving feedback in the real world, representing a key pathway for AI to move from the digital realm into the physical world. Vision-based physical AI systems powered by large AI models need to convert visual information from real-world environments into tokens that can be processed by AI models before feeding these tokens into Transformers for subsequent tasks.
In traditional visual perception pipelines, light signals must go through multiple stages, including image sensing, analog-to-digital conversion, data buffering and transfer, digital image patching and embedding, before being transformed into tokens that AI models can process. The frequent transfer of massive amounts of redundant data has resulted in high energy consumption at the edge, according to a report by Science and Technology Daily.
The LightTok chip directly addresses a key challenge in physical AI hardware: how to efficiently acquire and tokenize visual information from the physical world with low energy consumption, Miao said.
The LightTok chip consists of a photosensitive memory array and peripheral circuits. The team built the array based on single-layer molybdenum disulfide (MoS₂) floating-gate phototransistors, with each pixel capable of sensing light, storing information and performing analog computing. By processing optical information directly within the chip, the device can convert captured visual signals into tokens for AI models, according to the research team.
The current LightTok prototype has a resolution of 32×32 pixels, or 1,024 photosensitive pixels, which is still smaller than that of smartphone cameras and industrial imaging systems. However, Miao said the technology is compatible with CMOS manufacturing processes and can be scaled up. With wafer-level growth of molybdenum disulfide materials, the chip could potentially achieve a scale comparable to existing imaging devices.
Miao said the chip also draws inspiration from the information-processing mechanism of human vision. Similar to how the retina extracts key visual information before transmitting it to the brain, the chip also aims to process visual information at an early stage.
The research was conducted in collaboration with another research team from the National University of Singapore. The findings were published on Wednesday in Nature Sensors, an internationally renowned journal in the field of sensing technology, according to the release.
Potential applications for LightTok include drone systems, autonomous remote sensing and small-scale embodied AI systems, Miao said. In these scenarios, devices need to continuously detect, understand and track targets, generating massive amounts of visual data. By reducing the energy required for visual processing, the technology could extend the operating time of drones, satellites and small robots with limited power supplies, the expert said.
。    本报讯日前，根据森林防火工作实际，孤竹营子乡做出周密细致的安排部署，严格落实“网格化”管理机制，全员投入森林防火工作中。

二 | 　　孤竹营子乡党委、政府高度重视，成立森林防火指挥部，做到早安排、早部署，明确森林防火工作重点、关键节点、应急措施等工作内容，要求全乡干部群众统一思想，层层签订防火责任状，确保森林防火工作不留盲点、不留死角，做到万无一失。　　孤竹营子乡加大防火工作宣传力度，发放防火相关宣传单1000余份，悬挂宣传条幅50余面、森林防火旗帜400面，出动防火宣传车490台次，对已损坏的防火标牌进行了补充，并有专人负责看管。设立森林防火瞭望哨45个，进山路口检查站3个。　　截至目前，全乡已投入森林防火资金15万元，主要用于防火宣传、机具维修、物资储备、车辆维护、燃油防火人员培训，演练及启用村防火人员工资等。　　孤竹营子乡全体乡村干部54人，每天坚持到责任区域防火，半专业化扑火队员16人、临时防火队员50人已全部落实到位，并进行了三次专业化培训。森林防火物资齐备，车辆及机具完好，扑火队员处于高度戒备状态，24小时待命,每天不间断巡查。