卡萨布兰卡

Chinese team develops intelligent vision sensor that turns light signals into AI-ready tokens, reducing use of energy_我的网站

A |     每日商报讯 近期，杭州多地雨情严峻，地灾风险高。

B |     

    The new ultra-low-power intelligent vision sensor chip "LightTok" developed by a research team from Nanjing University. Photo: from Science and Technology Daily
A Chinese research team from Nanjing University has developed a new ultra-low-power intelligent vision sensor chip, dubbed "LightTok," that can convert light signals into tokens within the sensor, significantly reducing the high energy consumption caused by frequent transfers of massive amounts of redundant data, the principal investigator told the Global Times.
According to a release from the Institute of Brain-Inspired Intelligence of Nanjing University, tokens generated by the LightTok chip can be directly fed into a Transformer encoder for image recognition.
"Our design idea was to move token generation onto the sensor itself, allowing the chip to directly produce tokens that AI models can process once light reaches the sensor," Miao Feng, director of the Institute of Brain-Inspired Intelligence at Nanjing University, told the Global Times on Thursday. "These tokens contain complete image information."
Physical AI refers to intelligent systems capable of autonomously perceiving, reasoning, acting and receiving feedback in the real world, representing a key pathway for AI to move from the digital realm into the physical world. Vision-based physical AI systems powered by large AI models need to convert visual information from real-world environments into tokens that can be processed by AI models before feeding these tokens into Transformers for subsequent tasks.
In traditional visual perception pipelines, light signals must go through multiple stages, including image sensing, analog-to-digital conversion, data buffering and transfer, digital image patching and embedding, before being transformed into tokens that AI models can process. The frequent transfer of massive amounts of redundant data has resulted in high energy consumption at the edge, according to a report by Science and Technology Daily.
The LightTok chip directly addresses a key challenge in physical AI hardware: how to efficiently acquire and tokenize visual information from the physical world with low energy consumption, Miao said.
The LightTok chip consists of a photosensitive memory array and peripheral circuits. The team built the array based on single-layer molybdenum disulfide (MoS₂) floating-gate phototransistors, with each pixel capable of sensing light, storing information and performing analog computing. By processing optical information directly within the chip, the device can convert captured visual signals into tokens for AI models, according to the research team.
The current LightTok prototype has a resolution of 32×32 pixels, or 1,024 photosensitive pixels, which is still smaller than that of smartphone cameras and industrial imaging systems. However, Miao said the technology is compatible with CMOS manufacturing processes and can be scaled up. With wafer-level growth of molybdenum disulfide materials, the chip could potentially achieve a scale comparable to existing imaging devices.
Miao said the chip also draws inspiration from the information-processing mechanism of human vision. Similar to how the retina extracts key visual information before transmitting it to the brain, the chip also aims to process visual information at an early stage. 
The research was conducted in collaboration with another research team from the National University of Singapore. The findings were published on Wednesday in Nature Sensors, an internationally renowned journal in the field of sensing technology, according to the release.
Potential applications for LightTok include drone systems, autonomous remote sensing and small-scale embodied AI systems, Miao said. In these scenarios, devices need to continuously detect, understand and track targets, generating massive amounts of visual data. By reducing the energy required for visual processing, the technology could extend the operating time of drones, satellites and small robots with limited power supplies, the expert said.    
。

C | 市规划资源系统党员干部快速响应，全员到岗，下沉一线，切实强化地灾防御工作，确保人民群众生命财产安全。局主要领导全程统筹调度，其余局领导连夜带队赴西湖、萧山、余杭、富阳、临安、桐庐、淳安、建德等区、县（市），深入雨量较大、风险较高区域下沉服务，指导落实巡查排查、转移避险等各项防御举措。截至8月16日13时，市级维持地质灾害防御二级响应，持续落实24小时值班值守；累计下发工作提示单4份，发布地质灾害风险预报12期，按小时滚动发布实时预警38000次，下发实时预警告知单3177份，叫应叫醒防灾责任人和受威胁群众8362人次；自然资源领域转移群众9674人，其中“双控”点位转移7099人、其他风险点位扩面转移2575人。值得一提的是，拱墅分局对半山地质灾害巡查点老虎山一号矿、老虎山二号矿及娘娘庙周边等重点风险区域开展拉网式巡查；西湖分局邀请省地质院、地矿勘察院等专业技术力量开展现场地质情况研判；滨江分局会同相关力量30余人每日对41处已治理点位巡查不少于2次；萧山分局将骨干力量21人下沉至各街道（乡镇），统筹安排5个地质专家工作组，分赴重点区域驻点研判。“迎战强降雨，做好地灾防范，既是上级要求，也是对人民生命财产负责的有效措施，更是建设平安杭州和信用杭州的现实需要。”市规划资源局相关负责人说。

Current article:http://md2n.shouyouxumianpouluanteng.shop/xlr/p3qra.html

Published on:00:00:00