A research team from China has created a sensor that generates tokens for AI as soon as light is detected, doing away with a lengthy chain of energy-sapping steps that typical camera systems rely on. The device, known as LightTok, packs the functions of an analog-to-digital converter, a memory buffer, and a computation engine into one chip.
The team, led by Miao Feng of Nanjing University’s Institute of Brain-Inspired Intelligence, published the work Aug. 19 in the journal Nature Sensors. LightTok’s design takes the raw light that hits a pixel and turns it directly into a format AI models can read, without the intermediate steps that waste most of a sensor’s power.
How A Sensor Used To Work
A camera sensor gathers light and an analog-to-digital converter turns it into pixels. The information rests temporarily in memory before it is broken into smaller pieces on a separate chip — like cutting a photograph into a grid of square compartments — so that each piece can be changed into a token for an AI model to work with.
According to one study that other scholars have drawn on, an analog-to-digital converter accounts for 66% of an image sensor’s energy use on average. Shifting data processing away from the chip and into the cloud only aggravates the energy problem.
The Chip That Skips The Queue
LightTok combines sensing, memory, and computation into one step, compressing what would otherwise be five distinct phases into a single operation. The researchers constructed their array with a single-layer molybdenum disulfide floating-gate phototransistor, an approach that can detect light, keep track of what it detected, and then use that information for a calculation.
Molybdenum disulfide is a 2D material that reacts well to light and can be grown in sheets one atom thick. The phototransistor converts incoming photons into an electrical current, while the floating gate holds an electrical charge rather than letting it disappear after the light goes away.
Xinhua reported that Liang Shi-Jun, a physics professor at Nanjing University, described the chip as one that gets rid of data movement altogether, which is the main source of energy waste. The device takes in light and gives out tokens — that is where the name “LightTok.” comes from.
The Numbers On The Page
The researchers reported in their study that during testing, LightTok reached 87.3% accuracy in image recognition, and it did so while being 10 times more energy efficient at converting light into tokens than the conventional multi-step process.
The drawback is the camera’s current resolution, which sits at just 32 by 32 photosensitive pixels — well below the quality offered by today’s smartphone cameras. The researchers argue that the technology can be scaled up using the complementary metal-oxide-semiconductor manufacturing process, the same method used to fabricate chips in smartphones, laptops, and sensors in drones.
What Kumar Sokka Says
Kumar Sokka, who leads Acre Security, a company that provides real-world sensing for critical infrastructure, characterized the work as “small-scale demonstration.”. He was not involved in the research.
The direction of signal processing at the point of detection matters to anyone working in the physical world, Sokka explained. He said it could be “an enabler for pervasive physical AI, and a clever one, but it’s not a cure for the whole problem,” which is a significant shift in how data is handled.
The Broader Picture
Alongside other recent research on energy-efficient AI sensing, this work stands in line. One paper described how a “Rainbow-on-a-chip” came about by accident. Another paper detailed an AI chip that imitates the human brain’s split-second motor control, solving problems using 10,000 times fewer calculations.
The headline figure is LightTok’s 10-fold energy savings on token production, though the 32×32 limit currently keeps the system a proof of concept. The researchers point to standard chip manufacturing as the route to future scaling, with a disaster-scanning drone serving as the promised use case.
“Light comes in, tokens come out” — hence the name “LightTok.”
The concept is a promising one, as is its approach. The real issue is whether the production route can deliver on it, bringing the vision into drones and autonomous hardware.
Source material: “New LightTok chip converts light directly into 'tokens' for AI — slashing energy use in drones and other autonomous machines,” Live Science.
Get the Notebook.
The day's best stories and every fresh verdict, in plain English, in your inbox by seven. One email a day, no more.