Scientists have turned brain activity from mice into reconstructed videos of what the animals were seeing, a step toward understanding how the brain reshapes vision before it becomes perception. The work comes from researchers at University College London (UCL) and appears in eLife.
Reading Neurons for Sight
The team recorded signals from individual brain cells in the visual cortex of mice, then used those readings to build 10-second clips of what the animals were watching. The approach differs from earlier work that relied on broad brain scans. Here, the recordings come from single neurons, giving a finer picture of how visual information is held in the brain.
Lead author Dr. Joel Bauer, of the Sainsbury Wellcome Centre at UCL, explained the goal: “We wanted to have a better way of investigating how the brain interprets what we see. The current methods of understanding what specific groups of neurons are representing are not very generalizable to situations which haven’t been specifically tested for.”
Turning Neural Readings Into Video
The reconstruction method draws on a model first developed for the 2023 Sensorium Competition. That model predicts how individual neurons will react while mice watch movies, factoring in the animals’ movements and changes in pupil size. The UCL team refined the approach using the same dataset.
They first calculated how neurons would behave if the mouse looked at a blank screen. They then compared that prediction to the neurons’ actual activity while the mouse watched a movie. The difference between predicted and measured activity drove an algorithm that adjusted the pixels of an initially blank movie, slowly bringing it closer to the video the mouse had actually seen.
A New Test, A New Video
The real test came when the model faced something it had never seen before. The researchers recorded a mouse’s brain activity during a video that had not been used in training. Using only that neural activity, the system reconstructed a 10-second movie that resembled the unseen video.
Bauer noted the strength of the result: “Using this approach, we were able to achieve high-quality reconstructions of 10-second video clips. The accuracy of the reconstructions improved with the inclusion of data from more individual neurons, demonstrating the importance of comprehensive neural data.” The method was not simply remembering previously shown videos. It used patterns of neural activity to infer visual information from a new scene.
Matching the Originals
To check the quality of the reconstructions, the researchers compared corresponding pixels in the original and rebuilt movies. The analysis found only small differences in timing between the two videos. There is still room to improve image resolution and the amount of the visual scene that can be rebuilt, according to the team.
Future work will collect data that supports sharper reconstructions and covers a larger part of what the animals are seeing.
Perception and Reality
The researchers now plan to use the technique to probe a deeper question: how much does the brain’s internal picture differ from the world that is actually in front of us? Vision is not simply a camera-like recording process. The brain continuously interprets, filters, and modifies incoming sensory information.
Bauer put the point directly: “We don’t have a perfect representation of the world in our heads. The visual processing pipeline skews and warps our representation in a way that modifies information. This deviation between reality and representations in the brain is not necessarily an error but a feature, reflecting how our minds interpret and augment sensory information. We want to explore how this happens in the brain.”
The work offers a fresh way to study how visual information moves through the brain, and it opens a path to comparing how different species see the same surroundings.
| Method | Focus | Data Type |
|---|---|---|
| Broad brain scans | Group behavior | Broad signals |
| Single-neuron recordings | Individual neuron response | Fine, detailed readings |
The work is a technical proof of concept, showing what is possible when recordings come from individual neurons rather than broad brain signals.
See the video the story is built around at Sciencedaily.
Get the Notebook.
The day's best stories and every fresh verdict, in plain English, in your inbox by seven. One email a day, no more.