Brain-IT Reconstructs Images From Brain Activity With One Hour of Adaptation

Summary

Researchers at the Weizmann Institute of Science have developed Brain-IT, an AI model that reconstructs an image a person is viewing from their brain activity. The system is designed to recognize brain-activity patterns shared across people, rather than relying on a separate model trained for each individual. According to Prof. Michal Irani, existing brain-to-image systems can preserve an image’s general meaning but often lose basic details such as composition and color; Brain-IT improves reconstruction of both content and detail. It requires about one hour of brain scans to adapt to a new person, compared with the dozens of hours typically required by other approaches. The team addressed the limited amount of paired image-fMRI data by training both a decoder, which maps brain activity to images, and an encoder, which predicts brain activity from images. The encoder generated synthetic scans for images not viewed in an MRI scanner, allowing the models to expand their effective training data. To handle differences between individual brains, the researchers analyzed roughly 40,000 brain voxels and separated visual features such as color and location from semantic features such as faces or food. The encoder identified 128 functional regions shared across people, including previously unknown divisions of labor in the brain’s place-processing area. The study used the Natural Scenes Dataset, which contains about 73,000 image-fMRI pairs from eight participants across 30 to 40 scanning sessions each. Brain-IT was selected for presentation at the International Conference on Learning Representations. The lab is extending the approach to auditory decoding, while video remains difficult because images change faster than fMRI scans can capture them; decoding dreams is described only as a possible future direction if those challenges are overcome.