Abstract:EEG visual decoding remains difficult because scalp recordings are noisy, spatially coarse, and variable across trials and subjects. Most systems align each trial with a dense image embedding, although that embedding mixes semantic content with instance-specific texture, pose, and background. This target can demand details absent from EEG and provides no explicit account of the visual factors supporting a prediction. We propose \textbf{ProtoEEG}, a prototype-mediated framework that reformulates EEG--image alignment as soft composition over a shared learnable visual codebook. A trainable EEG encoder and a frozen visual teacher produce assignment distributions over the same $K=64$ prototypes. Their weighted reconstructions form a structured representation for retrieval and image generation. Prototype contrast supplies the main cross-modal training signal, distribution alignment coordinates codebook use across modalities, an instance anchor preserves fine-grained discrimination, and diversity regularization prevents redundant prototypes. The shared assignment profile also makes the learned bottleneck directly testable through prototype retention, random selection, and identity shuffling. Across five seeds, ProtoEEG reaches 64.0\% Top-1 and 91.5\% Top-5 on THINGS-EEG2 200-way retrieval. After retraining on EEG-ImageNet, it obtains 44.8\% Top-1 and 73.8\% Top-5 in 80-way retrieval. ProtoEEG also ranks first on all seven reconstruction metrics. Removing prototype contrast lowers Top-1 by 56.9 percentage points, while shuffling prototype identities reduces it to 0.5\%. Prototype analyses further show distributed assignments and broad codebook use rather than one-prototype routing. These results support structured alignment targets as a practical alternative to increasing encoder or teacher capacity alone.
Submission history
Access Paper:
View PDF
HTML (experimental)
TeX Source
Additional Features
Audio Summary
Current browse context:
References & Citations
Google Scholar
Semantic Scholar
Bookmark
Institution
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.