Language Supervision Improves Neural Decoding Across Brain-Computer Interface Tasks

Summary

Brain-computer interfaces commonly train classifiers with class-label supervision, which identifies categories but leaves relationships among them implicit. The paper introduces Language as Supervision (LaS), a method that aligns neural features with natural-language descriptions of classes through learned projections and then predicts using candidate text prototypes. The authors evaluate seven encoder architectures, with independently trained models, across visual, motor, and speech decoding from both scalp and intracranial recordings. LaS improves speech-decoding accuracy by as much as 17.15 percentage points over matched class-label baselines. Its learned representations can be reused with new candidate sets and support subject and session adaptation using label supervision. In steady-state visual evoked potential experiments, the method also enables conditional decoding at unseen frequencies. This generalization depends on matching each class with the correct description. The paper additionally presents dual-sided attribution visualizations that show how neural and textual inputs contribute to the same class-matching score, suggesting that language descriptions may provide both a reusable reference and an interpretability aid for neural decoding.