Abstract
This note revisits an early brain-supervised representation-learning project and contrasts its original evidential standard with the one used in later NeuroAI work. The earlier study used fMRI response similarity to train a contrastive transformation of visual features, then evaluated whether the transformed representation better matched neural geometry and predicted held-out fMRI responses. Those measurements supported a bounded claim about improved representation–brain correspondence under the tested pipeline, but they did not identify why the gain occurred or whether correct neural pairing was necessary. A later eight-participant residual-supervision experiment made that limitation explicit: paired neural supervision beat a nonlinear readout in all participants, yet lost to the best matched non-neural target in all participants and was effectively tied with a pairing-broken neural control. The resulting methodological revision is to treat “brain alignment” as a graded evidential claim rather than a binary label. Predictive alignment, representational similarity, specificity, generalization, identification, and causal correspondence require different tests.
Key result. The evidential standard changed from “the neural metric improved” to “which alternative explanations did the experiment rule out?” This is a retrospective about claim calibration, not a retraction of the earlier measurements.
1. Historical study
The original project, Contrastive Representation Learning in the Brain, asked whether neural response geometry could be used to reshape an artificial visual representation.
For pairs of images, similarity between fMRI response patterns in a region of interest defined positive and negative relationships. A shared nonlinear mapping f(x) was trained with a contrastive objective so that transformed visual features more closely reproduced those neural pairwise relationships.
The transformed representation was evaluated with two main criteria:
- representational geometry: pairwise similarities in the transformed space should more closely resemble fMRI-response similarities;
- held-out encoding: an encoding model built from transformed features should predict held-out fMRI responses better than one built from the original features.
Both quantities improved in the pilot analysis. The draft therefore described the transformed latent space as a “more accurate model of the brain.”
The empirical measurements remain meaningful. The stronger wording is what now requires qualification.
2. What the original result established
The original design supported the following bounded statement:
Under the tested participant, ROI, stimulus split, feature space, nonlinear transformation, and encoding readout, neural-supervised transformation improved measured correspondence with the observed fMRI responses.
That is a real result about a representation–mapping–data system.
However, several causal explanations remain observationally compatible with the same improvement:
- exact stimulus–brain pairing supplied unique information;
- broad category or semantic structure in the neural target was sufficient;
- the neural target acted primarily as a regularizer;
- a matched non-neural target would have produced a similar transformation;
- the nonlinear mapping created a basis that was simply easier for the encoding readout to use;
- the effect was specific to one participant, ROI, or stimulus distribution.
Neither improved encoding nor improved representational similarity distinguishes these alternatives by itself.
3. Later falsification test
A later project tested one of these unresolved alternatives directly.
A small residual adapter was trained using correctly paired fMRI-derived targets and evaluated in eight leave-one-participant-out folds. Under a conventional positive comparison, the result looked strong: the paired neural adapter outperformed a direct nonlinear readout in 8/8 participants, with a mean difference of approximately +0.0683.
The experiment also included claim-specific controls:
| Contrast | Mean difference | Positive participants |
|---|---|---|
| Paired neural − direct nonlinear readout | +0.0683 | 8/8 |
| Paired neural − best matched non-neural | −0.0221 | 0/8 |
| Paired neural − within-category shuffled neural | −0.00073 | 1/8 |
The correct neural pairing therefore did not provide the specificity required by the stronger neural-supervision claim. The adapter learned useful structure, but the experiment did not identify that structure as uniquely supplied by stimulus-specific brain residuals.
This result changed the interpretation standard more than the training objective itself.
4. Revised evidence ladder for “brain alignment”
“Brain-aligned” is more useful when decomposed into explicit evidential levels.
4.1 Predictive alignment
Does the representation predict held-out neural responses under a declared readout and evaluation contract?
This establishes predictive accessibility, not unique representation or mechanism.
4.2 Representational alignment
Do internal geometries or decodable variables resemble measured neural structure beyond what a scalar predictive score captures?
This strengthens the description of what is similar but still permits multiple causal explanations.
4.3 Specificity
Does the effect defeat controls that preserve nuisance structure while removing the information named in the claim?
For neural supervision, useful controls include matched non-neural objectives and pairing-broken neural targets.
4.4 Generalization
Does the result transfer to held-out participants, stimuli, tasks, datasets, or modalities without selecting the representation using those outcomes?
This determines the scope of the claim.
4.5 Identification
Can the evaluator distinguish the proposed representation from plausible alternatives when the ground truth is known?
Known-truth recovery is one way to calibrate this property.
4.6 Causal or mechanistic alignment
Do interventions produce the predicted changes in model states, neural responses, behavior, or closed-loop dynamics?
This is the strongest level and is not required for every useful NeuroAI study.
5. Why stronger optimization increases the need for external tests
Modern brain-alignment methods increasingly optimize models directly against EEG, fMRI, or human representational objectives. This makes the original research direction more relevant, not less.
It also changes what should count as a convincing evaluation.
If a model is trained explicitly to improve a neural similarity metric, improvement on that same or closely related metric is partly built into the objective. Stronger evidence therefore comes from tests the training procedure could not trivially satisfy:
- held-out participants;
- frozen external neural datasets;
- matched non-neural supervision;
- broken-pairing controls;
- behavioral transfer;
- metamers or adversarial perturbations;
- known-truth recovery;
- targeted causal interventions.
The stronger the optimization procedure, the more important these out-of-objective tests become.
6. What would change in the original design
A contemporary version of the early contrastive study would retain the basic neural-supervised transformation but add an explicit identification structure.
At minimum, the design would include:
- held-out stimuli for the primary representation and encoding gains;
- held-out participants for claims about shared human structure;
- matched non-neural targets with the same architecture, data, capacity, and optimization budget;
- pairing-broken neural controls preserving broad neural structure;
- external neural or behavioral tests excluded from model selection;
- known-truth recovery for the evaluator;
- targeted perturbations or metamers that force aligned and control models to disagree.
The main methodological addition is not a larger network. It is a better set of alternatives.
7. Claim boundary
This retrospective does not argue that the original neural-supervised measurements were invalid. It revises the claim attached to them.
The earlier evidence supports improved measured correspondence under the tested pipeline. It does not, by itself, support:
- unique neural information as the cause of the gain;
- participant-general representation learning;
- representational identity;
- a shared biological mechanism.
The broader conclusion is therefore procedural:
A brain-alignment claim should name the level of evidence reached and the alternatives the experiment actually excludes.
“Brain alignment” as a training objective, a metric improvement, and an explanatory claim are not interchangeable.
8. Archival and reproducibility boundary
The historical pilot manuscript, figures, and later falsification program are retained in private research repositories with versioned source and result records. The later eight-participant pairing-specificity experiment includes frozen controls, exact source identity, aggregate decision records, and protected participant-level artifacts.
This public retrospective reports publication-safe methodological lessons and aggregate results. It does not expose private neural data, owner-only artifacts, or internal repository contents.
Related methodological results are summarized in A Brain-Predictive Model Is Not Yet an Explanation and When Brain Supervision Helps for the Wrong Reason.
References
- Han, Y., Poggio, T. A., & Cheung, B. (2023). System Identification of Neural Systems: If We Got It Right, Would We Know?. ICML.
- Feather, J., Leclerc, G., Mądry, A., & McDermott, J. H. (2023). Model metamers reveal divergent invariances between biological and artificial neural networks. Nature Neuroscience.
- McNeal, N., Deb, M., & Murty, N. A. R. (2024). Small-scale adversarial perturbations expose differences between predictive encoding models of human fMRI responses.
- Lu, Z., Wang, Y., & Golomb, J. D. (2024). Achieving more human brain-like vision via human EEG representational alignment.
- Teaching CORnet human fMRI representations for enhanced model-brain alignment (2025).
- Human-like cognitive generalization for large models via mental representation-guided supervision (2026).