Letting the Neural Code Speak: Automated Neuron Characterization via Language

This skill covers methodology from the paper "Letting the neural code speak: Automated characterization of monkey visual neurons through human language" (arXiv:2605.12485), which develops a closed-loop framework that characterizes individual neurons in macaque V1 and V4 using natural language descriptions.

Core Findings

1. Natural language captures neuron selectivity: Across macaque V1 and V4, most neurons can be described by concise, verifiable semantic descriptions (oriented edges and spatial frequency in V1; conjunctions of form, color, and texture in V4).

2. Closed-loop verification framework:

   • Translate neuron's high/low activating images into dense captions
   • Generate a semantic hypothesis from captions
   • Synthesize images from hypothesis
   • Verify hypothesis in silico by testing synthesized images on neuron's digital twin

3. Quantitative validation: In V4, images generated from activating hypotheses drove 96.1% of neurons above the 95th percentile of natural-image responses; suppressing hypotheses drove 97.6% below the 5th percentile (vs ~10% for random images).

4. Language compression is lossy but semantically faithful: RSA reveals alignment lost in text bottleneck is recovered when hypotheses are rendered back into images.

Methodology

Digital Twin Construction

• Create neural digital twins of macaque V1 and V4 using large-scale neural recordings
• Digital twin predicts neural responses to arbitrary visual stimuli

Closed-Loop Framework

1. Image Selection: Identify high-activating and low-activating images for each neuron
2. Caption Generation: Convert images into dense natural language captions
3. Hypothesis Formation: Generate semantic hypothesis describing neuron's selectivity
4. Image Synthesis: Generate new images that match the semantic hypothesis
5. Verification: Test synthesized images on the neural digital twin
6. Iteration: Refine hypothesis based on verification results

Analysis

• Representational Similarity Analysis (RSA): Compare neural activity, vision embeddings, and language embeddings
• Layer-wise retrieval: Track how selectivity varies across cortical hierarchy
• Activation/Suppression asymmetry: V1 activation well-described, V1 suppression less describable

Key Insights

• Combines generative models with neural digital twins for interpretable, testable descriptions of neural function at scale
• Enables automated scientific discovery in neuroscience via agentic AI
• Natural language serves as a bridge between neural representation and human understanding
• Vision-aligned embeddings most closely match neural activity; language provides interpretable bottleneck

Resources

• Paper: https://arxiv.org/abs/2605.12485
• Authors: Vedang Lad, Katrin Franke, Tamar Rott Shaham, Surya Ganguli, Andreas S. Tolias, Sophia Sanborn, Nikos Karantzas
• Submitted: 12 May 2026 (v2: 18 May 2026)

Activation Keywords

• neural code language
• automated neuron characterization
• neural digital twin visual cortex
• closed-loop neuron description
• semantic hypothesis neural selectivity
• macaque V1 V4 neuron description
• 神经代码语言 自动神经元表征
• generative model neural characterization
• agentic neuroscience discovery