We develop self-supervised models and neural architectures that learn transferable representations from large-scale brain recordings.
Large neural datasets offer an opportunity to learn reusable representations of brain activity. We develop spatiotemporal transformers, neural tokenizers, and self-supervised objectives that account for the structure of neural recordings. BaRISTA studies spatial encoding and masking across recording channels, atlas parcels, and brain regions. WiCAT extends shared representation learning to brain imaging across subjects, tasks, and datasets.
BaRISTA learns representations of human intracranial recordings through self-supervised pretraining. Its transformer architecture separates the spatial scales used for encoding and masking, showing that brain-region information can improve downstream decoding while preserving the reconstruction of individual recording channels.
WiCAT learns shared representations from widefield calcium imaging across subjects. Atlas-aligned tokenization and self-supervised pretraining support transfer across tasks and datasets, including continuous behavior decoding and reconstruction of unseen brain regions in animals excluded from training.
We build dynamical models that reveal how neural populations encode behavior, respond to inputs, and support real-time decoding.
Neural recordings combine activity related to behavior, internal states, and external inputs. Our models separate and prioritize behaviorally relevant dynamics, identify how inputs change neural systems, and enable flexible inference from incomplete observations. This work connects interpretable dynamical models with recurrent neural networks and model-based approaches to neurotechnology.
DPAD uses recurrent neural networks to separate and prioritize behaviorally relevant neural dynamics. It improves neural and behavioral prediction, preserves behavior information in low-dimensional representations, and tests where nonlinearities arise in neural-to-behavior transformations.
Hamidreza Abbaspourazad, Eray Erturk, Bijan Pesaran, Maryam M. Shanechi
DFINE combines nonlinear neural manifolds with tractable latent dynamics to enable real-time filtering, offline smoothing and inference with missing observations, while improving neural and behavioral prediction across diverse datasets.
BRAID incorporates measured external inputs into nonlinear neural-behavioral models, separating intrinsic dynamics from input effects. Prioritized training and a forecasting objective improve learning of behavior-related dynamics and prediction of future activity and behavior.
IPSID jointly models neural activity, behavior and measured inputs to distinguish intrinsic behavior-related dynamics from input-driven activity. It reveals low-dimensional intrinsic dynamics shared across subjects and tasks in the studied motor datasets.
Omid G. Sani, Hamidreza Abbaspourazad, Yan T. Wong, Bijan Pesaran, Maryam M. Shanechi
PSID learns neural dynamical models that prioritize activity related to measured behavior. It reveals lower-dimensional, more behavior-predictive dynamics than behavior-agnostic models, with demonstrations across reach-and-grasp and saccade tasks.
Yuxiao Yang, Shaoyu Qiao, Omid G. Sani, J. Isaac Sedillo, Breonna Ferrentino, Bijan Pesaran, Maryam M. Shanechi
Dynamic input-output models predict how temporally varying electrical stimulation changes activity across brain regions, providing a basis for precise neuromodulation and model-based control.
We learn from neural images and complementary signals, capturing brain dynamics across subjects, modalities, and timescales.
Neural imaging captures rich patterns across space and time. WiCAT learns shared representations of widefield calcium imaging across subjects; SBIND identifies behaviorally relevant dynamics in calcium and functional ultrasound images. In multimodal recordings, MRINE combines signals such as spikes and field potentials with different sampling rates, statistical distributions, and missing observations to support real-time decoding.
Mohammad Hosseini, Eray Erturk, Saba Hashemi, Maryam M. Shanechi
WiCAT learns shared representations from widefield calcium imaging across subjects. Atlas-aligned tokenization and self-supervised pretraining support transfer across tasks and datasets, including continuous behavior decoding and reconstruction of unseen brain regions in animals excluded from training.
SBIND models behaviorally relevant dynamics directly from neural image sequences. By capturing local and long-range spatial relationships and separating behavior-related activity from other dynamics, it improves neural and behavioral prediction in widefield calcium and functional ultrasound imaging.
MRINE combines neural modalities with different sampling rates and statistical distributions to decode behavior in real time. Its multiscale dynamical model fuses complementary information from signals such as spikes and field potentials while handling asynchronous observations and missing samples.
