Mackenzie Mathis

Brain dynamics uncovered using a machine-learning algorithm

CEBRA is a machine-learning method that can be used to compress time series in a way that reveals otherwise hidden structures in the variability of the data. It excels at processing behavioural and neural data recorded simultaneously, and it can decode activity from the visual cortex of the mouse brain to reconstruct a viewed video.

NATURE PORTFOLIO2023

Learnable latent embeddings for joint behavioural and neural analysis

Jin Hwa Lee, Mackenzie Mathis, Steffen Schneider

Mapping behavioural actions to neural activity is a fundamental goal of neuroscience. As our ability to record large neural and behavioural data increases, there is growing interest in modelling neural dynamics during adaptive behaviours to probe neural representations(1-3). In particular, although neural latent embeddings can reveal underlying correlates of behaviour, we lack nonlinear techniques that can explicitly and flexibly leverage joint behaviour and neural data to uncover neural dynamics(3-5). Here, we fill this gap with a new encoding method, CEBRA, that jointly uses behavioural and neural data in a (supervised) hypothesis- or (self-supervised) discovery-driven manner to produce both consistent and high-performance latent spaces. We show that consistency can be used as a metric for uncovering meaningful differences, and the inferred latents can be used for decoding. We validate its accuracy and demonstrate our tool's utility for both calcium and electrophysiology datasets, across sensory and motor tasks and in simple or complex behaviours across species. It allows leverage of single- and multi-session datasets for hypothesis testing or can be used label free. Lastly, we show that CEBRA can be used for the mapping of space, uncovering complex kinematic features, for the production of consistent latent spaces across two-photon and Neuropixels data, and can provide rapid, high-accuracy decoding of natural videos from visual cortex.

NATURE PORTFOLIO2023

Reaching an understanding of cortico-medullary control of forelimb behaviors

Mackenzie Mathis

How the neocortex modulates hindbrain and spinal circuits is of fundamental interest for understanding mo-tor control and adaptive behaviors. New work from Yang, Kanodia, and Arber demonstrates that there is an exquisite anatomical organization and functional modulation from the anterior (motor) cortex on downstream medulla populations during forelimb behaviors in mice.

CELL PRESS2023

Multi-animal pose estimation, identification and tracking with DeepLabCut

Jessy Lauer, Mackenzie Mathis, Alexander Mathis, Tanmay Nath, Steffen Schneider, Shaokai Ye

Estimating the pose of multiple animals is a challenging computer vision problem: frequent interactions cause occlusions and complicate the association of detected keypoints to the correct individuals, as well as having highly similar looking animals that interact more closely than in typical multi-human scenarios. To take up this challenge, we build on DeepLabCut, an open-source pose estimation toolbox, and provide high-performance animal assembly and tracking-features required for multi-animal scenarios. Furthermore, we integrate the ability to predict an animal's identity to assist tracking (in case of occlusions). We illustrate the power of this framework with four datasets varying in complexity, which we release to serve as a benchmark for future algorithm development. DeepLabCut is extended to enable multi-animal pose estimation, animal identification and tracking, thereby enabling the analysis of social behaviors.

NATURE PORTFOLIO2022

Next-generation brain observatories

Mackenzie Mathis, Shreya Saxena

We propose centralized brain observatories for large-scale recordings of neural activity in mice and non-human primates coupled with cloud-based data analysis and sharing. Such observatories will advance reproducible systems neuroscience and democratize access to the most advanced tools and data.

CELL PRESS2022

Perspectives in machine learning for wildlife conservation

Benjamin Alexander Kellenberger, Mackenzie Mathis, Alexander Mathis, Devis Tuia

Inexpensive and accessible sensors are accelerating data acquisition in animal ecology. These technologies hold great potential for large-scale ecological understanding, but are limited by current processing approaches which inefficiently distill data into relevant information. We argue that animal ecologists can capitalize on large datasets generated by modern sensors by combining machine learning approaches with domain knowledge. Incorporating machine learning into ecological workflows could improve inputs for ecological models and lead to integrated hybrid modeling tools. This approach will require close interdisciplinary collaboration to ensure the quality of novel approaches and train a new generation of data scientists in ecology and conservation.

2022

Striatal dopamine explains novelty-induced behavioral dynamics and individual variability in threat prediction

Mackenzie Mathis, Alexander Mathis

Animals both explore and avoid novel objects in the environment, but the neural mechanisms that underlie these behaviors and their dynamics remain uncharacterized. Here, we used multi-point tracking (DeepLabCut) and behavioral segmentation (MoSeq) to characterize the behavior of mice freely interacting with a novel object. Novelty elicits a characteristic sequence of behavior, starting with investigatory approach and culminating in object engagement or avoidance. Dopamine in the tail of the striatum (TS) suppresses engagement, and dopamine responses were predictive of individual variability in behavior. Behavioral dy-namics and individual variability are explained by a reinforcement-learning (RL) model of threat prediction in which behavior arises from a novelty-induced initial threat prediction (akin to "shaping bonus") and a threat prediction that is learned through dopamine-mediated threat prediction errors. These results uncover an algorithmic similarity between reward-and threat-related dopamine sub-systems.

CELL PRESS2022

AcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs in the Wild

Mackenzie Mathis, Alexander Mathis

Animals are capable of extreme agility, yet understanding their complex dynamics, which have ecological, biomechanical and evolutionary implications, remains challenging. Being able to study this incredible agility will be critical for the development of next-generation autonomous legged robots. In particular, the cheetah (acinonyx jubatus) is supremely fast and maneuverable, yet quantifying its whole-body 3D kinematic data during locomotion in the wild remains a challenge, even with new deep learning-based methods. In this work we present an extensive dataset of free-running cheetahs in the wild, called AcinoSet, that contains 119,490 frames of multi-view synchronized high-speed video footage, camera calibration files and 7,588 human-annotated frames. We utilize markerless animal pose estimation to provide 2D keypoints. Then, we use three methods that serve as strong baselines for 3D pose estimation tool development: traditional sparse bundle adjustment, an Extended Kalman Filter, and a trajectory optimization-based method we call Full Trajectory Estimation. The resulting 3D trajectories, human-checked 3D ground truth, and an interactive tool to inspect the data is also provided. We believe this dataset will be useful for a diverse range of fields such as ecology, neuroscience, robotics, biomechanics as well as computer vision. Code and data can be found at: https://github.com/African-Robotics-Unit/Acinoset.

IEEE2021

Motor control: Neural correlates of optimal feedback control theory

Mackenzie Mathis, Steffen Schneider

Recent work is revealing neural correlates of a leading theory of motor control. By linking an elegant series of behavioral experiments with neural inactivation in macaques with computational models, a new study shows that premotor and parietal areas can be mapped onto a model for optimal feedback control.

CELL PRESS2021

Mackenzie Mathis

Brain dynamics uncovered using a machine-learning algorithm

Learnable latent embeddings for joint behavioural and neural analysis

Reaching an understanding of cortico-medullary control of forelimb behaviors

Multi-animal pose estimation, identification and tracking with DeepLabCut

Next-generation brain observatories

Perspectives in machine learning for wildlife conservation

Striatal dopamine explains novelty-induced behavioral dynamics and individual variability in threat prediction

The neocortical column as a universal template for perception and world-model learning

AcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs in the Wild

Motor control: Neural correlates of optimal feedback control theory

Learnable latent embeddings for joint behavioural and neural analysis

Reaching an understanding of cortico-medullary control of forelimb behaviors

Multi-animal pose estimation, identification and tracking with DeepLabCut

Brain dynamics uncovered using a machine-learning algorithm

Next-generation brain observatories

Perspectives in machine learning for wildlife conservation

AcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs in the Wild

The neocortical column as a universal template for perception and world-model learning

Striatal dopamine explains novelty-induced behavioral dynamics and individual variability in threat prediction

Motor control: Neural correlates of optimal feedback control theory