A Primer on Motion Capture with Deep Learning: Principles, Pitfalls and Perspectives

1 code implementation • 1 Sep 2020 • Alexander Mathis, Steffen Schneider, Jessy Lauer, Mackenzie W. Mathis

Extracting behavioral measurements non-invasively from video is stymied by the fact that it is a hard computational problem.

SuperAnimal pretrained pose estimation models for behavioral analysis

2 code implementations • 14 Mar 2022 • Shaokai Ye, Anastasiia Filippova, Jessy Lauer, Steffen Schneider, Maxime Vidal, Tian Qiu, Alexander Mathis, Mackenzie Weygandt Mathis

We illustrate the utility of our models in behavioral classification in mice and gait analysis in horses.

Animal Pose Estimation Transfer Learning

Multi-animal pose estimation, identification and tracking with DeepLabCut

2 code implementations • Nature Methods 2022 • Jessy Lauer, Mu Zhou, Shaokai Ye, William Menegas, Steffen Schneider, Tanmay Nath, Mohammed Mostafizur Rahman, Valentina Di Santo, Daniel Soberanes, Guoping Feng, Venkatesh N. Murthy, George Lauder, Catherine Dulac, Mackenzie Weygandt Mathis & Alexander Mathis

Estimating the pose of multiple animals is a challenging computer vision problem: frequent interactions cause occlusions and complicate the association of detected keypoints to the correct individuals, as well as having highly similar looking animals that interact more closely than in typical multi-human scenarios.

Animal Pose Estimation

AmadeusGPT: a natural language interface for interactive animal behavioral analysis

1 code implementation • NeurIPS 2023 • Shaokai Ye, Jessy Lauer, Mu Zhou, Alexander Mathis, Mackenzie W. Mathis

To overcome the context window limitation, we implement a novel dual-memory mechanism to allow communication between short-term and long-term memory using symbols as context pointers for retrieval and saving.

Descriptive