Linking 3D Pose Estimation with Neural Activity as a Key to Understand Spatial Cognition

Abstract

Understanding how the brain computes spatial representations requires linking neural dynamics with precise behavior and movement measurements. Although past research using rodent animal models typically relates neural signals to two-dimensional positions, natural behavior occurs in three dimensions (3D). To address this limitation, we developed a robust 3D kinematic tracking pipeline, utilizing an Aligned Neural Network for Computational Ethology (DANNCE)—a markerless deep-learning framework for 3D pose estimation in freely moving animals. We applied DANNCE to video data obtained from six synchronized cameras positioned around a recording arena. To optimize this setup, we generated a custom dataset via Label3D that consisted of approximately 500 representative frames capturing diverse behaviors such as rearing, grooming, and walking. We annotated each frame manually by labeling key anatomical landmarks across synchronized camera views while monitoring corresponding triangulated 3D skeletal reconstructions. With about 300 frames annotated to date, the dataset can be used to fine-tune an existing pretrained DANNCE network, significantly improving accuracy for our lab-specific arena settings by eliminating skeletal jitter, and producing continuous, high-resolution 3D pose estimates. Ultimately, these 3D skeletal trajectories will allow us to precisely align behavior with simultaneously recorded neural activity, delivering an important methodological advance for investigating how fine-scale body posture and movement interact with the neural dynamics underlying spatial representation.