Skip to main content

Behavioral Traits from Videos

illustration of two zebras on a savannah with one tree and a drone flying above.

Project Members

Sowbaranika Balasubramaniam, Namrata Banerji, Tanya Berger-Wolf, Tessa Cotron, Roi Harel, Maksim Kholiavchenko, Jenna Kline, Dan Rubenstein, Alec Sheets, Luke Song, Sam Stevens,Chuck Stewart, Nina van Tiel, Michelle White

Project Goals

To analyze the behavior of baboons, zebras, and giraffes through the application of computer vision techniques by utilizing videos that were captured using drones in Kenya. With computer vision algorithms, the team is able to recognize a range of distinct behavioral patterns exhibited by the animals, including activities such as walking, grazing, auto-grooming, mutual grooming, trotting, running, drinking, herding, lying down, mounting-mating, sniffing, urinating, defecating, dusting, fighting, chasing, and more. This project highlights the potential of computer vision and artificial intelligence to enhance our understanding of animal behavior in the wild.

Project Overview

The approach of using computer vision to analyze animal behavior allows for a deeper understanding of the animals' behavioral trends, and can provide insights into their social structures, mating habits, and daily activities. By analyzing patterns of behavior over time, we may be able to identify changes in behavior that could be indicative of environmental or social stressors, disease, or other factors affecting the animals' well-being. 

We collected video and gpa data to create "mini-scenes" for animal tracking, and trained computer models for behavior recognition. A mini-scene focuses on an individual and its surrounding environment, which are used to train the behavior recognition model. We have continued to use this mini-scene model to compensate for drone and animal movements and provide a focused context for categorizing individual animal behavior. We plan on using this structure to continue research into different species and their individual behaviors.  

Related Publications

Jenna M Kline; Tanya Berger-Wolf; Daniel Rubenstein; Chuck Stewart, Christopher Stewart. (2023). Autonomous UAV Missions for Studying Wildlife Behavior: A Case Study for the Individual Identification of Zebras. Midwest Machine Learning Symposium. Chicago, IL.

Maksim Kholiavchenko; Jenna M Kline; Reshma R Babu; Michelle Ramirez; Samuel R Stevens ; Alec Sheets; Nina van Tiel; Elizabeth Campolongo; Sowbaranika Balasubramaniam; Tanya Berger-Wolf; Charles V Stewart; Daniel Rubenstein. (2023). Kenyan Animal Behavior Recognition from Videos: Taking the First Steps. Midwest Machine Learning Symposium. Chicago, IL.

Tools & Repositories

KABR: In-Situ Dataset for Kenyan Animal Behavior Recognition from Drone Videos