A novel biomechanical approach for animal behaviour recognition using accelerometers

Data from animal-borne inertial sensors are widely used to investigate several aspects of an animal's life, such as energy expenditure, daily activity patterns and behaviour. Accelerometer data used in conjunction with machine learning algorithms have been the tool of choice for characterising animal behaviour. Although machine learning models perform reasonably well, they may not rely on meaningful features, nor lend themselves to physical interpretation of the classification rules. This lack of interpretability and control over classification outcomes is of particular concern where different behaviours have different frequency of occurrence and duration, as in most natural systems, and calls for the development of alternative methods. Biomechanical approaches to human activity classification could overcome these shortcomings, yet their full potential remains untapped for animal studies. 2. We propose a general framework for behaviour recognition using accelerometers, and develop a hybrid model where (a) biomechanical features characterise movement dynamics, and (b) a node-based hierarchical classification scheme employs simple machine learning algorithms at each node to find feature-value thresholds separating different behaviours. Using triaxial accelerometer data collected on 10 wild Kalahari meerkats, and annotated video recordings of each individual as groundtruth, this hybrid model was validated in three scenarios: (a) when each behaviour was equally represented (EQDIST), (b) when naturally imbalanced datasets were considered (STRAT) and (c) when data from new individuals were considered (LOIO). 3. A linear-kernel Support Vector Machine at each node of our classification scheme yielded an overall accuracy of >95% for each scenario. Our hybrid approach had a 2.7% better average overall accuracy than top-performing classical machine learning approaches. Further, we showed that not all models with high overall accuracy returned accurate behaviour-specific performance, and good performance during EQDIST did not always generalise to STRAT and LOIO. 4. Our hybrid model took advantage of robust machine learning algorithms for automatically estimating decision boundaries between behavioural classes. This not only achieved high classification performance but also permitted biomechanical interpretation of classification outcomes. The framework presented here provides the flexibility to adapt models to required levels of behavioural resolution, and has the potential to facilitate meaningful model sharing between studies.

Behavioural compass: animal behaviour recognition using magnetometers

Kamiar Aminian, Pritish Chakravarty, Maiki Maalberg

Animal-borne data loggers today often house several sensors recording simultaneously at high frequency. This offers opportunities to gain fine-scale insights into behaviour from individual-sensor as well as integrated multi-sensor data. In the context of behaviour recognition, even though accelerometers have been used extensively, magnetometers have recently been shown to detect specific behaviours that accelerometers miss. The prevalent constraint of limited training data necessitates the importance of identifying behaviours with high robustness to data from new individuals, and may require fusing data from both these sensors. However, no study yet has developed an end-to-end approach to recognise common animal behaviours such as foraging, locomotion, and resting from magnetometer data in a common classification framework capable of accommodating and comparing data from both sensors. We address this by first leveraging magnetometers’ similarity to accelerometers to develop biomechanical descriptors of movement: we use the static component given by sensor tilt with respect to Earth’s local magnetic field to estimate posture, and the dynamic component given by change in sensor tilt with time to characterise movement intensity and periodicity. We use these descriptors within an existing hybrid scheme that combines biomechanics and machine learning to recognise behaviour. We showcase the utility of our method on triaxial magnetometer data collected on ten wild Kalahari meerkats (Suricata suricatta), with annotated video recordings of each individual serving as groundtruth. Finally, we compare our results with accelerometer-based behaviour recognition. The overall recognition accuracy of > 94% obtained with magnetometer data was found to be comparable to that achieved using accelerometer data. Interestingly, higher robustness to inter-individual variability in dynamic behaviour was achieved with the magnetometer, while the accelerometer was better at estimating posture. Magnetometers were found to accurately identify common behaviours, and were particularly robust to dynamic behaviour recognition. The use of biomechanical considerations to summarise magnetometer data makes the hybrid scheme capable of accommodating data from either or both sensors within the same framework according to each sensor’s strengths. This provides future studies with a method to assess the added benefit of using magnetometers for behaviour recognition.

2019

Seek and learn: Automated identification of microevents in animal behaviour using envelopes of acceleration data and machine learning

Kamiar Aminian, Pritish Chakravarty, Hooman Dejnabadi

Animal‐borne accelerometers have been used across more than 120 species to infer biologically significant information such as energy expenditure and broad behavioural categories. While the accelerometer’s high sensitivity to movement and fast response times present the unprecedented opportunity to resolve fine‐scale behaviour, leveraging this opportunity will require overcoming the challenge of developing general, automated methods to analyse the nonstationary signals generated by nonlinear processes governing erratic, impulsive movement characteristic of fine‐scale behaviour. We address this issue by conceptualising fine‐scale behaviour in terms of characteristic microevents: impulsive movements producing brief (85% during leave‐one‐individual‐out cross‐validation, and exceeded that of the best classical machine learning approach by 8.6%. μEvId was found to be robust not only to inter‐individual variation but also to large changes in model parameters. Our results show that microevents can be modelled as impulse responses of the animal body‐and‐sensor system. The microevent detection step retains only informative regions of the signal, which results in the selection of discriminative features that reflect biomechanical differences between microevents. Moving‐window‐based classical machine learning approaches lack this prefiltering step, and were found to be suboptimal for capturing the nonstationary dynamics of the recorded signals. The general, automated technique of μEvId, together with existing models that can identify broad behavioural categories, provides future studies with a powerful toolkit to exploit the full potential of accelerometers for animal behaviour recognition.

2020

Learning to Detect Objects with Minimal Supervision

Karim Ali

Many classes of objects can now be successfully detected with statistical machine learning techniques. Faces, cars and pedestrians, have all been detected with low error rates by learning their appearance in a highly generic manner from extensive training sets. These recent advances have enabled the use of reliable object detection components in real systems, such as automatic face focusing functions on digital cameras. One key drawback of these methods, and the issue addressed here, is the prohibitive requirement that training sets contain thousands of manually annotated examples. We present three methods which make headway toward reducing labeling requirements and in turn, toward a tractable solution to the general detection problem. First, we propose a new learning strategy for object detection. The proposed scheme forgoes the need to train a collection of detectors dedicated to homogeneous families of poses, and instead learns a single classifier that has the inherent ability to deform based on the signal of interest. We train a detector with a standard AdaBoost procedure by using combinations of pose-indexed features and pose estimators. This allows the learning process to select and combine various estimates of the pose with features able to compensate for variations in pose without the need to label data for training or explore the pose space in testing. We validate our framework on three types of data: hand video sequences, aerial images of cars, as well as face images. We compare our method to a standard Boosting framework, with access to the same ground truth, and show a reduction in the false alarm rate of up to an order of magnitude. Where possible, we compare our method to the state-of-the art, which requires pose annotations of the training data, and demonstrate comparable performance. Second, we propose a new learning method which exploits temporal consistency to successfully learn a complex appearance model from a sparsely labeled training video. Our approach consists in iteratively improving an appearance-based model built with a Boosting procedure, and the reconstruction of trajectories corresponding to the motion of multiple targets. We demonstrate the efficiency of our procedure by learning a pedestrian detector from videos and a cell detector from microscopy image sequences. In both cases, our method is demonstrated to reduce the labeling requirement by one to two orders of magnitude. We show that in some instances, our method trained with sparse labels on a video sequence is able to outperform a standard learning procedure trained with the fully labeled sequence. Third, we propose a new active learning procedure which exploits the spatial structure of image data and queries entire scenes or frames of a video rather than individual examples. We extend the Query by Committee approach allowing it to characterize the most informative scenes that are to be selected for labeling. We show that an aggressive procedure which exhibits zero tolerance to target localization error performs as well as more sophisticated strategies taking into account the trade-off between missed detections and localization error. Finally, we combine this method with our two proposed approaches above and demonstrate that the resulting algorithm can properly perform car detection from a small set of annotated image as well as pedestrian detection from a handful of labeled video frames.