A Real-Time Computer Vision Framework for Skeletal Anomaly Detection Integrating Gaussian Filtering, LSTM, and XGBoost

Kinematic Framework for Real-Time Anomaly Detection

Abstract

Background: Real-time video anomaly detection in physical surveillance is still limited by the computational cost and illumination noise of traditional 3D Convolutional Neural Networks (CNNs). Detecting sudden kinematic changes, such as a physical struggle, a sudden loss of balance or an unexpected fall, requires lightweight solutions that run on edge hardware, both in security settings and in remote tele-rehabilitation monitoring.

Methods: This work proposes a vision-based skeletal kinematics approach that detects human keypoints in real time with YOLOv8-Pose. Frames are first smoothed with a 2D Gaussian filter, and keypoint coordinates are then converted into kinematic measures: joint velocities, acceleration peaks and joint angles. Temporal patterns were analyzed with an unsupervised Bi-directional Long Short-Term Memory (Bi-LSTM) Autoencoder and a statistical XGBoost classifier. Both models were evaluated on the Custom Binary Surveillance (CBS) dataset, which contains 2,268 standardized 150-frame windows extracted from public and proprietary surveillance videos.

Results: On the hold-out test set, the Bi-LSTM Autoencoder achieved an accuracy of 89.3% (95% CI 86.7% to 91.9%) and an anomaly recall of 91.3%. The full pipeline ran at 55 frames per second (18 ms latency) on an NVIDIA T4 GPU.

Conclusion: Extracting compact 1D kinematic signals from high-dimensional video pixels greatly reduces computation. This lightweight architecture suits edge deployment and may support patient fall screening and movement monitoring, pending validation in clinical settings.

DOI: https://doi.org/10.59564/amrj/04.04/010