BoxGym is a fusion of computer vision and gamification that turns your phone or laptop into an interactive boxing trainer. The goal was an app that is immediately accessible on all mobile devices in the browser — no app store downloads, no installations, just open and punch.

The implementation leverages modern web APIs — MediaPipe, Web Audio, Canvas — wrapped in a clean React architecture with a custom game state machine that coordinates the workout experience.

What it does

The device's camera and machine learning detect your body movements in real time. You see your live video feed with a skeleton pose overlay, and follow punch combinations shown as colored circles — red for a left punch, blue for a right punch. Incoming gloves have to be dodged by moving your head left or right, and workouts are structured into multiple rounds with rest periods.

Sound carries the workout: punch sounds on every hit, bells between rounds and beeps for countdowns give immediate, immersive feedback.

Key challenges solved

1. 1

   Real-time pose detection in the browser. MediaPipe's BlazePose model with the TensorFlow.js WebGL backend achieves 30+ FPS tracking without any native code.

2. 2

   Mapping body movements to game mechanics. A custom hit-detection algorithm calculates punch trajectories and compares them against target zones.

3. 3

   Cross-device compatibility. Responsive canvas rendering adapts to any screen size while maintaining accurate pose-to-screen coordinate mapping.

4. 4

   Audio synchronization. The Web Audio API with pre-loaded sound sprites provides instant, lag-free audio feedback.

Everything runs entirely in the browser: MediaPipe's BlazePose model handles the pose estimation, TensorFlow.js provides WebGL-accelerated inference for smooth tracking, and React keeps the game logic maintainable.