AI Hand Interaction System
A real-time 3D particle simulation controlled by hand gestures via your webcam. 1,000 particles morph between 5 mathematical shapes and react to your pinch and spread gestures with neon post-processing effects.
β Click "Start Hand Tracking" to try it live β’ Use pinch and spread gestures
// How to Interact
Hand Gesture Controls
Idle / Relaxed Hand
Particles drift gently in shape
Pinch (Thumb + Index)
Particles contract and implode to center
Wide Open Hand
Particles explode outward with chaotic energy
// Mathematical Morphology
5 Procedural Shapes
Heart
x = 16sinΒ³(t), y = 13cos(t) β 5cos(2t) β 2cos(3t) β cos(4t)
Classic parametric heart curve with volumetric depth
Saturn
Sphere + tilted ring (r = 13β19, tilt = Ο/6)
Spherical planet body with a flat, tilted particle ring orbiting around it
Flower
r = 10Β·sin(3u)Β·sin(v) β Harmonic Rose
3D harmonic rose curve that creates petal-like formations
Buddha
Head sphere + body ellipsoid + base ring
Abstract seated figure constructed from 3 mathematical primitives
Firework
Random spherical distribution (r β [0, 18])
Explosive random scatter simulating a fireworks burst
// Pipeline
How It Works
Webcam Feed
Camera captures 640Γ480 video frames at ~30fps
MediaPipe AI
Detects 21 hand landmarks in real-time using ML
Gesture Math
Calculates pinch distance between thumb & index tip
Particle Physics
Maps gesture intensity to particle scale & explosion force
// Visual Effects
Post-Processing Pipeline
// Code Sample
Heart Shape Generator
function calculateShape(type) {
for (let i = 0; i < particleCount; i++) {
const i3 = i * 3;
let x, y, z;
if (type === 'Heart') {
const t = Math.random() * Math.PI * 2;
const thick = (Math.random() - 0.5) * 5;
// Parametric heart equations
x = 16 * Math.pow(Math.sin(t), 3);
y = 13 * Math.cos(t)
- 5 * Math.cos(2 * t)
- 2 * Math.cos(3 * t)
- Math.cos(4 * t);
z = thick; // volumetric depth
x *= 0.5;
y *= 0.5;
}
targetArray[i3] = x;
targetArray[i3 + 1] = y;
targetArray[i3 + 2] = z;
}
}// Technical Deep Dive
Challenges & Solutions
Real-Time Hand Detection Performance
Problem
Running MediaPipe hand tracking at 30fps while simultaneously rendering 1,000 particles with post-processing effects caused frame drops
Solution
Used modelComplexity: 0 for lightweight detection, capped pixel ratio at 1.5, and disabled WebGL antialiasing since the bloom pass provides its own smoothing
Mapping 2D Gestures to 3D Space
Problem
MediaPipe returns 2D normalized coordinates (0-1) for hand landmarks, but the particle system lives in 3D space with arbitrary camera angles
Solution
Instead of mapping position, I mapped the gesture intensity (pinch distance between thumb and index finger) to a scalar value that controls particle scale and explosion force
Smooth Shape Transitions
Problem
Snapping particles instantly to new target positions looked jarring when switching shapes
Solution
Each frame interpolates particle positions toward targets using lerp (posArray += (target - posArray) * 0.05), creating smooth morphing transitions between any two shapes
Try It Yourself
The full source code is a single HTML file β open it with a local server and grant camera access. Have fun! π
Enjoyed this project? Consider supporting my work β
βBuy Me a Coffee