🏗 System Stack
Everything runs 100% in the browser — no video leaves your device.
MediaPipe FaceMesh — Google WASM, 30fps cap, 468 3D landmarks per face, up to 8 faces.
face-api.js — 4 TensorFlow.js models: TinyFaceDetector (bounding box) · AgeGenderNet (age ±3yr, gender after 10+ readings) · FaceRecognitionNet (128-dim descriptor) · FaceExpressionNet (7-class, async ~1/sec)
Three.js r160 — WebGL live 3D face mesh (898 triangles + back-of-head geometry)
localStorage — face library persisted in browser. Optional Python save_server.py (port 8787) saves photos to disk.
🔄 Per-Frame Pipeline
Every frame: MediaPipe → 468 landmarks → draw mesh overlay on canvas
Every 5th frame: Crop face → 128-dim descriptor → library match (d<0.50=match, d>0.58=new, d<0.35=high confidence) · Age+gender · Skin ITA (forehead+cheeks only, beard-safe) · Ethnicity (7 geometry ratios, ~55–65% accurate) · Hair+shirt colour signature (secondary verification)
Background ~1/sec: Expression analysis — never blocks frame loop
📊 Accuracy Reference
Skin ITA scale: Very Light >55° · Light 41–55° · Intermediate 28–41° · Tan-Brown 10–28° · Brown −30–10° · Dark Brown <−30°
Photos saved only when d<0.35. Open profile → ✕ to remove a photo · 💾 to save current live frame. Hair+shirt colour chips shown as identity markers.
Expressions are running averages since first detection — a brief expression dilutes slowly over time.
🔵 Status Indicators
disk: localStorage — Face library (names, descriptors, demographics) is saved to your browser's localStorage automatically. It persists between page reloads as long as you don't clear your browser data. Photos (full-res) are held in memory only while the tab is open; a 64px thumbnail is saved to localStorage so faces remain identifiable after reload.
disk: on — The optional Python server save_server.py is running on localhost:8787. When active, full-resolution face photos and the library are saved to disk in the project folder — they survive browser cache clears and can be backed up. Run it with: python3 save_server.py
WS: off — WebSocket output to external apps (e.g. smart glasses companion app) is disabled. Click the WS: off button to toggle. When on (WS: on), detected face data is streamed in real-time over WebSocket for use by other applications. This is the integration layer for the glasses assistant.
📐 Sampling Zones (Profile Debug View)
□ ITA (skin tone) — Forehead-only zones: top-centre, mid-forehead, inner brow left/right, temples. All above the eyeline — beard-safe. Uses ITA (Individual Typology Angle) from L*a*b* colour space.
□ Hair — Sampled just above the top-of-head landmark (lm[10]), 5% of frame height above the forehead peak = hairline area.
□ Shirt — Sampled 22% of frame height below the chin, offset left and right by 120% of face half-width to reach the shoulder/chest area — well clear of neck and beard. Used as a secondary identity check alongside the face descriptor.
⌨ Shortcuts
M Merge selected · Ctrl+Z Undo · Click card to select for merge