CONDUCTOR

A project by Qalarc — AI systems studio, Sydney, Australia. Built, shipped and maintained by Qalarc.

A webcam gesture instrument — MediaPipe hand tracking turns your hands (and voice) into ~50 normalised control sources broadcast over the muse-bridge-v1 protocol to any consuming app. The performer's hands are the interface.

MediaPipe tasks-visionBroadcastChannel (muse-bridge-v1)postMessage APIWeb Audio APIVanilla JS (ES Modules)Chrome DevTools Protocol (headless integration tests)

CONDUCTOR is a control surface, not a renderer: it watches your hands (and optionally your voice) and broadcasts ~50 normalised sources — pinch aperture, per-finger curl, spread, palm roll, pitch, onsets — to other apps over the muse-bridge-v1 protocol. Think of it as a MIDI controller whose knobs are your fingers. It does not make sound or pictures itself; the consuming app decides what the numbers mean. The primary consumer is MUSEALL, where the binding wizard turns gesture chips into live effect parameters at ~30 Hz.

The bridge speaks BroadcastChannel for same-origin tabs (zero setup) and postMessage for cross-origin windows, with byte-identical payloads across both. Heartbeats keep peers honest — close the CONDUCTOR tab and bound values decay to zero over ~600 ms instead of sticking. A voice module adds pitch tracking, onset detection and an FX chain; a skeleton overlay renders the tracked hands on top of the camera feed.

It is a reimagining of the gesture-mapper v5.x demo: the gesture maths is ported (muscle memory carries over) but everything around it is new — modern MediaPipe Tasks, ES modules, no build step. Integration-tested against MUSEALL v3.3 headless via CDP, 18/18 green. Open source; the MUSEALL visualiser half of the suite is closed.

Capabilities

~50 normalised control sources from both hands — pinch, per-finger curl, spread, palm roll, position, velocity
Voice module — pitch tracking, onset detection, FX chain, monitoring
Skeleton overlay rendering tracked hands over the camera feed
muse-bridge-v1 protocol — BroadcastChannel same-origin + postMessage cross-origin, byte-identical payloads
Heartbeat + decay: dead peers fade bound values to zero (~600 ms) instead of freezing them
MUSEALL consumer shipped — binding wizard maps gesture/voice chips to any effect param at ~30 Hz
No build step, no dependencies — ES modules served straight from disk
Integration-tested headless (CDP): 18/18 green, zero console errors

CONDUCTOR

Capabilities

Tags