Building a Robot Hand That Can Watch Itself Move

A 3D-printed tendon-driven hand that mirrors your movements — with a second camera that checks whether the robot actually did what it was told.

The Idea

Point a webcam at your hand. The AI reads your finger positions 30 times a second. A 3D-printed robot hand copies them in real time.

That's teleoperation — remote control using your own body as the input. It's been done in robotics labs with expensive hardware. The interesting question is: can you do it with a 3D printer and off-the-shelf servo motors?

The more interesting question is: how does the robot know when it's wrong?

A second webcam watches the robot hand. The same AI that reads your hand also reads the robot's hand — because the robot is designed to look like a real hand. The system compares what you told the robot to do against what it actually did, finger by finger, and logs where it's failing.

This closed-loop feedback is what separates "a robot that moves" from "a robot that learns where it's failing."

Why Score 20 Designs Before Printing Anything

There are dozens of open-source 3D-printable robot hands. Before printing a single piece, every candidate was evaluated across six criteria in a weighted scoring matrix.

The winner was the InMoov i2 by French sculptor Gaël Langevin. InMoov is unusual in the robot-hand world because it was sculpted starting from real human anatomy. The finger proportions, knuckle shapes, and palm geometry are based on actual hand measurements.

Why does that matter here? Because the AI tracking system (MediaPipe Hands) was trained on millions of photographs of human hands. It learned to find hands by recognising shapes it had seen before. A mechanical-looking robotic hand — the kind that resembles an industrial gripper — sits completely outside the training distribution. The AI won't find landmarks on it.

InMoov i2 sits close enough to real human anatomy that the AI finds it and tracks it reliably. The design choice becomes a machine learning argument.

Design Choice as an AI Argument

This is worth dwelling on. Most robot hand projects optimise for: strength, speed, range of motion, cost, printability. Appearance is aesthetic.

Here, appearance is functional. If the robot hand doesn't look enough like a human hand, the feedback loop doesn't work. The tracking fails. The whole closed-loop architecture collapses.

This drove several material and finish choices. Finger segments are printed in matte skin-tone PLA. Matte because reflective surfaces confuse camera tracking. Skin tone because it reduces false positives from background objects. The material and colour choices were partly driven by asking: what does MediaPipe need to see in order to reliably find this hand?

Structural parts use PETG — stronger and slightly flexible, better for high-stress mechanical joints where appearance doesn't affect tracking.

How the Feedback Loop Works

Two cameras, one AI model, one comparison:

1. Camera 1 (pointing at you) reads 21 landmarks on your hand 30 times per second, each with 3D position.
2. Software converts landmark positions into finger angles — how curled each finger should be.
3. Those angles become servo commands. The motors move.
4. Camera 2 (pointing at the robot) reads the robot hand's landmarks.
5. Software compares operator landmarks vs robot landmarks. The difference per finger is the error.
6. A dashboard shows both hands side by side and plots error over time.

The error data can eventually be used for automatic correction. If the robot's ring finger is consistently 15° behind target, the system compensates by increasing the motor command. This closed-loop control is standard in industrial robotics — applied here to a printed hand with commodity motors.

The servos also report back their own position, temperature, and whether they're stalled. A stall means the motor is trying to move but can't — a blocked tendon, a jammed joint. That's another error signal.

How the Fingers Move

InMoov uses tendons — 0.4mm Dyneema braided line running through PTFE tubes inside each finger segment, connected to a motor spool at the wrist. Pull the tendon → finger curls. Release → a spring extends it back.

Five fingers, ten servos (one to curl, one to extend each), all daisy-chained on a single serial cable. The FEETECH STS3215 serial bus servo uses one wire for all motors — each has an address. Send a command with an address and an angle. One cable, ten motors.

These are the same servos used in the HuggingFace LeRobot SO-ARM100 robot arm, so there's active open-source community support and existing Python libraries.

Where It Is Now

Software architecture complete. Documentation thorough. Hardware owned: Bambu printer, all six STS3215 servos, driver board, materials.

The next step is printing and testing a single index finger — one motor, one tendon, one camera test. The critical question: does MediaPipe actually track the printed finger, or does PLA at 0.15mm layer resolution fall outside the model's tracking tolerance?

That test hasn't happened yet. The viability report predicts it will work based on InMoov's design heritage. But until the camera points at a real printed finger and the landmarks appear, it's still a prediction. This post will be updated when the first tracking result is confirmed.