The Data
What We Capture
Every session is a synchronized, multimodal record of skilled work — what the expert saw, how their hands moved, what they touched and why.
Strip ¾″ of insulationGrip force · 14 NTool · wire stripperEgocentric Video
Stereo 60 fps video and depth from the tradesperson's point of view
3D Hand Pose
Full articulated pose for both hands, every joint, every frame
Touch & Force
Fingertip and palm pressure that shows how hard — and where — they grip
Narration & Labels
Spoken intent plus expert-reviewed task, tool and outcome annotations
Channels
Every order lists exactly which channels are delivered. Raw channels come straight from the sensors; derived channels are computed from them and documented in the manifest.
| Channel | Source | Raw or derived |
|---|---|---|
| Egocentric video | Head-mounted camera | Raw |
| Wrist video | Camera on the dominant forearm | Raw |
| Fixed operator view | Tripod camera framed on the work area (bench cells only) | Raw |
| Audio | Head camera microphone, with third-party speech muted | Raw, redacted |
| Head and wrist IMU | Camera-embedded gyroscope and accelerometer | Raw |
| Fingertip pressure | Glove force sensors, as raw counts and calibrated newtons | Raw and derived |
| Calibration record | Per-session calibration against a reference load cell | Raw and derived |
| Hand pose | Markerless estimation from video: 21 keypoints per hand with confidence | Derived |
| Timestamps and sync log | One session clock with sync events at start and end | Raw |
| Labels | Task, episode and step boundaries, outcome, tool and material tags | Derived (human) |
| Metadata | Contributor ID, credential, handedness, experience, glove size, site ID, rig serials | — |
How it's delivered
- One directory per episode, with raw/ and derived/ folders
- A manifest listing every file's channel, rate, units, raw or derived status, producing software and SHA-256 checksum
- Every channel aligned to a single session clock, with residual drift reported per episode
- Video as MP4, sensor streams as CSV or Parquet, labels as JSON, or in your own schema
- Privacy processing applied, and contributors identified only by a pseudonymous ID
Train on the Real Thing
Tell us what your robots need to learn. We'll design a custom capture program around your tasks, channels and schema.