Egocentric stereo human-demonstration data is incomplete without a calibration package. Dual-eye RGB, IMU, and poses may look ready for training, but stereo depth, triangulation, and multi-sensor fusion all depend on the matrices and extrinsics that describe how the sensors were measured. If those products are missing, versioned poorly, or delivered as a vague “calibration available on request,” buyers cannot reproduce geometry, auditors cannot close acceptance, and model teams burn cycles reverse-engineering the capture rig.
This guide answers a practical procurement question: what calibration files must ship with egocentric stereo data for a delivery to count as complete? It is written for robot-learning, imitation-learning, and foundation-model buyers who need paste-ready SOW language—not marketing adjectives. The numbers and products below come from a public sample you can inspect yourself: two episodes, 4,914 frames, 163.47 seconds total, published as LeRobot v3 on Hugging Face under a documented schema that also aims to pass Physical-Intelligence/pi-data-sharing checks. The optical envelope of that sample is dual-eye colour at 1920×1200 per eye, hardware-synced stereo, with timestamps recorded on device and a continuous IMU near 300.48 Hz.
Why “complete calibration” is a delivery requirement, not a footnote
Stereo pipelines assume a known baseline, known intrinsics, and known relative pose between the two eyes. Egocentric pipelines add another layer: the IMU and camera must be registered so head motion and pixels share a geometric model. Without shipping those parameters alongside the episodes, a buyer receives pixels and motion traces that cannot be fused under a single consistent camera model.
Three failure modes show up repeatedly in egocentric stereo procurement:
- Video-only delivery. Frames arrive without K, distortion model, rectification matrices, or a Q matrix for disparity-to-depth. Downstream teams cannot run a deterministic stereo pipeline.
- Calibration without versioning. Parameters exist somewhere in a shared drive, but there is no calibration version tied to the batch. Months later, nobody knows which matrix set belongs to which recording window.
- Camera-only calibration. Stereo products are present, but IMU↔camera extrinsics are missing. Visual-inertial fusion and head-pose alignment then become guesswork.
Completeness means the package ships with the episode data, is named explicitly in the SOW, and includes a version identifier so drift across production months is auditable.
The minimum stereo product set: K, KB4, R1/R2, P1/P2, Q
For egocentric stereo, “intrinsics and extrinsics” is too loose. Name the products. A complete stereo calibration delivery should include at least:
| Product | Role in the pipeline |
|---|---|
| K | Camera intrinsic matrix (focal length, principal point) for each eye |
| KB4 | Distortion model parameters (KB4 convention) used to undistort or model the lens |
| R1 / R2 | Rectification rotation matrices that align the stereo pair into a common epipolar geometry |
| P1 / P2 | Projection matrices in the rectified coordinate system |
| Q | Disparity-to-depth mapping matrix used when converting stereo disparity into metric depth |
Why these specifically? Stereo depth and triangulation are not “approximate alignment” problems. They depend on a consistent chain from raw pixels → undistortion → rectification → disparity → depth. Omitting Q forces every consumer to re-derive disparity-to-depth from incomplete metadata. Omitting R1/R2 and P1/P2 leaves rectification undefined. Shipping K without the distortion model (here, KB4) breaks any pipeline that expects the same lens model used at calibration time.
Buyer tip: Write the SOW so each of K, KB4, R1, R2, P1, P2, and Q is a named deliverable, not a bullet under “calibration files as applicable.”
IMU↔camera extrinsics and calibration version
Egocentric capture is head-worn. The IMU measures motion at approximately 300.48 Hz in the public sample context, while the cameras deliver hardware-synced stereo frames with timestamps recorded rather than backfilled from a nominal FPS. That temporal story only becomes a geometric story when IMU↔camera extrinsics ship with the batch.
Without IMU↔camera extrinsics:
- 6-DoF head pose and stereo pixels sit on parallel timelines that never meet in one rigid transform.
- Visual-inertial odometry and fusion experiments cannot reproduce the capture geometry.
- Acceptance reviews cannot verify that pose and RGB belong to the same sensor package.
Equally important is a calibration version identifier. Production capture spans weeks or months. Lenses settle, mounts are re-torqued, and recalibration events happen. A version string (or equivalent package ID) tied to each delivered batch lets QA answer: “Which calibration was active when these episodes were recorded?” Without versioning, two batches that look identical in schema may silently use different geometric models.
Recommended SOW language
- Ship IMU↔camera extrinsics with every accepted batch.
- Ship a calibration version (or package ID) that uniquely identifies the parameter set used for that batch.
- Do not accept “latest calibration on the shared drive” as a substitute for a versioned artifact next to the episode files.
Geometric quality gates that belong next to the files
Shipping matrices is necessary but not sufficient. Buyers also need residual evidence that the calibration is usable. From the public sample’s reported geometric checks, three figures are concrete enough to put in an SOW without inventing new thresholds:
- Stereo baseline approximately 60.7 mm
- Sampson median 0.166 px
- Stereo reprojection 0.800 px
- 15,811 inliers (at the public sample’s calibration check scale)
These are not universal industry laws; they are documented outcomes from a inspectable sample (2 episodes / 4,914 frames / 163.47 s). Use them as reference gates when the vendor’s capture envelope matches that sample’s optical class—1920×1200 per eye, hardware-synced stereo—rather than as invented pass/fail numbers for unrelated devices or library footage.
Recommended SOW language
- Report stereo baseline with the calibration package (reference class: ~60.7 mm).
- Report Sampson median and stereo reprojection residuals with inlier count for the calibration check (reference sample: 0.166 px Sampson median, 0.800 px stereo reprojection, 15,811 inliers).
- Tie residual reports to the same calibration version that ships with the episodes.
Do not claim that every clip in a mixed media library was recorded on the same device. Completeness here means: for this egocentric stereo delivery, the calibration products and residual report travel with the data.
How calibration completeness fits the rest of the delivery envelope
Calibration files do not sit in isolation. They only make sense inside a coherent capture and schema envelope. For the public sample referenced throughout this series:
- Video: dual-eye colour RGB at 1920×1200 per eye, hardware-synced stereo.
- Time: timestamps recorded on device (not FPS-backfilled).
- IMU: measured near 300.48 Hz, continuous with the visual stream.
- Schema: LeRobot v3, intended to load cleanly and to align with Physical-Intelligence/pi-data-sharing expectations.
- Inspectability: public sample of 2 episodes / 4,914 frames / 163.47 s on Hugging Face (
tatezhou/ego-stereo-cn-v1), with tooling at ego-stereo-cn-v1-tools.
When calibration products are missing, even a perfect LeRobot layout still fails geometric consumers. When the schema is wrong, perfect matrices are hard to attach to the right frames. Write both into the same acceptance block.
Paste-ready deliverable checklist
Copy the block below into an SOW, RFP, or vendor questionnaire. Adjust commercial terms as needed; keep the named products.
EGOCENTRIC STEREO — CALIBRATION DELIVERABLE CHECKLIST
Batch / PO ID: ____________________ Delivery date: __________
A. Stereo camera products (must all ship with the episode data)
[ ] K (intrinsics) for each eye
[ ] KB4 distortion parameters
[ ] R1 / R2 rectification rotations
[ ] P1 / P2 rectified projection matrices
[ ] Q (disparity-to-depth) matrix
B. Multi-sensor registration
[ ] IMU ↔ camera extrinsics
[ ] Calibration version / package ID recorded on the batch manifest
C. Geometric quality report (tied to the same calibration version)
[ ] Stereo baseline reported (reference class ~60.7 mm)
[ ] Sampson median residual reported (reference sample 0.166 px)
[ ] Stereo reprojection residual reported (reference sample 0.800 px)
[ ] Inlier count reported (reference sample 15,811 inliers)
D. Capture envelope consistency (same delivery)
[ ] Dual-eye RGB 1920×1200 per eye
[ ] Hardware-synced stereo
[ ] Timestamps recorded (device-side; not FPS-backfilled)
[ ] IMU near 300.48 Hz with continuous stream documentation
E. Dataset schema
[ ] LeRobot v3 layout
[ ] Compatible with Physical-Intelligence/pi-data-sharing checks
[ ] Calibration artifacts linked from the dataset manifest / README
F. Sign-off
Vendor: __________________ Date: ______
Buyer QA: ________________ Date: ______
Pass / Fail / Conditional: ______________
Notes: _________________________________
If any box in sections A–B is unchecked, treat the delivery as incomplete even if video and IMU files are present. Sections C–E are the quality and schema envelope that make the calibration usable in training and audit.
What “incomplete” looks like in practice
Reject or hold these patterns:
- A ZIP of left/right MP4s with a README that says “OpenCV stereo calibrate offline.”
- Intrinsics only (K) without KB4, rectification, or Q.
- A single
calib.yamlwith no calibration version and no link from the LeRobot dataset card. - Stereo products without IMU↔camera extrinsics when the SOW includes IMU and head pose.
- Residual metrics quoted from a different device class, or thresholds invented beyond the documented sample.
Accept only when the named products ship with the data, the version is recorded, and the residual report matches the calibration package you are accepting.
Soft next step
If you are drafting an SOW for egocentric stereo human demos and want a reference package to compare against, inspect the public sample (2 episodes / 4,914 frames / 163.47 s) on Hugging Face at tatezhou/ego-stereo-cn-v1, review the companion tools at github.com/TateZhouSiu/ego-stereo-cn-v1-tools, or browse capture notes at 10khoursdata.com. For commercial delivery scoping—calibration packaging, LeRobot v3 layout, and acceptance language—email tate@10khoursdata.com. No hard sell: the checklist above is usable whether or not you buy from 10K HoursData.
10K HoursData — egocentric stereo human-demonstration data with recorded timestamps, hardware-synced dual-eye capture, and versioned calibration products intended to ship with every complete batch.