Knowing where a hand touches the world, and how close each part of it is to touching, matters for grasp understanding, AR/VR and robot learning from video. We present JointHand, which predicts for every vertex of the MANO hand mesh both a contact probability and a metric hand–object distance from a single RGB image. JointHand sits on a frozen WiLoR hand-reconstruction backbone: each mesh vertex becomes a query that attends to the image patches, and separate contact and distance branches are coupled through a gated, detached injection, so the distance estimate sharpens contact without destabilising it. Trained on a large mix of hand-interaction datasets, JointHand improves per-vertex contact F1 over the state-of-the-art HACO on ContactPose, EPIC-Contact, MOW, ARCTIC and RICH, including the two datasets held out from training, while HACO remains stronger on Hi4D and HIC. The distance read-out also gives a continuous proximity field rather than a binary mask.
Pick an example, move the thresholds and switch the read-out. Every view below is recomputed in your browser from the model's per-vertex outputs, so you can see exactly where JointHand and HACO agree, where they differ, and how precision, recall and F1 respond. The benchmark examples include cases where HACO does better.
Benchmark examples are the evaluation loader's own input crops from ContactPose, EPIC-Contact (both held out from training), ARCTIC and RICH, chosen as a mix of JointHand wins, ties and HACO wins at the paper's read-out. Predictions come from the released checkpoints run in fp32. The paper evaluated JointHand under bf16 autocast, so an individual sample's F1 can differ from the paper's by a few hundredths. In-the-wild images use the WiLoR hand detector, exactly as in the live demo.
Each frame is processed independently. Top row: contact drawn on the image; bottom row: the same prediction on the canonical hand. Choose a clip below.
Per-vertex contact on seven test sets (Skip protocol: fully non-contacting hands are excluded, as in HACO). Both models are scored by the same script on identical samples. HACO uses its public checkpoint. Hover a bar for precision and recall.
Standard threshold: JointHand reads contact OR distance at t = 0.52 / d < 12 mm where the distance head has signal, and the contact head at t = 0.52 elsewhere. HACO uses t = 0.52. Hi4D is scored against regenerated contact labels, so it is not comparable with published Hi4D numbers. On MOW our crops used a YOLO hand box (expand 1.3) and HACO's did not.
Dataset averages hide how often each model wins. Every dot is one test sample: above the diagonal JointHand is better, below it HACO is. Widen the margin to see how robust the split is.
3,048 scored test samples: ARCTIC (1,200), ContactPose (1,200), EPIC-Contact (237) and RICH (411), with the paper's read-out for each dataset. HACO does better on Hi4D and HIC (see the chart above), which are not shown here.
A live demo that runs JointHand and HACO side by side on your own photos and short clips is coming to Hugging Face. Until then, the interactive examples above run entirely in your browser, and the model weights are already public.
WiLoR: End-to-end 3D hand localization and reconstruction in-the-wild, the frozen backbone JointHand builds on.
HACO: Learning Dense Hand Contact Estimation from Imbalanced Data (NeurIPS 2025), the baseline compared throughout.
DECO: Dense Estimation of 3D Human-Scene Contact in the Wild.
MANO: Embodied Hands, the hand model all predictions live on.
@article{xue2026jointhand,
title = {JointHand: Dense Hand Contact and Hand--Object Distance from a Single Image},
author = {Xue, Junhan and Potamias, Rolandos Alexandros},
journal = {arXiv preprint arXiv:XXXX.XXXXX},
year = {2026}
}