Inside the Round-Trip Filter: How We Guarantee Training Data Quality

Every training pair SolidMake's text branch learns from starts as a DeepCAD parametric construction sequence, gets parsed into a typed intermediate representation, and gets emitted as a CadQuery Python program. That transpilation step is deterministic code, and deterministic code has bugs like any other code — the emitter can get a boolean operation backwards, misplace a sketch plane, or round a coordinate the wrong way. If a subtly wrong CadQuery program enters the training set labeled as correct, the model learns from bad supervision, and the failure is invisible until an eval run months later comes back worse than expected. The round-trip filter exists to catch this before a single training pair is trusted.

How the check actually works

After transpiling a DeepCAD sequence to CadQuery, the pipeline executes the emitted program in a subprocess and exports the resulting geometry as a STEP file. That geometry gets compared against DeepCAD's own reconstruction of the identical sequence — the ground-truth B-Rep the dataset ships alongside the JSON — using two independent signals.

The first is topology counts: the number of solids, faces, edges, and vertices in each shape. These have to match exactly. The second is Chamfer distance, computed by uniformly sampling points across both surfaces and measuring the average nearest-neighbor distance between the two point clouds, with a tolerance of 0.5 millimeters (DeepCAD models are normalized into a 100 mm bounding box, so that's about half a percent of the part's scale). A pair only survives the filter if both checks pass — topology matches exactly and Chamfer distance falls under tolerance.

Why two signals instead of one

Either check alone would let real errors slip through. Topology counts are cheap and exact, but two shapes can have identical solid/face/edge/vertex counts while being geometrically quite different — a hole cut 2mm off-center has the same topology as a correctly centered one, but it's the wrong part. Chamfer distance alone has the opposite blind spot: it's a bulk distance metric, so it can be fooled by a shape that's dimensionally close on average but structurally wrong — say, missing a small feature that barely moves the average point-to-surface distance but changes what the part actually is.

Running both closes each other's gap. Topology counts catch structural disagreements — wrong number of features, a boolean that dropped a solid — that a distance metric might not flag if the bulk shapes happen to look similar. Chamfer distance catches geometric disagreements — right structure, wrong dimensions — that a topology count, which only counts things rather than measuring them, is blind to by construction. A pair has to pass both to be trusted, which is a meaningfully stricter bar than either check applied alone.

What the rejection rate tells us

Based on the slice-level results referenced in the M0 transpiler plan, we expect roughly 90% of the source corpus to survive this filter — something on the order of 140,000 of Text2CAD's 158,000 pairs, once the transpiler is complete and run at scale. Read that the right way: about one pair in ten gets thrown away, and that's evidence the filter is doing real work rather than existing as a compliance checkbox.

If the round-trip filter passed 100% of everything we fed it, that wouldn't mean the transpiler was flawless — it would mean the filter probably wasn't actually checking anything meaningful, or the tolerances were set so loose they couldn't catch a real disagreement. A double-digit rejection rate is what you'd expect from a genuinely deterministic pipeline translating between two different representations of geometry: edge cases in curve types, degenerate sketches, boolean sequences that hit ambiguous cases in the emitter. The filter's job isn't to make that number zero. It's to make sure that whatever number it lands on, everything on the other side of it is actually correct — because a model trained on unverified data doesn't fail loudly. It fails quietly, weeks later, in an eval report nobody can explain.