Text2CAD and the Rise of Language-Model-Driven Engineering

The idea of generating CAD from a prompt didn't start with this project, and it's worth being explicit about the research lineage we're building on rather than pretending the approach appeared from nowhere. Two datasets in particular shaped the decision to treat CAD generation as a code-generation problem rather than an image-generation one: DeepCAD and Text2CAD.

Where the training data actually comes from

DeepCAD is a dataset of parametric construction sequences — sketch-and-extrude operation histories — mined from real designs on Onshape, a cloud CAD platform. Crucially, these aren't renders or photographs of finished parts; they're the actual sequence of operations a designer performed to build the part, which is exactly the kind of structured, procedural information a code-generation approach needs. Text2CAD, presented at NeurIPS 2024, builds directly on top of DeepCAD's geometry by adding text annotations at multiple levels of abstraction — from an abstract one-line description of a part down to a detailed, feature-by-feature prompt describing each operation. That range matters for training a model meant to handle real user phrasing, which spans from "a bracket with two mounting holes" to a precise, dimensioned specification.

Our transpiler work sits directly downstream of this lineage: it converts DeepCAD's construction sequences into CadQuery so that Text2CAD's prompts can be paired with executable programs instead of a proprietary JSON schema. Fusion360 Gallery and the ABC dataset play supporting roles — Fusion360 Gallery as a distribution-shift evaluation set held out from training, and ABC as a pretraining source for the image branch — but DeepCAD and Text2CAD are the backbone of the supervision itself.

Why a code-generation model, not a diffusion model

The more consequential decision is what kind of model to fine-tune on this data. The current default instinct for "text to 3D" is a diffusion model, typically trained on rendered images, that learns to hallucinate plausible-looking geometry. We didn't go that direction. The base model is Qwen2.5-Coder-1.5B-Instruct — a model pretrained specifically on code — and the reasoning is about what kind of prior a model needs walking in.

A CadQuery program is Python. It has strict syntax, it composes through function calls and method chaining, and producing it correctly requires the same underlying skill as producing any other piece of correct code: tracking state (what shape exists so far), applying operations in a valid order, and respecting a real grammar where one wrong character breaks execution entirely. A code-generation model has already learned all of that from a much larger, more general pretraining diet of real code before we ever show it a CAD sequence. Fine-tuning transfers those priors onto a narrower target rather than asking the model to learn "how to write syntactically valid, well-structured programs" and "what a bracket with two mounting holes looks like as geometry" simultaneously from a comparatively small CAD-specific dataset.

A diffusion model trained on renders learns something different: what a shape looks like from the outside. It has no natural mechanism for expressing "this dimension is driven by that dimension" or "this hole must be concentric with that boss," because renders don't encode construction history — only appearance. That's a fine prior for photorealistic imagery and arguably fine for organic or artistic 3D content, but it's a mismatch for parametric, feature-based mechanical design, which is defined by the history of operations, not just the final silhouette.

Engineering tools are becoming software-generation tools

There's a broader pattern here worth naming. Parametric CAD has always been closer to programming than most people notice — a feature tree is a sequence of operations with dependencies, not unlike a call graph, and CAD kernels like OpenCascade already treat models as programs to be evaluated. Framing CAD generation as a code-generation problem isn't a trick to make an LLM applicable where it doesn't belong; it's a recognition that parametric CAD already is a kind of program, and the tools built to generate and understand code are a natural fit for generating and understanding it. That's the bet underlying the choice of base model, and it's a big part of why this project looks more like a code-generation fine-tune than a 3D-graphics research project.