Fine-Tuning a 1.5B Code Model on a Single Colab GPU

Fine-tuning plans in papers tend to assume compute you don't have — a rack of A100s, a training run measured in single-digit hours because nobody is sharing the cluster. The actual constraint on this project was a free-tier Google Colab T4, roughly 15GB of VRAM, with no guarantee of a stable, uninterrupted session. Every decision in the training plan works backward from that constraint rather than from what would be ideal with unlimited compute.

Why 1.5B and not 7B

The base model is Qwen2.5-Coder-1.5B-Instruct, and the size is not incidental. A 1.5-billion-parameter model, loaded in 4-bit quantization via QLoRA, fits comfortably within a free T4's memory budget alongside activations, optimizer state for the LoRA adapters, and a reasonable sequence length. A 7B model in the same setup is a much tighter fit at best and simply doesn't fit at worst, and the design explicitly treats a jump to Qwen2.5-Coder-7B as an escalation path — something to reach for on rented A100 compute only if the smaller model's geometric fidelity turns out to be inadequate, not the default plan. Scoping the base model to the hardware actually available, rather than the hardware that would make the problem easiest, was a deliberate trade-off: it caps how much capacity the model has to work with, but it keeps the entire training loop runnable on infrastructure that costs nothing and that a student project can actually rely on.

The QLoRA configuration

QLoRA — quantized low-rank adaptation — is what makes fine-tuning a language model on a single consumer-grade GPU practical at all: the base model's weights are loaded in 4-bit precision and frozen, and only small, low-rank adapter matrices are trained on top. The specific configuration: 4-bit NF4 quantization, LoRA rank 32, alpha 64, applied to the attention and MLP projection layers (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj) — effectively every linear projection in each transformer block, which gives the adapters enough surface area to meaningfully shift the model's behavior without touching the frozen base weights. Sequence length is capped at 2048 tokens with a per-device batch size of 1 and gradient accumulation of 16, another concession to available memory: real throughput comes from accumulating gradients across many small steps rather than one large batch that wouldn't fit.

Sanity-overfit before the expensive run

Before committing to the full training run — an estimated 14 to 18 hours spread across two or three Colab sessions — the plan calls for a much cheaper sanity check first: overfitting the same setup on Seek-CAD, a smaller 23,000-example dataset, for a single epoch, taking roughly 4 hours. The point of this step isn't to produce a useful model; it's to produce a fast, cheap failure signal. If the training loop, the loss masking, the tokenization, or the LoRA configuration has a bug, that bug will show up as a model that fails to overfit a small dataset it's seen repeatedly — and it's far better to discover that in 4 hours than partway through an 18-hour run, or worse, at the very end of one. This is a standard ML engineering discipline, but it's especially load-bearing here because there's no budget to redo an 18-hour run casually on free-tier compute with session limits.

Checkpointing as a survival strategy, not an optimization

Free Colab sessions don't run indefinitely — they're subject to time limits and can disconnect without warning, which is the single biggest practical risk to a multi-session training plan. The mitigation is checkpointing every 1000 steps directly to Google Drive, with the training loop written to be resume-safe: a session that gets cut off mid-run picks back up from the last saved checkpoint rather than starting over. Kaggle's P100 instances serve as a backup compute source if Colab access becomes unreliable during a critical stretch. None of this is sophisticated infrastructure — it's the training-loop equivalent of saving your work often — but it's the difference between a training plan that survives contact with free-tier infrastructure and one that doesn't.

Scoping to the compute you have

The through-line across all of these decisions — model size, QLoRA rank, sequence length, the overfit sanity check, the checkpointing cadence — is the same discipline applied at every level: figure out what compute is actually available, and design the entire training plan to fit inside it rather than design an ideal plan and hope the compute shows up. That's a less glamorous way to do machine learning than describing a training run in a paper's appendix as if the GPUs were a given, but it's the version of the plan that's actually executable on a free-tier T4, which is the compute this project has.