Pricing products with a 3B model on one consumer GPU

Llama-3.2-3B, quantized to 4 bits, with a LoRA adapter trained on a single 16 GB desktop GPU. The reference recipe for this configuration asks for a paid A100 with high memory.

Fine-tuned against the untuned base

Both models scored by the same harness on the same 250 rows. The base-model column is the honest control: it separates what the fine-tune taught from what Llama already knew about prices.

The striking figure is the base model's R² of -406.9%. Negative R² means it scores worse than a constant predictor that guesses the average price every time. The next section shows why.

Every model scored, same rows, same harness

Hosted frontier baselines are not included. Google's free tier now caps Gemini at 10–20 requests per day per model, which is far short of the 250 needed to compare at the same sample size, and quoting someone else's benchmark as if it were measured here would defeat the point. pricer/baselines.py runs them on a paid key.

The base model recites prices; it does not estimate them

Across 250 products the untuned model produced only 27 distinct answers, and its five most frequent replies account for 76% of everything it said — $99.99 ×78, $9.99 ×47, $12.99 ×26. It answers with common internet price strings largely independent of the product in front of it. Fine-tuning roughly doubles that variety to 53 distinct answers with the top five covering 55%, and, more importantly, ties the answer to the item.

The same weights, with and without the adapter

A LoRA adapter is a small set of matrices layered over the frozen base model, so it can be switched off in place — the same loaded weights answer twice. These are real answers on recognisable products, which is the nearest thing to a live demo this page can offer:

Read the right-hand column on its own. Every answer is a stock price ending in .99 or a round thousand — the same handful of strings the untuned model reaches for whatever it is shown. The left column is the same network, same quantization, same prompt, with 389 M adapter parameters layered on top.

Error by price band

A single average can hide a model that is only good on cheap items, so the same errors split by true price:

The gap widens as items get more expensive, which is what you would expect if the base model is anchoring on cheap round numbers.

Actual predictions

Sampled evenly across the price range rather than hand-picked — sorted by true price, then stepped through at fixed intervals, so the weak cases are included:

What was hard

Windows killed the run at step 607. Training died inside loss.backward() with CUDA error: unknown error. The cause was TDR — Windows resets the display driver if a GPU operation appears hung for longer than TdrDelay, which defaults to two seconds. A rank-256 backward pass on a card that also drives monitors exceeds that. No ECC errors, no retired pages: the hardware was fine, the default was simply wrong for compute.

Then it got 9× slower with no error at all. Training needs about 15 GB of 16.3 GB. When a few hundred megabytes of desktop applications tipped the allocation over, the driver silently paged to system RAM over PCIe instead of raising a CUDA OOM. The tell is counter-intuitive: GPU utilization pinned at 100% while power draw sat at 114 W of 300 W. High utilization with low power means the GPU is waiting on the bus, not computing — utilization alone never reveals it, only step time does. The fix was a smaller micro-batch at an identical effective batch of 256, trading 55% throughput for a run that finished unattended.