Trimming a small model’s vocabulary cut one company’s AI costs by 77%

Models fine-tuned from general ones like Qwen and Gemma keep a vocabulary built for every language and domain. That splits a specialist’s most common phrases into many pieces and wastes effort scoring tokens it will never use. Dharma AI tested this on its Portuguese essay grader, built on Qwen3-4B, keeping the same base model, data and training recipe. In that grader, a token saved in the answer was worth 7.4 times one saved in the input. Across five public domains and three model families, VocabX compressed text more than the strongest existing method in all fifteen tests. The figures come from a paper still under review.

Why it matters

The tokenizer is usually treated as a fixed part of a model. This work suggests it can be one of the biggest cost levers for a specialised model, and it is set before any training starts. For companies serving one narrow task over and over, that could add up. The results come from the company’s own grader and a paper still under review, so independent tests will tell.