October 24, 2024

Large Models, Small Models: A Practical Distinction for BD Teams

Not every task needs the largest available model, and using the largest one everywhere is often the more expensive mistake, not the safer one.

What size actually buys you

Larger models tend to handle ambiguous, open-ended tasks better, synthesizing a recommendation from mixed signals, or writing in a specific voice consistently across a long document. Smaller models are often just as accurate on narrow, well-defined tasks, classifying a message, extracting a date, checking whether a name appears in a webpage, and they run faster and cheaper.

Where teams overspend

Running every routine extraction task through the largest available model is common and usually wasteful. The task does not need the extra capability, and the extra cost and delay add up across thousands of small operations a week.

A simple rule of thumb

Use the smallest model that reliably gets the narrow task right, and reserve larger models for steps that require judgment, synthesis, or a specific tone. Test both on a sample of real tasks before deciding, the right answer is empirical, not theoretical, and it changes as models improve.

A simple test to run internally

Take a batch of twenty recent routine extraction tasks and run half through the largest available model and half through a smaller, cheaper one. Compare accuracy directly. In most teams that run this test, the smaller model matches the larger one closely enough on narrow tasks that the cost difference is not justified, and the result is enough to change the default setting for that task category going forward.

Repeat the same test periodically, since the right choice shifts as models improve at different rates, a smaller model that lagged noticeably a year ago may have closed the gap since, and a default set once should not be assumed permanent.

Documenting the decision, not just making it

Once a test like this settles which model size to use for a given task, write the decision down along with the date and the sample results, rather than relying on someone remembering it a year later when a new hire asks why the team uses a smaller model for extraction. A short, dated internal note prevents the same test from being re-litigated informally every time staff turns over.

A short note on why this test needs repeating, not just running once

A model comparison run once, a year ago, tells a team almost nothing about the right choice today, since both the larger and smaller models on either side of that comparison have likely changed since. Treat the test as a recurring practice tied to a calendar reminder, not a one-time finding to be filed away and trusted indefinitely.

A short note on avoiding vendor lock-in through model choice

Building a workflow around a single vendor's specific model, rather than a task-based routing layer that can call different models depending on need, creates a switching cost that grows every month the workflow runs. Teams that keep the routing logic in their own hands, even if it means slightly more setup work up front, retain the ability to move to a better or cheaper option later without redesigning how the whole team works.

This is worth raising directly with any vendor during evaluation: can the underlying model be swapped without disrupting the team's workflow, or is the routing decision locked inside the vendor's own product in a way that ties the firm to whatever choices that vendor happens to make going forward. The answer says as much about the vendor's own incentives as it does about the technology.

A final word on avoiding analysis paralysis

None of this framework is meant to turn every drafting task into a formal model-selection exercise. Most day-to-day work should default to whatever the standard workflow already routes to, and the more deliberate task-by-task thinking described here is reserved for genuinely new task types or periodic reviews, not for every single message a team sends. Overthinking a routine task defeats the purpose of having a sensible default in the first place.

VIPMarketing Editorial