One of the fundamental promises of enterprise AI is productivity gains at a lower cost. However, as enterprises invest more in agentic capabilities, that promise is showing cracks.
A 2026 study by Stanford researchers found that agentic coding tasks consumed up to 1,000 times more tokens than code-chat and code-reasoning tasks. The study also found that token consumption and accuracy are not proportional.
Interestingly, token cost is decreasing on paper. Yet, the real cost of completing tasks using frontier models has increased by 80% since Q4 2024. Gartner analysts predict that by 2028, high token costs could make hiring human coders a cheaper option.
This raises an important question: Do enterprises need a frontier model for every task?
In our experience, token costs often compound because teams shove frontier models into use cases where their reasoning capabilities aren’t needed. In those use cases, frontier LLMs need extensive domain-specific instructions, complex decision steps, and excessive LLM API calls just to be effective enough.
Small language models (SLMs), however, can be more affordable and accurate.
What are SLMs in an enterprise context?
Small language models share similar architectures to LLMs but have significantly fewer parameters. Where LLMs can easily have hundreds of billions to trillions of parameters, SLMs are usually capped around 10 billion.
Their smaller size comes from several techniques used to build and optimize compact models:
Knowledge distillation: A smaller language model learns from a larger, more capable model by training on larger model outputs, predictions, or reasoning patterns. This helps the SLM replicate specific capabilities of the larger model while using fewer parameters and less computing power.
Pruning: Less important weights or connections from a large AI model get removed to make it smaller and more computationally efficient.
Quantization: Sometimes the highest level of numerical precision is not required. In such cases, quantization is used to reduce the numerical precision used to represent model weights, which makes the model smaller (while maintaining acceptable accuracy).
Purpose-built training: This technique involves training a compact model from scratch on curated, domain-specific data.
All these techniques focus on making an SLM specialized, at the cost of some general capabilities. Instead of trying to understand everything, it concentrates on a narrower set of responsibilities.
In enterprise settings, AI agents are often part of an agentic ecosystem, where each agent specializes in a specific task. Frontier LLM-powered agents can accomplish such specialized tasks only after extensive additional context and instructions.
An enterprise SLM can be trained or fine-tuned for a specific domain, task, or set of workflows. Therefore, SLM-based agents can do the same task without extra support. This efficiency shows up as low latency, high accuracy, and low token consumption.
The same principle is behind Fabrix’s three specialized models: Argos Vx, Argos VE, and Argos AIOps. Each of them has distinct capabilities:
- Argos VX: Supports VibeOps by helping teams build agentic applications and operational dashboards using natural-language instructions.
- Argos AIOps: Provides domain-specific intelligence for interpreting operational events and supporting incident investigation.
- Argos VE: Supports vulnerability exposure workflows by helping agents determine relevant security checks and assess exposure.
These models are based on a 4B-8B parameter open foundation model that underwent QLoRA fine-tuning on a purpose-built dataset. Because of their specialization, agents built on these SLMs understand the domain semantics needed to interact with systems by default.
The Advantages of SLMs in the Enterprise Context (Fabrix’s Perspective)
The case for SLMs gets stronger when contexts change from general benchmarking to enterprise reality. Let’s look at key advantages that show up in practice, using Fabrix’s experience to show how they play out.
1. Specialized intelligence grounded in enterprise context
An effective language model in an enterprise setting must be rooted in organizational reality. It should be able to access data sources, update as new data arrives, understand semantics, and use that context to produce more relevant agentic outputs.
Compared to frontier models, specialized SLMs have an edge:
- When an SLM is trained or fine-tuned on organization-specific data, it can dedicate more model capacity to understanding the organization’s domain, workflows, and requirements. This specialization can allow it to outperform a much larger LLM on specific enterprise tasks.
- SLMs take far less computational power and time to train. They can be updated with new data, policies, and changes faster and more frequently.
- SLMs can reside inside an enterprise environment and become more effective as they are exposed to new data.
But an SLM can reach its full potential only when it can access the full enterprise context. Enterprises still need to connect their data sources, contextualize that data, and make that data available to SLM-powered agents.
Fabrix platform addresses this with two capabilities: Data Fabrics and Ontology. Data Fabrics helps connect enterprise-wide data sources, ingests incoming data, contextualizes it, and routes it to the right agents. Ontology helps language models understand the semantic relationships between entities, business logic, and data.
When organizational context is fed into Fabrix’s proprietary Argos SLMs (and supported frontier LLMs too), the outputs become more grounded.
2. Greater control over where AI runs
When agents are built on closed LLMs, every request goes to a third-party server through model APIs. In many cases, this is not a problem. However, when these requests carry sensitive information about proprietary data and infrastructure, it can create big security and regulatory risks.
Enterprises, however, can host specialized SLMs within their own infrastructure. This gives enterprises more control over where model inference occurs and how AI models handle operational data. It also provides an alternative to depending exclusively on an external model provider for recurring operational tasks.
Many enterprise operational use cases include infrastructure configurations, incident details, service dependencies, and security information. By switching to SLMs for operational tasks, enterprises can protect sensitive data, minimize vendor lock-in risk, and deploy, operate, and govern models.
For example, Fabrix’s Argos models can be deployed in the cloud, on-prem, and fully air-gapped. Fabrix’s agentic platform itself offers tools to observe and govern models for its users.
3. Low latency and higher accuracy
Low latency and high accuracy are desirable across all enterprise operational agentic use cases. They are even more important in high-impact areas such as networking and security.
Specialized SLMs often have lower latency and higher accuracy (for domain-specific tasks) for several reasons:
- SLMs often have terminology, task format, and rules baked in. As a result, they need shorter and fewer prompts to complete a task, which speeds them up significantly.
- Specialized models are domain experts to begin with. They need fewer reasoning and self-correction steps to generate an acceptable answer or reach the right conclusion.
- Fewer parameters mean less computation, which directly translates into faster responses.
- SLMs respond faster because they can run on in-house infrastructure. This removes unnecessary network round trips and queuing delays that otherwise add up across multi-step agentic workflows.
- Because of domain-specific knowledge and organizational context, SLMs are generally more accurate.
For context, when benchmarked, Argos fine-tuned models have shown 1.5 sec latency (compared to 12.6 sec for Gemma 4) thinking off and 7.6 sec latency thinking on (36.7 sec for Gemma 4).
Our tests showed that these specialized SLMs can reach 90th-percentile accuracy.
4. More predictable and lower costs
Agentic workflows can make repeated model calls, carry context between steps, and consume additional tokens while checking or refining results. Even when individual calls are inexpensive, costs can accumulate across thousands of agents and recurring workflows.
SLMs cost fewer tokens for the same reasons they have low latency. They carry domain knowledge and organizational context, need shorter, fewer prompts, and require less reasoning and fewer corrective steps, so each action consumes fewer tokens.
However, they can reduce the token bill in a few more ways:
- The cost of self-hosted SLMs is mostly a fixed infrastructure cost. With more queries, the per-query cost drops significantly. LLMs, on the other hand, bill based on usage and can easily exceed the cost of a similar level of SLM usage.
- A smaller model needs less memory and computational power. This reduces the hardware cost of each response.
- Often, steps in agentic flows are not very complicated: classify a request, search a table, or hand something over to another agent. Using LLMs for these straightforward tasks is poor token optimization. SLMs can handle grunt work and free up LLMs for more complex tasks.
Frontier models Will Co-exist
As powerful as SLMs are, they are not a replacement for frontier models across the board.
Some enterprise tasks require open-ended reasoning, interpreting unfamiliar situations, synthesizing information across a long context, and planning complex sequences of actions.
For example, an agent investigating a major production outage may need to correlate application failures, network changes, security alerts, and infrastructure dependencies. It may also need to evaluate competing hypotheses and determine the best course of action.
Model selection should not be an “either/or” situation. A specialized model can help interpret events or identify relevant evidence. A frontier model may be better suited to synthesizing that evidence and reasoning through a complex, unfamiliar problem.
Enterprises should retain access to frontier models for workloads that benefit from their broader capabilities:
- The task is exploratory: Agents need to investigate unfamiliar problems or evaluate multiple possible explanations.
- Long-context analysis: Agents need to synthesize large amounts of information across documents, logs, and operational records. Longer context windows in LLMs usually help in these situations.
- Complex planning and execution: Agents must coordinate multi-step workflows that require broader reasoning and adaptation.
The objective is to avoid paying for capabilities a task doesn’t need.
How Fabrix Promotes a Hybrid Approach
A practical enterprise AI architecture needs both specialized and general-purpose intelligence because enterprise workloads can be a mix of highly exploratory and deterministic.
Fabrix supports this through its LLM Gateway that routes requests to the appropriate model based on the task. That model may be one of three Argos models or a frontier model, depending on the task.
However, Fabrix’s hybrid approach goes beyond model routing and covers context building, MCP connectivity, and governance.
- The Agentic AI Harness connects language models (both Argos and any supported Frontier model) to enterprise data, tools, APIs, and workflows. Fabrix’s Universal MCP Server allows agents to interact with existing enterprise tools through standardized interfaces. Fabrix can consume other vendors’ MCP servers and expose its own capabilities for other agentic layers to call.
- This also makes Fabrix’s approach relatively UI-agnostic. Its observability pipelines are exposed as governed tools and resources rather than being locked inside the Fabrix interface. The same pipelines can therefore provide data and actions to Fabrix’s own agents or to agents running elsewhere.
- Enterprises need reliable model governance: which model an agent used, what it did, what tools it called, how much it cost, and whether it operated within defined controls. Fabrix’s Agent Control Plane provides observability, cost and token insights, evaluation, security, and guardrails across agentic workflows.
That makes the hybrid approach more practical. Enterprises can use smaller models where they are sufficient, reserve frontier models for tasks that genuinely require broader reasoning, and continuously evaluate those decisions using real operational data.
If you want to see Fabrix and Argos models in action, feel free to book a demo.
Frequently Asked Questions
What is a small language model (SLM) in an enterprise context?
A small language model shares a similar architecture to a large language model but has far fewer parameters, usually capped around 10 billion instead of hundreds of billions to trillions. In the enterprise, an SLM is trained or fine-tuned for a specific domain, task, or set of workflows, so agents built on it can handle specialized work without extensive extra instructions.
How are small language models made smaller than LLMs?
SLMs are built and optimized with techniques such as knowledge distillation (learning from a larger model’s outputs), pruning (removing less important weights), quantization (reducing numerical precision), and purpose-built training on curated, domain-specific data.
Why do SLM-based AI agents cost less to run than frontier models?
SLMs carry domain knowledge and organizational context, so they need shorter and fewer prompts, less reasoning, and fewer corrective steps, which means fewer tokens per action. Self-hosted SLMs are mostly a fixed infrastructure cost, so per-query cost drops as usage grows, and smaller models need less memory and compute per response.
Are small language models faster than frontier LLMs?
For domain-specific tasks they often are. In benchmarks, Fabrix Argos fine-tuned models showed 1.5 seconds latency with thinking off (compared with 12.6 seconds for Gemma 4) and 7.6 seconds with thinking on (compared with 36.7 seconds), and Fabrix tests showed these specialized SLMs can reach 90th-percentile accuracy.
Can enterprises run small language models on-premises or air-gapped?
Yes. Enterprises can host specialized SLMs within their own infrastructure, which gives them more control over where inference occurs and how operational data is handled. Fabrix Argos models can be deployed in the cloud, on-prem, or fully air-gapped.
Do small language models replace frontier models?
No. Frontier models remain better suited to exploratory investigation, long-context analysis, and complex multi-step planning. The practical approach is hybrid: use SLMs where they are sufficient and reserve frontier models for tasks that need broader reasoning.
What are the Fabrix Argos models?
Argos is a family of three specialized models built on a 4B-8B parameter open foundation model with QLoRA fine-tuning: Argos VX supports VibeOps for building agentic applications and dashboards, Argos AIOps interprets operational events and supports incident investigation, and Argos VE supports vulnerability exposure workflows.
How does Fabrix route work between SLMs and frontier models?
The Fabrix LLM Gateway routes each request to the appropriate model, either one of the Argos models or a supported frontier model, based on the task. The Agentic AI Harness, Universal MCP Server, and Agent Control Plane add enterprise context, tool connectivity, and governance across those workflows.
