Small Language Model (SLM)
A language model with far fewer parameters than a large language model, built to run efficiently on limited hardware or for narrow tasks.
SLMs can run on company servers, laptops or edge devices, which reduces cost and latency per request and keeps sensitive data in-house for tasks like classification, extraction and routing.
A law firm fine-tunes a 3.8-billion-parameter Phi model on 20,000 labeled emails to classify matters and extract deadlines, running it on an on-premises GPU so client data never leaves its network.
SLMs have narrower knowledge and weaker reasoning on complex or open-ended tasks, and they can still hallucinate and inherit bias from teacher models. Results usually depend on task-specific Fine-Tuning and evaluation.
SLMs use the same transformer architecture as a Large Language Model (LLM) but typically have from a few million to a few billion parameters, compared with hundreds of billions. They are built or reduced through knowledge distillation from a larger teacher model, pruning of weights or layers, quantization from 32-bit floats to 8-bit or lower integers, and low-rank factorization. Examples include Microsoft Phi, Google Gemma, IBM Granite and Ministral.
AI Governance, Systems Architecture
IBM — What are small language models? — https://www.ibm.com/think/topics/small-language-models; Microsoft Azure — Phi open models — https://azure.microsoft.com/en-us/products/phi