Plain-English definitions for technical buyers - covering compression, deployment, retrieval, and enterprise infrastructure patterns referenced across our site.
Matches titles and aliases. Diacritics are ignored. Use hash links such as /glossary#quantization to jump directly.
Showing 43 of 43 entries.
Also known as: small language model, small LM
A compact language model with far fewer parameters than general-purpose LLMs, tuned for specific tasks so inference is cheaper, faster, and easier to run on private infrastructure.
Related services
Also known as: large language model
A broad-coverage language model with many parameters, strong zero-shot behavior, and higher compute cost - often used as a teacher, baseline, or private deployment when breadth beats smallest footprint.
Related services
Also known as: distillation, model distillation
Training a smaller student model to mimic a larger teacher’s outputs or internal signals, preserving task quality while shrinking size and inference cost.
Related services
Also known as: teacher
The larger reference model used during distillation or evaluation; the student SLM learns to approximate its behavior on target tasks.
Related services
Also known as: student
The smaller model being trained to match the teacher; in SLM programs this is typically the deployable artifact after distillation and compression.
Related services
Also known as: INT8, INT4, weight quantization, post-training quantization
Reducing numeric precision of weights (and sometimes activations) to shrink memory footprint and speed up inference - common schemes include 8-bit and 4-bit formats with accuracy trade-offs.
Related services
Also known as: structured pruning, unstructured pruning
Removing weights, neurons, or channels that contribute little to outputs, reducing model size and sometimes improving throughput after retraining or fine-tuning.
Related services
Also known as: Low-Rank Adaptation
A parameter-efficient fine-tuning method that trains small adapter matrices instead of full weights, lowering GPU memory and storage needs for domain adaptation.
See also
Related services
Also known as: parameter-efficient fine-tuning
A family of techniques (including LoRA) that update only a tiny fraction of parameters while adapting a base model to new data or style.
See also
Related services
Also known as: supervised fine-tuning, SFT
Continuing training on task-specific data to shift model behavior; can be full-weight or parameter-efficient depending on budget and infra.
See also
Related services
Also known as: foundation model, pretrained model
The starting checkpoint before task-specific adaptation - often a general LLM or mid-size model that downstream fine-tuning or distillation builds on.
Related services
Also known as: GGML successor
A single-file container format for quantized models popular in local inference stacks; useful for packaging SLMs for edge or on-prem runtimes that support compatible loaders.
See also
Related services
Also known as: model inference, serving
Running a trained model on new inputs to produce outputs - in production, inference economics (latency, throughput, cost) usually dominate architecture choices.
Related services
Also known as: response time, time-to-first-token
End-to-end delay before users see results; SLMs often reduce latency versus large cloud APIs for repetitive, structured workloads.
See also
Related services
Also known as: requests per second, tokens per second
How many queries or tokens a system processes per unit time; quantization and smaller models typically improve throughput per GPU.
Related services
Also known as: tokenization
The atomic units a tokenizer splits text into; pricing, context limits, and batching are all expressed in tokens rather than raw characters.
See also
Related services
Also known as: sequence length, max context
Maximum tokens the model can attend to in one forward pass; larger windows help long documents but increase memory and latency.
Related services
Also known as: prompting
The input text (and optional system instructions) that conditions model output; production systems often template and validate prompts for safety and consistency.
Related services
Also known as: zero-shot
Asking the model to perform a task without labeled examples in the prompt - larger models excel here; SLMs may need few-shot or fine-tuning for reliability.
Related services
Also known as: few-shot prompting
Providing a handful of example input–output pairs in the prompt to steer behavior without weight updates - useful when fine-tuning cycles are too slow.
See also
Related services
Also known as: retrieval-augmented generation
Retrieving documents or facts from a datastore at query time and conditioning generation on them - reduces hallucination risk when knowledge changes often.
Related services
Also known as: vector embedding
A dense numeric representation of text or media used for similarity search, clustering, and retrieval pipelines beside language models.
See also
Related services
Also known as: vector store, similarity search
A datastore optimized for nearest-neighbor search over embeddings, commonly backing RAG and semantic search features.
Related services
Also known as: confabulation
Confident but incorrect or unsupported model outputs; mitigation combines retrieval, evaluation harnesses, human review, and task-specific SLMs with narrower scope.
See also
Related services
Also known as: reinforcement learning from human feedback
Using human preference signals to align model outputs with policy and tone goals - common in modern LLM post-training stacks.
Related services
Also known as: Open Neural Network Exchange
An interchange format for models that helps port networks across runtimes and hardware accelerators - handy when standardizing deployment pipelines.
See also
Related services
Also known as: graphics processing unit, accelerator
Hardware used for parallel tensor math in training and inference; SLMs typically need less GPU headroom than frontier LLMs at similar traffic.
See also
Related services
Also known as: total cost of ownership
Full cost of a system over time - hardware, power, ops labor, and software - not just per-token API fees; private SLMs often shift spend toward fixed infra.
Related services
Also known as: data residency, digital sovereignty
Keeping data and model artifacts within legal or policy boundaries you control - often a driver for VPC, on-prem, or region-locked deployments.
See also
Related services
Also known as: virtual private cloud
An isolated network environment in a public cloud where you run workloads with private addressing and policy controls - common home for private models.
See also
Related services
Also known as: air gap, offline environment
Systems without routine outbound internet access; model delivery and updates rely on physical or one-way transfer processes.
See also
Related services
Also known as: on-prem, on prem
Hardware and software running in facilities you operate rather than multi-tenant SaaS inference - favored for strict compliance programs.
See also
Related services
Also known as: vLLM inference server
An open high-throughput inference server for transformer models with features like PagedAttention - often used in private serving stacks.
See also
Related services
A local model runtime popular for developer workflows; useful for experiments though enterprise production usually adds hardening, observability, and policy layers.
Related services
Also known as: token streaming, SSE
Emitting partial outputs as tokens are generated - improves perceived latency for chat UIs while complicating logging and safety filters.
Related services
Also known as: offline inference
Processing large queued inputs where latency per item matters less than total cost - common for document extraction or nightly scoring.
See also
Related services
A structured summary of a model’s intent, data, evaluation, and limitations - helps governance teams review deployments consistently.
See also
Related services
Also known as: eval harness, benchmark
Standard tasks and metrics used to compare models; enterprise programs add private evals aligned to internal KPIs, not only public leaderboards.
See also
Related services
Also known as: JSON mode, schema-constrained generation
Forcing or strongly biasing the model to return machine-parseable shapes - critical for downstream automation and API contracts.
See also
Related services
Also known as: sampling temperature
A sampling knob controlling randomness; lower values yield more deterministic outputs, higher values increase diversity and risk.
Related services
Also known as: model routing, cascading models
Sending each request to SLM, LLM, or external toolchains based on policy - balancing cost, latency, and quality across a portfolio.
Related services
Also known as: AI agents, multi-step workflows
Coordinating tools, memory, and sub-models to complete multi-step tasks - often layered on top of reliable SLM or LLM cores.
See also
Related services
Also known as: inference endpoint
The HTTP or gRPC surface clients call to run a model; enterprise setups add auth, rate limits, and audit logging around raw inference.
See also
Related services