Token
Also known as: tokenization
The atomic units a tokenizer splits text into; pricing, context limits, and batching are all expressed in tokens rather than raw characters.
Contact if you need a term added for a security or procurement review.
Also known as: tokenization
The atomic units a tokenizer splits text into; pricing, context limits, and batching are all expressed in tokens rather than raw characters.
Contact if you need a term added for a security or procurement review.