Cost per Token

Granular cost metric for inference; optimize via prompt compression, caching, batching, and model choice.

Related terms

Token (in LLMs)TokenizationBatching (LLM Serving)
View on glossary index