Inference Cost
The cost of running an AI model to produce an output, measured per token or per query. It is a key operational consideration when scaling AI across a large document set or portfolio, because it adds up quickly.
Example
A team budgets for inference cost when summarizing thousands of customer contracts.
Related terms
Missing a term, or want to suggest one we should add? Talk to Adrian Scott →