AI Token Economics: A Pricing Puzzle for Companies

Firms like Microsoft, Google, and Anthropic have invested hundreds of billions of dollars in developing Large Language Models (LLMs), the technology behind popular AI services. These companies offer both free and paid versions of their AI, with extra features for tasks such as coding or billing.

However, setting a price for these services is surprisingly difficult due to rapidly changing economics around tokens, the building blocks of LLMs. The process of breaking down user prompts into mathematical chunks called tokens, processing them, and converting the response back into text or code is not entirely predictable.

Subtle variations in the prompt can produce different answers, while the same prompt will not always produce the same answer. Different models will also produce different answers. This unpredictability makes it challenging for companies to determine how many tokens they are burning through and ultimately pass those costs onto their customers.