The Falling Cost of AI: How Technology, Competition and Open Source are Driving Down Token Prices

Recently Andrew Ng in his post shared about the falling cost of ChatGPT tokens, The cost has plummeted by approximately 80%, from $36 per million tokens in March 2023 to a mere $4 per million tokens today. Such a significant drop in price within a short period is certainly noteworthy and sparks curiosity about the factors driving this change.

In this article, I delve into the factors driving the decreasing cost of tokens for AI models like ChatGPT. The cost reduction stems from a combination of technological advancements, economies of scale, and increased competition.

Technological progress, both in hardware and software, has significantly improved the efficiency of AI models. The release of open-source models like Meta’s LLaMA has further intensified competition, fostering innovation and driving down prices. The efficient handling of training data and smarter tokenization techniques have also contributed to cost reduction.

Today in this document, I will cover the key factors driving this cost reduction:
•   Technological Advancements (pg. 2)
o   Hardware Improvements (pg. 2)
o   Software Optimizations (pg. 3)
•   Economies of Scale (pg. 4)
o   Increased Usage (pg. 4)
o   Cloud Providers (pg. 5)
•   Open-Source Models (pg. 6)
o   Increased Competition (pg. 6)
o   Innovation and Efficiency (pg. 7)
o   Lower Barriers to Entry (pg. 8)
•   Improved Data Handling (pg. 9)
o   Data Deduplication (pg. 9)
o   Smarter Tokenization (pg. 10)
•   Market Forces (pg. 11)
o   Increased Market Players (pg. 11)
o   Open-Source Collaboration (pg. 12)
•   Conclusion (pg. 13)

Hope you find this document insightful. Follow along for posts on Generative AI and Data Science for All.