Brilliant, extraordinary assimilation of knowledge ! But I approve more of linguistic analysis, and more particularly the very inspired grammatical analysis, than economic analysis.
I had already seen the name of Akerlof, but it seems logical to me that sellers (of lemons) know more (about lemons) than consumers, even if consumers have made a lot of progress. It reminds me of a microeconomics class where I asked a rather well-known professor presenting a simplistic theory of markets, where sellers and buyers set the right price based on perfectly distributed information (perhaps inspired by Walras), what influence did advertising have on consumer information. It was late and he had promised to respond the next day, but he never did.
Akerlof's theory is therefore the opposite since it highlights information asymmetries, but his example seems a bit specious to me, although there may have been such cases in the United States at the time he described this mechanism. It’s nevertheless very smart of you to have invoked it because there is a great deal of opacity about the way models work, but even for suppliers :). That is why they cannot commit to consistency.
Moreover, the increase in the size of context windows initially responded to a pressing demand from users and offered them a clear pricing advantage (price per token, for example between GPT-5.2 and GPT-5.4), even if it is an incentive to consume.
On the other hand, suppliers may not have an interest in bearing "context rot" for too long because if one does not know their economic model, it seems that they themselves bear most of the cost. And the market might move towards more economical and efficient solutions in a « context » where everyone tinkers, as you mentioned just before the conclusion.
Thank you for the thoughtful comments here, Marc. And very good point about context size window variations—which I believe only obfuscates anomalies and quality issues such as context rot.
After I finished the piece and published, I thought about it in terms of cellular phone charges and billing—where regulations rolled out well after regulations forced transparency, consistency and disclosure.
Imagine if a Claude bill or invoice was a utility bill—like your cell phone. I’m still grappling with if it’s even fair to draw that type of equivalency, given that we are dealing with a new technology with a unique market.
Telecom and Internet have gone through these issues and phases
It is indeed a sensible approach to look for comparisons with the conditions under which earlier technologies emerged, but I am not familiar myself with the conditions that prevailed for the cell phone in the U.S. However, it was a regulated and monopolistic industry, which was deregulated in France at the same time as the emergence of mobile telephony, shortly after the breakup of AT&T.
However, the “incumbent operators” still dominate this market in major European countries, except in the UK, which has not prevented a great deal of litigation over mobile telephony—not unrelated to fierce competition, which has nonetheless benefited consumers.
Indeed, by comparison, the AI market is “the Wild West” and you’re absolutely right to point out this disparity between billing and service quality. In fact, it seems to me that clients have broken out in a cold sweat upon seeing the bills for experiments and short-lived POCs. While calls to go live sometimes border on wishful thinking, the concern for profitability now takes precedence over the wow factor.
But I don’t know at what point in the production chain we’ll soon start seeing “utility bills”—perhaps if the major providers control the lion’s share of the market through their platforms (Vertex, Bedrock, Azure…), which wouldn’t be good news :).
"Context rot" - thanks for the new word! Really helpful "big picture" perspective.
A simple takeaway - context will always get added, somewhere, by the user, by the model, or somewhere in between. How well it's managed (a rarity) determines the cost and reliability. Does that make sense?
One more random takeaway - "reports" in the BI world seem equivalent to "prompts" in the AI world. A good BI solution reduces report logic by pushing consensus business logic lower in the architecture. It's a metaphor :)
Makes total sense-! Will organizations start to be priced out of AI use (eventually) for lack of context or poor context? Spend and quality or absolutely connected :)
In the RGEM method, we use the Latin root of context, so in our knowledge compiler, KRC1, every term has related terms, that gives it "context". You could also say that context is "the hypergraph of a subject having all its known predicates with all their known objects". Or, "the context of a subject is specified in direct and indirect triples".
Thank you , highly informative. You are so right . Tokens are not context . So may ways to measure context efficiency, but the variability in token usefulness has created a pricing conundrum. In the end not all tokens are created equal .
Hello Jessica,
Brilliant, extraordinary assimilation of knowledge ! But I approve more of linguistic analysis, and more particularly the very inspired grammatical analysis, than economic analysis.
I had already seen the name of Akerlof, but it seems logical to me that sellers (of lemons) know more (about lemons) than consumers, even if consumers have made a lot of progress. It reminds me of a microeconomics class where I asked a rather well-known professor presenting a simplistic theory of markets, where sellers and buyers set the right price based on perfectly distributed information (perhaps inspired by Walras), what influence did advertising have on consumer information. It was late and he had promised to respond the next day, but he never did.
Akerlof's theory is therefore the opposite since it highlights information asymmetries, but his example seems a bit specious to me, although there may have been such cases in the United States at the time he described this mechanism. It’s nevertheless very smart of you to have invoked it because there is a great deal of opacity about the way models work, but even for suppliers :). That is why they cannot commit to consistency.
Moreover, the increase in the size of context windows initially responded to a pressing demand from users and offered them a clear pricing advantage (price per token, for example between GPT-5.2 and GPT-5.4), even if it is an incentive to consume.
On the other hand, suppliers may not have an interest in bearing "context rot" for too long because if one does not know their economic model, it seems that they themselves bear most of the cost. And the market might move towards more economical and efficient solutions in a « context » where everyone tinkers, as you mentioned just before the conclusion.
Thank you for the thoughtful comments here, Marc. And very good point about context size window variations—which I believe only obfuscates anomalies and quality issues such as context rot.
After I finished the piece and published, I thought about it in terms of cellular phone charges and billing—where regulations rolled out well after regulations forced transparency, consistency and disclosure.
Imagine if a Claude bill or invoice was a utility bill—like your cell phone. I’m still grappling with if it’s even fair to draw that type of equivalency, given that we are dealing with a new technology with a unique market.
Telecom and Internet have gone through these issues and phases
It is indeed a sensible approach to look for comparisons with the conditions under which earlier technologies emerged, but I am not familiar myself with the conditions that prevailed for the cell phone in the U.S. However, it was a regulated and monopolistic industry, which was deregulated in France at the same time as the emergence of mobile telephony, shortly after the breakup of AT&T.
However, the “incumbent operators” still dominate this market in major European countries, except in the UK, which has not prevented a great deal of litigation over mobile telephony—not unrelated to fierce competition, which has nonetheless benefited consumers.
Indeed, by comparison, the AI market is “the Wild West” and you’re absolutely right to point out this disparity between billing and service quality. In fact, it seems to me that clients have broken out in a cold sweat upon seeing the bills for experiments and short-lived POCs. While calls to go live sometimes border on wishful thinking, the concern for profitability now takes precedence over the wow factor.
But I don’t know at what point in the production chain we’ll soon start seeing “utility bills”—perhaps if the major providers control the lion’s share of the market through their platforms (Vertex, Bedrock, Azure…), which wouldn’t be good news :).
"Context rot" - thanks for the new word! Really helpful "big picture" perspective.
A simple takeaway - context will always get added, somewhere, by the user, by the model, or somewhere in between. How well it's managed (a rarity) determines the cost and reliability. Does that make sense?
One more random takeaway - "reports" in the BI world seem equivalent to "prompts" in the AI world. A good BI solution reduces report logic by pushing consensus business logic lower in the architecture. It's a metaphor :)
ZH
Makes total sense-! Will organizations start to be priced out of AI use (eventually) for lack of context or poor context? Spend and quality or absolutely connected :)
If you can explain these things to me, you can explain them to anyone haha. Thanks!
ZH
In the RGEM method, we use the Latin root of context, so in our knowledge compiler, KRC1, every term has related terms, that gives it "context". You could also say that context is "the hypergraph of a subject having all its known predicates with all their known objects". Or, "the context of a subject is specified in direct and indirect triples".
Thank you , highly informative. You are so right . Tokens are not context . So may ways to measure context efficiency, but the variability in token usefulness has created a pricing conundrum. In the end not all tokens are created equal .