AI can be a miracle pill, but not at zero cost: the token cost of messy data

01What does "tokenomics" mean for a business using AI?

Here it means the LLM token cost of using AI: what you pay for every question.

AI models charge by the token. A token is a chunk of text, roughly three quarters of an English word, or about four characters. You pay per million tokens the model reads (input) and per million it writes (output). So the cost of any AI question comes down to two things: how much the model has to read, and how much it writes back.

The answer is usually short. What changes the bill is what the model has to read to get there, and that depends almost entirely on the state of your data.

Messy data burns tokens. Clean data spends them.

Is this the same as crypto tokenomics?

No. In crypto, "tokenomics" means the economics of a digital coin: how many exist, who holds them, how they are issued. In AI, a token is a unit of text, and "AI tokenomics" is the cost of the text a language model reads and writes. Nothing here involves buying, selling or holding any token. If you searched for crypto, this is the wrong article. If you searched because your AI bill keeps going up, read on.

02How many tokens does one AI question really burn?

More than most people think, because the model does not only read your question.

I checked one of my own working AI chats today, 28 Oct 2026. It was a long working session, the kind I run most days. When I sent my next message, the conversation held about 303,000 tokens. About 207,000 of them were the conversation history: everything said so far, plus the files and results pulled in along the way. The other 96,000 or so were standing instructions and tool descriptions that sit in the background of every message.

My message itself was a sentence or two. The model read about 303,000 tokens to answer it.

All of that is re-sent and re-read every time I send a message, because the model keeps no memory between messages. Anthropic's own documentation says it plainly: the API is stateless, so you always send the full conversation history. ChatGPT, Claude and every other chat tool work the same way underneath. The chat window looks like a conversation. To the model, every message is a fresh read of the whole thing.

The saving grace is caching. Most of my 303,000 tokens were billed at the cached rate, because the start of the conversation was identical to the last time it was sent. On Anthropic's API, a cache read costs 0.1 times the normal input price, and some newer models go lower. Illustratively, at US$3 per million tokens, 303,000 tokens would cost about US$0.91 per message at full price, or about US$0.09 if all of it were read from cache. The cache helps a lot. It does not make the reading go away.

What happens to the files you give an AI tool?

It depends on the tool, and the difference matters for your bill. There are three common ways.

1. The full file sits in the conversation. You attach a document and its text goes into the context, the working memory the model reads. From then on it is re-read with every message, and the pile grows as the chat grows. This is the most accurate way, because the model sees everything, and the most expensive. Anthropic estimates a PDF page uses about 1,500 to 3,000 tokens, depending on how dense it is, and each page is also read as an image, which costs more on top. At that rate a 100-page research deck is about 150,000 to 300,000 tokens on its own.

2. File search, also called retrieval. The tool splits your documents into chunks, stores them in a searchable index, and for each question pulls back only the chunks that look relevant. OpenAI's file search, for example, splits files into 800-token chunks by default, with 400 tokens of overlap between them. In OpenAI's Assistants version, the tool returns up to about 20 chunks per question, inside a token budget of about 16,000 tokens on gpt-4-class models. Developers on OpenAI's own forum report around 16,000 to 18,000 tokens per question with these defaults. That is far cheaper than reading 200,000 tokens. The catch: the model only sees the chunks it found. It can miss the one paragraph that mattered, and it cannot count or total across every document, because it never reads every document.

3. Code execution, for spreadsheets. For CSV and Excel files, tools like ChatGPT's data analysis load the file into a code sandbox and write Python against it, instead of reading every row into the model. The model reads the code and a summary of the result, not the whole sheet. That keeps the token bill small. The risk moves elsewhere: if the columns are messy, the code can be wrong, and the answer looks just as confident.

Each of these is a workaround for data that was never cleaned. The cheap and accurate option is a fourth one: clean tables with clear meanings, which the model queries instead of reads.

03Why does a longer chat cost more?

Because conversation history compounds. Every message re-reads everything that came before it, including the file you attached at the start.

Here is a simple illustration. You attach a 50,000-token file, which is a long report or a few decks, and ask ten questions in the same chat. Assume each question and answer adds about 2,000 tokens to the history.

So a ten-message chat with a 50,000-token file reads roughly 590,000 tokens in total, not 50,000. At about US$3 per million tokens, that is about US$1.77 for the chat instead of about US$0.15. Caching brings the repeated part down sharply, but only if the conversation is resent within the cache window, which is five minutes by default on Anthropic's API, and up to an hour if you pay more to keep it.

This is how my own chat reached 303,000 tokens. Nothing dramatic happened. It just kept going.

Two habits follow from this. Start a new chat when the topic changes, instead of carrying a long history into an unrelated question. And do not attach the same big files each time you ask when the facts in them could live in a clean table.

04What does a basic AI analytics tool cost with messy files versus clean data?

Take a tool almost every marketing team now has: a basic "ask your data" AI assistant. Two ways of using it.

Messy files. The team attaches four raw files each time they ask (an illustrative mix): a 30-page research deck (~60K tokens), 1,000 survey comments (~60K), 800 customer reviews (~64K) and a few email threads (~16K). That's about 1,800 data points and about 200,000 tokens, roughly 150,000 words, re-read each time someone asks. Nothing is cleaned or joined up, so each new question starts from zero and the AI reads the lot again.

Clean data. The same ~1,800 data points, drawn from the same database, cleaned once into tagged tables, with a short description of what each table and column means (a semantic layer). For each question, the AI reads that description, writes a query, and summarises a small result. Call it about 3,000 tokens per question.

Same people. Same questions. Same AI. Same data. The only difference is whether it's clean.

At an assumed mid-range price of about US$3 per million input tokens:

That is about 60 times more per question. (On tokens alone the ratio is closer to 67x; we round the costs to US$0.60 and US$0.01 and call it 60x.)

Sixty cents does not sound like much. The problem is that nobody asks one question.

The assumptions, so you can swap in your own

05What does it cost at your size?

Multiply it out. 30 questions a day, 260 working days, AI token cost only:

The ratio does not change with size. What changes is whether anyone notices.

For an SME, US$94K a year is often more than the AI subscription, the analytics tool and a part-time analyst put together. It rarely shows up as one line, because it is spread across seats, credits and "why is the bill higher this month?"

For an enterprise, US$9.4M is a budget line someone will eventually be asked to defend. It is also the kind of number that kills an AI programme quietly: usage grows, the bill grows faster, and the conclusion drawn is "AI is expensive", when the real finding is "our data is expensive to read".

Here is a more useful way to read the table. The clean side excludes the cost of cleaning. So ask: how much could you spend on cleaning before messy wins? For the enterprise, the people and tooling to clean the data would have to cost more than about US$9.2M a year. For the team of 20, more than about US$92K a year. Very few cleaning projects cost that, and the cleaning is mostly a one-off with upkeep, not a repeat bill.

06How do you calculate your own AI token cost?

You need five numbers, and most of them you can estimate in ten minutes.

The formula: Annual AI token cost = tokens read per question × price per million tokens ÷ 1,000,000 × questions per person per day × working days × people

Step by step:

1. Tokens read per question. Add up what the model reads each time you ask: the files attached, the conversation so far, and the instructions in the background. Many tools show this in a usage panel or API log. If not, estimate: about 0.75 words per token, or 1,500 to 3,000 tokens per PDF page.
2. Price per million input tokens. Take it from your provider's pricing page for the model you actually use.
3. Divide by 1,000,000. That gives you the cost of one question.
4. Multiply by questions per person per day. Look at real usage for a week if you can.
5. Multiply by working days (about 260 a year) and by the number of people.
6. Optional: adjust for caching. If the same content is re-sent within the cache window, multiply that part by the cache rate (0.1 on most Anthropic and OpenAI models).
7. Optional: add output tokens. Answers are usually short, but long reports or code add up. Output is priced higher than input, typically five to eight times more per token on the models checked for this article.

Worked example (the team of 20 in this article): 200,000 tokens × US$3 ÷ 1,000,000 = US$0.60 a question. × 30 questions × 260 days × 20 people = US$93,600 a year, about US$94K.

Fill in your own (illustrative example): a team of 10, about 50,000 tokens per question, a model at US$2 per million, 15 questions a day each.

Then run it again with the tokens your clean setup would read. The difference is what your messy data costs you each year, before you count a single wrong answer.

07Isn't 60x a straw man?

Fair question. Here are the four strongest objections, and what is left of the number after each.

1. "Nobody re-reads everything. Good tools search for the relevant bits." True. A tool that retrieves a few relevant passages per question reads far less, so it costs less per question: about 16,000 tokens on OpenAI's default file search settings, as above. But retrieval can only answer from the passages it found. Ask "how many reviews this quarter mention price?" or "what share of survey respondents are unhappy with delivery?" and it cannot count or total across everything, so it gives a confident answer from a sample. Questions about the whole base force the full read. That is why the messy side is worded as "if AI has to re-read everything for each question". It is the worst case, not the typical one.

2. "Our data is in spreadsheets, not documents." Then the token bill is smaller. Tools like ChatGPT's data analysis load a CSV or Excel file into a code sandbox and write Python against it, rather than reading every row into the model. So the 60x applies most to documents and unstructured files that get re-read each time you ask: PDFs, decks, survey verbatims, review and email dumps. Spreadsheets have a different problem, which is accuracy. More on that below.

3. "Prompt caching makes repeat reads cheap." It does. Major providers discount repeated input by up to about 90% when the same content is sent again (Anthropic and OpenAI both charge about a tenth of the normal input price on a cache hit, and Anthropic's newest models go lower). With caching on every repeat read, the gap closes to roughly 6x: about US$936K against US$156K a year for the enterprise. But caching only works when the same content arrives in the same order within the cache window, which is minutes to an hour by default. Writing to the cache also costs extra: 1.25 times the normal price for a five-minute cache on Anthropic's API, and twice the price for an hour. Twenty people attaching their own versions of the files on different days get much less of it. Real life sits somewhere between 6x and 60x.

4. "Model prices keep falling." They do, and that shrinks both bills by the same proportion. The ratio stays. A cheaper model makes messy data cheaper to re-read, but it still has to re-read it, and a pricier model for harder questions widens the gap in dollars. One more detail: token counts are not fixed either. Anthropic notes its newer models use a tokenizer that produces about 30% more tokens for the same text, so the same pile of documents can cost more to read on a newer model even at a similar list price.

What is left: on documents, messy data costs somewhere between 6 and 60 times more in AI tokens, and the people cost of cleaning does not close that gap at any team size above a handful.

08Why do you pay for messy data again and again?

I call it double credits burnt. Every time someone asks, you pay the AI to clean the mess on the fly: work out which column means what, match people across files, ignore the duplicates. Then the next question pays it to clean the same mess again, because the clean version never went back into your data. It only ever existed inside one chat.

This is not a caching problem. Caching makes re-reading cheaper. It does not make the data clean. Cleaning once, at the source, is the only thing that stops you paying for it again.

Without cleaning, you don't save the cleaning cost. You pay it again with every question.

The alternative is to process data once, on arrival. When a new survey export or review dump lands, extract the facts from it, clean them, link them to customer IDs, and store them in tables with clear meanings. After that, questions run against the tables for cents, in seconds, and give the same answer every time.

Pay once to clean your data, not every time you ask.

09What does "clean once on arrival" look like in practice?

It sounds like a data team project. For most marketing data it is three small steps, run every time new data arrives, not every time someone asks.

Step 1: extract the facts. A cheap, small model reads each new document once and pulls out the facts you care about.

Step 2: store them in a table, against a customer ID. One row per fact, plain column names, one date format.

Step 3: describe the table once. A short note on what each column means and which values are allowed. This is the semantic layer. The AI reads the note, not the raw data.

Example 1: survey verbatims into a themes table

Say a brand survey comes back with 5,000 open-text answers to "What would make you buy from us more often?" The messy route: attach the export each time you ask, and let the AI read 5,000 answers to tell you what people said. Every time.

The clean route: when the export lands, a small model reads each answer once and tags it with a theme from a fixed list (price, delivery, range, quality, service) and a sentiment. The result is a table like this:

Now "what share of respondents are unhappy with delivery?" is a count on one column. The AI reads the column description, writes the query, and gets an exact number in seconds. It can also go back to the original verbatims for quotes, because each row keeps a link to its source.

Example 2: reviews into sentiment by product

Same idea with product reviews. Instead of pasting a review dump each time you ask, each review is read once on arrival and stored as: customer ID if known, product, date, star rating, sentiment, and the main topic. "Which product has the fastest-falling sentiment this quarter?" becomes a query on a few thousand small rows, not a read of every review ever written.

What the semantic layer adds

The tables hold the facts. The semantic layer holds the meanings: what "active customer" means (bought in the last 90 days, say), which revenue figure is the official one, that "negative" sentiment was assigned by a model on arrival and how. It is short, it rarely changes, and it is the same for every tool and every team.

Two things follow. It keeps the token count per question small, because the description is a few hundred to a few thousand tokens, not a few hundred thousand. And it keeps answers consistent, because everyone's question runs on the same definitions. Being short and unchanging, it is also exactly the kind of content that caching handles well.

Part 2 of this series, on the four-layer data model, covers the four layers in more detail. It is coming soon.

10What happens without a customer ID? A 500,000-customer projection

Now scale the question up from one team's documents to a whole customer base. This is an illustrative projection, with every assumption labelled.

The base: 500,000 customers, each with about 20 records across five sources: sales transactions, brand-love survey answers, sentiment from reviews and social, research insights, and web and app tagging events. That is about 10 million records. At about 300 tokens per raw record, the whole base is about 3 billion tokens.

No ID, no correlation

Without a shared customer ID, sales cannot be joined to sentiment, brand love or tagging. So the questions that matter most, like "do people who love the brand buy more?", cannot be answered at all.

The usual workaround is to ask AI to match people across sources by name and email. Suppose it is 95% accurate, which is good. That still leaves about 25,000 customers wrongly matched: someone's purchases attached to someone else's survey answers. Every correlation built on top is polluted, and you cannot tell which 25,000 they are. You also pay tokens for every match attempt, every time the matching is redone.

With an ID set at capture (sign-up, loyalty, login), the join is exact and costs nothing. This is the same point as Part 1 of this series: no ID, no story.

Whole-base questions: re-read versus clean once

Assume a small, cheap model at about US$1 per million tokens read and US$5 per million written, and about 10 whole-base questions a week:

That is about 170x on AI token cost alone. Now add the part a token count leaves out: people. Even with about US$150K a year of data-team time on the clean side, it is about US$159K against US$1.56M, still about 10x cheaper.

So the honest range is about 10x all-in to about 170x on tokens only. (Small-model prices have fallen since this projection was first sketched; at today's cheapest rates both columns shrink by the same proportion and the ratio holds.)

What a clean, ID-linked base lets you do

Once every record carries the same customer ID, questions that were impossible become routine:

• Segment by sales × sentiment × brand love, not one at a time.
• Find the people who love the brand but stopped buying, and ask why.
• Tie tagged behaviour on the website or app to what people actually purchase.
• Project the impact of a campaign on the people it reached, not on averages.

11Does messy data also give wrong answers?

Yes, and that is the bigger cost.

In a 2023 benchmark built on an enterprise insurance database, Sequeda, Allemang and Jacob found GPT-4 answered 16% of business questions correctly when querying the raw SQL database directly, against 54% when the same data was described through a knowledge graph, a structured layer that spells out what the data means.

The token bill is the cost you can see. Wrong answers that look right are the one you can't. That is Part 4 of this series, coming soon: messy data costs you twice.

12What are the signs your data is costing you tokens?

If three or more of these sound familiar, your AI bill is paying for messy data:

• People attach the same files each time they ask, because the facts in them live nowhere else.
• The same question gets a different answer depending on who asks it, or when.
• Long chats get slow, or the tool warns that a file was too big and only part of it was read.
• Your AI bill or credit usage grows faster than the number of people using the tool.
• Nobody can say what one AI question costs, only what the monthly total was.
• Survey verbatims, reviews and research decks are only ever read by AI in chat, never turned into tables.
• Sales, survey and review data cannot be joined, because there is no shared customer ID.
• "Revenue", "active customer" and "region" mean different things in different teams' files.

13Which levers cut the LLM token bill?

Five, in roughly the order they pay back:

1. What you feed the model. Clean tables and short descriptions, not raw dumps.
2. How often. Process data once on arrival, not once per question.
3. One ID per customer. No ID means paying the AI to guess who is who, again and again.
4. Which model. A small, cheap model for extraction and tidying; a bigger model only for judgement.
5. Caching. Keep repeated context, such as table descriptions, identical and at the start of the prompt so it is billed at the cache rate.

Two smaller habits help too: start a new chat when the topic changes, and limit how many chunks a file search tool returns. OpenAI notes that returning fewer results cuts token usage and latency, at some cost to answer quality.

A note on agents. Splitting a job across several AI subagents is a way to get work done faster, as I wrote about in how subagents made one of our workflows 85x faster. Each subagent works in its own context and hands back a short summary, so the main conversation stays small. But it is not a way to spend fewer tokens overall. Anthropic reports that its multi-agent research system uses about 15 times more tokens than a chat. Use subagents for speed, and clean data for cost.

14What should you do on Monday?

If you are an SME (around 20 people):

1. Stop attaching the same files each time you ask. Pick the three files your team re-uploads most and turn each into one clean table: one row per record, plain column names, one date format.
2. Put them in one simple database you own. You do not need a data lake.
3. Give every customer one ID, and keep it on every export: sales, survey, reviews.
4. Connect your AI tool to that database instead of uploading files, and write one short note on what each column means.
5. Check your AI bill or credit usage this month and next. That is your before and after.

If you are an enterprise (around 2,000 people):

1. Find where the tokens go. Ask your AI platform team for usage by team and by kind of request, and look for large, repeated inputs.
2. Clean the core tables first: customers, sales, products, and the survey and review data marketing keeps asking about. Extract facts from documents on arrival, not in chat.
3. Put a semantic layer on top, so every tool and team uses the same definitions of revenue, active customer and region. (Part 2 of this series, coming soon, covers the four layers.)
4. Make the customer ID non-negotiable at every point of capture: sign-up, loyalty, login, CRM.
5. Add governance: who can see what, and who owns each definition.
6. Turn caching on, and route simple extraction work to a smaller model.
7. Report cost per question, not just total spend. It is the number that tells you whether the data is getting easier to use.

If you want to see what clean, tagged data makes possible before you commit to anything, the free tools in our Lab are built on exactly this kind of setup.

15Want to know what your data is costing you per question?

16What do the key AI cost terms mean?

Token

A chunk of text that an AI model reads or writes, roughly three quarters of an English word, or about four characters. AI providers price their models per million tokens.

Input tokens and output tokens

Input tokens are everything the model reads for a request: your question, the conversation so far, attached files and background instructions. Output tokens are what it writes back. Output usually costs several times more per token, but input is usually where most tokens go.

Context window

The maximum number of tokens a model can read in one request. Everything the model "knows" about your chat has to fit inside it, and it is re-read every time you send a message.

Prompt caching

A discount for re-sending the same content. If the start of a request matches one sent recently, the provider bills that part at a cache rate, about a tenth of the normal input price on most Anthropic and OpenAI models. The cache expires after a few minutes to an hour.

Retrieval (RAG)

Retrieval-augmented generation. Documents are split into chunks and indexed; for each question, the tool fetches only the chunks that look relevant and gives those to the model. Cheaper than reading everything, but it can miss things and cannot count across all documents.

Semantic layer

A short, shared description of what your data means: which tables exist, what each column holds, and how key business terms such as revenue or active customer are defined. AI reads the semantic layer instead of the raw data, which keeps questions cheap and answers consistent.

Customer ID and identity resolution

A customer ID is one identifier that follows the same person across every system: sales, surveys, reviews, web and app. Identity resolution is the work of linking records to the right person. Done at capture, it is exact and free; done later by guessing from names and emails, it is costly and error-prone.

First-party data

Data a business collects directly from its own customers, through its website, app, sales, loyalty programme, surveys and service. It is the data most worth cleaning, because it is the data only you have.

18How were these figures built?

• All cost figures are illustrative AI token cost, built by Konverta.ai from the assumptions stated in each table. They are not based on any client's data, any real visitor data or any provider's invoice.
• Hero example: tokens per question (200,000 messy, 3,000 clean) are scenario assumptions, priced at US$3 per million input tokens, a mid-range list price checked on Anthropic's and OpenAI's pricing pages on 28 Oct 2026. The messy side is an illustrative mix of four raw files (a 30-page research deck, 1,000 survey comments, 800 customer reviews and a few email threads, about 1,800 data points); the clean side is the same data points, cleaned once into tagged tables. Output tokens and caching are left out on purpose, to show the worst case. Annual figures use 30 questions per person per day and 260 working days.
• Caching: the ~6x figure applies the 0.1x cache-read rate to every repeat read and ignores cache-write premiums, which would nudge it slightly above 6x.
• 500,000-customer projection: customers, records per customer, tokens per record, question frequency and the US$150K people cost are assumptions, labelled as such. Arithmetic is shown in the table.
• My own chat (303,000 tokens): read from the usage figures of one of my working AI sessions on 28 Oct 2026. It is one measurement of one session, given as a real example of how context builds up, not as an average.
• Longer-chat table: illustrative, assuming a 50,000-token file, about 2,000 tokens added per question and answer, and no caching.
• Your-own-cost example (team of 10): illustrative numbers to show the formula.
• Sourced facts (statelessness, cache rates, file-search chunking, PDF tokens per page, the 16% vs 54% benchmark, multi-agent token use) are linked below with access dates.

19Sources

1. Anthropic, Claude API pricing (model prices and prompt-caching rates), accessed 28 Oct 2026.
2. Anthropic, Prompt caching (cache reads at 0.1x base input price; 5-minute writes at 1.25x and 1-hour writes at 2x; 5-minute default lifetime; Opus 5.5 and Sonnet 5.5 cache hits at 0.05x), accessed 28 Oct 2026.
3. Anthropic, Using the Messages API (the API is stateless, so you always send the full conversation history), accessed 28 Oct 2026.
4. Anthropic, PDF support (about 1,500 to 3,000 text tokens per page; each page also processed as an image), accessed 28 Oct 2026.
5. Anthropic Engineering, How we built our multi-agent research system (multi-agent systems use about 15x more tokens than chats; subagents work in their own context windows), accessed 28 Oct 2026.
6. OpenAI, API pricing (model prices and cached-input rates), accessed 28 Oct 2026.
7. OpenAI, Retrieval guide (default chunking 800 tokens with 400 overlap; 10 results by default, up to 50), accessed 28 Oct 2026.
8. OpenAI, File search tool guide (fewer results cut token usage and latency, may reduce answer quality), accessed 28 Oct 2026.
9. OpenAI Developer Community, Token budget in file_search tool (documentation quoted: up to 20 chunks, 16,000-token budget for gpt-4 models, 4,000 for gpt-3.5-turbo), accessed 28 Oct 2026.
10. OpenAI Developer Community, Why is token use so high when using file search? (about 18,000 tokens per question reported; 800-token chunks, up to 20 returned), accessed 28 Oct 2026.
11. OpenAI Help Center, What are tokens and how to count them (a token is about four characters, or three quarters of a word, in English), accessed 28 Oct 2026.
12. OpenAI Help Center, Data analysis with ChatGPT (spreadsheets analysed by running Python), accessed 28 Oct 2026.
13. Sequeda, J., Allemang, D. and Jacob, B. (2023), A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model's Accuracy for Question Answering on Enterprise SQL Databases, arXiv:2311.07509, accessed 28 Oct 2026.
14. All cost figures: Konverta.ai illustrative projections; assumptions as stated in the tables above. Not based on any client's data.