Save your AI tokens with this super (skill|prompt|style|etc.)!

by Nikita Chernyi in AI (RSS)

TL;DR for the bots (humans, scroll on)

Do Token-Saving Skills and Prompts Cut LLM Costs? Chat Is 0.7% of the Tokens

Measured across 62 coding-agent sessions on a Go monolith, chat messages accounted for 0.7% of the 1.62 billion total tokens. The bill is dominated by cache re-reads of the baseline system prompt and tool schemas, tool results, and injected context, so the savings have to come from the harness rather than from shorter prompts.

I’ve seen lots of variations of the same idea: cut token usage by using a skill, prompt, output style, extension, hook, etc., and people star such repos, upvote the posts, even share them on LinkedIn, so they should work, right? RIGHT?!

Hi there,

The internet is lying to you, but let me be your savior and bring the scroll of truth.

“WHOAH! How is that even possible? There are so many options for the same thing, they sure should work, otherwise who are the people promoting them?”

Turns out, they don’t work, because all those “tricks” target literally the cheapest item on the bill: chat messages.

I’m pretty annoyed right now, because too many “cut-your-tokens” salesmen got to me today, so this post will just show you an example of actual usage during regular Go backend development work, using stats from actual agent sessions.

A smirking catgirl accountant holds up a huge printed bill and points at one line circled in red marker, with 0.7% written inside the circle AI-generated catgirl, contractually obligated to lure you into the post.

The answer: 0.7%

Usually I give you some clearly provocative take, joke around, and slowly explain the point, reaching the climax in the conclusion at the end of the blog post, but I see that in AI slop you start with the punchy line and give the answer first, so in a post where half of the content is tables of AI-generated stats, who am I to be a rebel?

The answer: 0.7% (11.3M of the 1.62 billion total tokens of the project described below) are chat messages, exactly what you can optimize with all those magical super duper tricks. “Tokens are not money?” - then gimme spare 1.62b tokens ;)

Satisfied? Then have a nice day!

The example

Currently I’m working on a big project: refactoring + bugfixing a Go monolith. At the moment of writing, the whole project is 62 agent sessions.

The sessions are divided into 4 categories:

And the stats of raw tokens for each category (cache writes in the second table):

CategoryINcache readsOUT
discovery3k95.6M957k
planning8k562.2M4.7M
implementation11k894.2M5.9M
other<1k1.8M22k
ALL22k1,553.7M11.6M

Now, the most interesting part: stats by type:

CategoryWritten once / cache writeTotal (with cache re-reads)Share
chat in (you)4k83k<0.1%
chat out (agent)1.3M11.3M0.7%
thinking3.9M105.4M6.5%
tool calls out: args6.4M158.1M9.8%
tool calls in: results16.2M450.5M27.9%
baseline9.2M475.7M29.4%
injected12.3M363.3M22.5%
compaction1.3M52.6M3.3%
total50.7M1,617.0M100%

A bit unexpected, right? The “be concise” tricks literally optimize the cheapest part.

Note: the 0% chat in (you) may be different for you, because of the way I work with the agent in this project: each session type has its own skill (part of baseline), and all the decisions and instructions are in the tasks (agent reads them, so they are part of tool calls in: results), so my prompts were like /discovery <decisions-path>, /plan <TASK-ID> or /implement <plan-path>. Don’t mistake prompt length vs prompt content: in this project prompt content is in discovery docs and tasks.

Conclusion (the real one)

The stats above are pretty clear: there is no point in optimizing chat, but there is huuuge room for improvement in the approach and harness (baseline, injected, tool calls categories), because together they carry about 90% of the tokens.

How can you optimize them?

Session per task

Do you need to plan work? Start a session, do the planning, end the session. Implementation goes into the next one: saving on cache re-reads (bonus: it also keeps the “smart zone” (opens in new tab), see the overlooked bonus part for the details).

Compact mid-session

Cache re-reads are costly, so compacting mid-session cuts your bill by turning, say, 100k tokens that are re-read on each tool call into 10k (yes, they are still re-read, but now every go test costs you 10x less in cache re-reads).

For example, I plan work (during planning sessions) into phases, so each plan is divided into 2 or more self-contained blocks. After each phase I do compaction. That allows you to save tokens (does your agent really need the full content of some throwaway script it did at the start of the session in the context? Not really), and also keeps the context clear of non-relevant info.

Choose harness wisely

By choosing a proper harness and tweaking it to your needs, you can achieve a lot. Yes, in this project I do not use my regular (Oh My) Pi setup, because of the project owners’ requirements, but I chose OMP (opens in new tab)/Pi (opens in new tab) for my own setup exactly because of these stats.

For example, OMP’s read tool manages limit/offset automatically, so the agent asks for a range of a 100k-token file and gets that range, and it gives AST summaries from the code files, so the agent gets what really matters instead of 300 lines of outdated comments. Raw content comes back when the agent deliberately asks for it.

Even if you are locked into proprietary Codex/Claude Code/etc. and can’t do much with the harness itself, there are still options for improvement, like replacing the default huge system prompt with a short one (stop worrying about the tool schemas: they are delivered separately, so you could just name the tools briefly in the custom system prompt), or optimizing AGENTS.md files (btw, I have a Write AGENTS.md skill, how convenient!).

AI, SI, AGI, ASI, LLMs, and other buzzwords from the position of a backend engineer already tired of ‘Dear colleagues, we are AI-first now!’

Too much AI for today? See Misc, SMTP for a change

llms.txt