Claudinator devlog: v1.10.0, and why my numbers were wrong
A month of measuring instead of reading turned up four defects, one of them worth 1.86 billion tokens, and the first pull request from someone else.
v1.10.0, 40 of 40 tests, still public under MIT. Still one Node script with no dependencies on port 8752, reading transcripts that are already on your disk and sending nothing anywhere.
Why the numbers were wrong
I spent September on accuracy rather than features, and the method was simple: stop reading the code and start checking its output against the transcripts by hand. That turned up four real defects.
The biggest was attribution. When a conversation is resumed or compacted, a new transcript file is started, the history is copied into it, and the working directory and session id are rewritten on the way. I had not understood that. It means half of all API calls exist in two to eight files, each claiming a different home, and my dashboard was filing the duplicates under whichever session happened to make them. Once I deduplicated on message and request id and folded resumed files back into their parent conversation, 1.86 billion tokens moved out of the meaningless catch-all bucket and the compaction panel went from 24 cards to 9. Everything else followed from getting that one thing right.
Two pricing mistakes as well. One model’s cache-read rate was set to a tenth of its input price when it is a fortieth - $303 off the estimate, about 7 %. And any model missing from the price list was being costed at the most expensive rate, so an unlisted small model came out 6.3 times over. Unknown models are now priced by family, and a banner names them so you know an estimate is an estimate.
Smaller things
- Six live knobs in an options drawer; no config editing, no restart.
- The size a session shrinks to after compaction is now measured from your own compactions rather than assumed.
- Prices reload from the pricing file on every fetch.
The first pull request
Someone else sent the first PR, fixing a path bug I would never have hit myself: path.dirname behaves differently per platform, so Windows-style paths collapsed to . on a Linux host. Merged, and grateful for it.
Open
One flaky test, and I know exactly why: both suites write the same two state files while the runner executes them in parallel. Not fixed yet.