isutools Dev Diary — Building an ISUCON Profiler in a Day with Claude Code and Codex

isutools is an all-in-one profiler for ISUCON. What it does is in the introduction; what it scored is in the tuning log. This is the third post: how it was built, and what I am still unsure about.

The timeline is what the git log says: the first commit (the design doc) at 21:22 on August 3, the v1.0.0 tag at 01:18 on August 4, with private-isu being tuned from 0 to 541,650 points in parallel. Here is how it went.

Aug 3, daytime — it started with a teammate's everyday tool#

This did not start as my own project. go-sql-logger is a SQL logger that a teammate of mine uses all the time, and I was using it for private-isu practice the same way.

While using it, I started wanting more: view the results in a browser, pre-sorted, with pprof and nginx logs in the same place. To be honest, I had neither the implementation skill nor the time to build that myself. So I had been extending it with Claude Code and Codex — that is the accurate version of events.

As the extensions drifted further from the original shape, and because the original repository has no license file, I decided to rebuild it as a separate module — with credit to the source of the idea (the driver-wrapping approach is still the core of isutools today).

21:22 — the first commit is a design doc#

The repository's first commit is DESIGN.md, not code. Requirements were fixed in writing first:

• Support MySQL / PostgreSQL / MariaDB / nginx / Apache / GraphQL / HTTP1-3 / WebSocket
• App changes within 3 lines; overhead verified under 2%
• git hash (+dirty) and host info (CPU/cores/memory) on every report
• Snapshot-first: measure remotely, carry the files home, open the dashboard locally
• TDD with 80%+ coverage

"Snapshot-first" was added at 21:25 and shaped everything after it. The live view is a bonus; the real product is the file each benchmark leaves behind. Run history, diffs, and score records all followed from that decision.

The design doc itself was reviewed before implementation: I had a different LLM critique it, which flushed out ambiguities in the on/off contract and persistence.

21:49 — the first working thing#

The first implementation commit came 27 minutes after the design doc: SQL normalization (literals masked into WHERE id = ?), aggregation, build info, host info, a zero-dependency web UI, and the admin server. Tagged v0.1.0.

The one-line integration rides on a driver proxy (shogo82148/go-sql-proxy):

db, err := sqlx.Open(isutools.SQLDriverName("mysql"), dsn)

If measurement fails, the app must not go down (fail-open). For a contest tool I consider that a precondition.

22:37–23:40 — the dashboard, and reversing the auth design#

• 22:37 v0.2: the home page becomes a dashboard; DB schema inspection (DSN auto-captured from the first connection, index list fetched on every reset); HTTP / access-log / process collectors
• 22:57–23:02 The auth reversal. The first cut required a token everywhere, and dogfooding made it clear I could not stand not opening localhost casually. The final shape: loopback needs no auth; remote exposure requires a token (refused otherwise); SSH-tunnel setups opt into unauthenticated mode via an explicit environment variable. Fail-open measurement, fail-closed exposure
• 23:10 Home becomes a run index: every benchmark adds a timestamped row with its score and git revision. Using it is what made me realize that "which commit scored what" never getting lost is the most valuable thing the tool does
• 23:25 Timestamps pinned to JST (time.FixedZone, works in containers without tzdata)
• 23:40 pprof integration — CPU profiles captured automatically on reset

All the while, private-isu tuning ran in parallel. The loop — ship a feature, use it in battle immediately, discover what's missing, next commit — turned over every 15–40 minutes.

00:12–01:18 — the advisor, then v1.0#

• 00:12 v0.5: whole-machine CPU busy/idle breakdown. This is the feature that revealed "the app got faster but the hardware is only 11.6% busy"
• 00:39 v0.6: the advisor. It reads MySQL, nginx, OS, and Go settings and flags unconfigured ISUCON staples. On its first run it caught three in my own environment — no interpolateParams, no gzip, an undersized buffer pool — worth 16% in score once applied. The author became the first beneficiary
• 00:56 v0.7: path normalization rules, the counter API, WebSocket/SSE separation
• 01:07 v1.0: run-to-run diff, session User Flow, a k6 example, and the ABBA measurement script
• 01:18 ABBA result (off→on→on→off) recorded at -0.58% (within noise), and the v1.0.0 tag

The next morning: README polish, these blog posts, and more tuning to close at 541,650.

About the development setup#

As written above, most of the implementation was done by Claude Code and Codex. My part was deciding requirements, making the calls at design forks, reviewing what came back, and verifying on the actual environment.

Looking back, three things made it fast:

1. The first commit was a design doc. Agents are weakest against ambiguous specs; freezing requirements, non-functionals (overhead budget, the auth policy), and non-goals in writing is what kept parallel work from colliding
2. TDD and the race detector as merge gates. Nothing generated gets merged unless every package passes tests and -race. That safety net is why midnight speed didn't cost quality
3. Dogfooding as the final test. Every feature went straight into private-isu the same day, so the benchmarker — a merciless integration test — ran on every change

There was one accident: both agents edited the same tree simultaneously, and a half-written file got swept into an unrelated commit. Recovery took a quarantine branch plus a temporary index (GIT_INDEX_FILE); the rule since then is "only one agent touches the tree at a time."

What got easier, and what I now worry about#

Some honest reflections.

It has unquestionably become easier. Studying for ISUCON used to mean spending a whole day standing up a practice environment, configuring alp and pt-query-digest, arranging where logs go — and sometimes the weekend ended there. This time, I built the exact measurement environment I wanted in a day. I don't think "setup is too heavy" will ever again be the reason I skip practice.

At the same time, two worries have grown.

Am I actually learning? The hands that wrote the code were the agents'; I only judged and reviewed. The design forks and the reading of measurement results do feel like they stayed with me — but if you asked me to write the SQL normalizer myself, I still couldn't, same as before. The suspicion that I merely feel like I understand has not gone away.

Will this matter in a real contest? If everyone can use AI the same way at ISUCON (regulations permitting), being able to build a tool in a day is not an edge. If the only remaining difference is the quality and speed of judgment — which measurement to trust, which fix to bet the limited eight hours on — then what exactly should practice train? Now that setup takes no time, that question is laid bare.

I don't have answers yet. But the one thing AI cannot practice for me is reading the numbers and deciding the next move, so that is where I intend to spend the time.

Timeline#

Lessons#

1. A design doc doesn't slow you down; it speeds you up. In parallel development, a written contract is the only thing that prevents collisions
2. Be your own first user. The advisor's three findings and the auth reversal both came out of dogfooding
3. Fail-open measurement, fail-closed exposure. A tool earns trust through that asymmetry
4. Cut releases in small increments. Eleven tags in one night looks like a lot, but tags are why "which feature moved which score" is still traceable

Links#

• GitHub: github.com/ekusiadadus/isutools
• Feature tour: isutools — a One-Line All-in-One Profiler for ISUCON
• Field log: Breaking 500k on private-isu in One Day with isutools
• ISUCON resources: ekusiadadus/awesome-isucon