For any fixed set of spells, getting the game world right is a finite problem. The project works it systematically, not bug by bug.
How a bug is judged#
Agents maximise win rate, so they will find any server bug that helps them win. A bug's severity is therefore whether an agent could exploit or learn from it, not how visible it is to players:
| Severity | Meaning | When it must be fixed |
|---|---|---|
| High | Changes who wins, or can be exploited deliberately | Before training on the mechanic |
| Medium | Distorts some matchups or maps | Before the affected comp trains |
| Low | Minor or rare, a judgement recorded with its reason for each bug; those re-checked live are in server bugs checked | Tracked |
The tools#
- Known bugs. The first pass read every open AzerothCore report touching arena, line of sight, pathing, diminishing returns and queueing, triaged each by the scale above, and did a class-by-class pass for each comp's spells.
- A spec. Game-rule claims kept as data: a cooldown, a coefficient, a range, a diminishing-returns category. Each claim has a source from the evidence ladder and a confidence, and many carry a recipe that tests them live.
- Data diffs. The server keeps its own copy of much of the game's data. A tool compares it field by field with the data files of the 3.3.5a client and of the WotLK Classic client, and lists every difference: the start of a datamining practice. The rogue came out clean. The priest's spell coefficients differ from the client data, which is being worked through.
- Live tests. Headless 3.3.5a clients log in, queue, fight on random arenas and check the outcome, through the real protocol. The full suite takes about 45 minutes and runs daily and before server changes go live.
Two rules keep the tests honest. A test that asserts a game rule cites its evidence. And every new check is shown failing on a broken setup before it is trusted: twice a check here passed for the wrong reason.
What has been found#
Six server bugs have been fixed so far, each with a test that failed before the fix: poisons scaling too strongly with attack power, re-cast crowd control resetting, buffs on pets surviving the gates, the Dalaran Sewers waterfall not blocking sight in its first cycle, Focused Will ignoring resilience, and professions' enhancements usable without the profession. Several reported bugs did not reproduce. Every checked mechanic, with its source and status, is on checked mechanics.
Checking the checks#
- Invariant monitors watch rules that hold whatever the details: resources stay in bounds, nothing happens before the gates open, crowd control respects diminishing returns, cooldowns are respected. The first ones are built; they will run over every recorded match, and a surprising win is checked for a bug before it counts as good play.
- Refutation. Before a fix for a high-severity bug is built, a second agent tries to prove the claim behind it wrong.
- Calibration. Confident claims later found wrong are counted. So far: none. If two turn up within a month, the bar for evidence rises to two independent sources.
- Flaky tests are re-run in soaks to measure how often they fail, never re-run until they pass.
What comes next#
- The same passes for the Frost Mage, and then for each class a new comp brings.
- Sweeps of scenarios that probe known kinds of error before training.