Blog
How Jiro Hill-Climbed APEX-Accounting
Accounting agents are easy to demo and hard to trust. We wanted to see whether one could get better on its own, the way a new staff accountant does: by doing the work, getting reviewed, and writing down what it learned.
Jiro keeps what it learns in Accounting Operating Procedures (AOPs): instructions and scripts it runs for each kind of task. After every round of review, it updates the AOPs where it went wrong.
On APEX-Accounting, a month-end close benchmark from Mercor and Ramp, Jiro’s average task score went from 64.8% to 97.4% in three rounds. Each task took under five minutes; APEX estimates an accountant needs 2.4 hours.
What is APEX-Accounting
APEX-Accounting is Mercor's benchmark for month-end close work, built with Ramp. Accountants built ten companies frozen at month-end, wrote 160 close tasks inside them, and graded each task with an expert rubric. An eleventh company is public: Sterling, Marsh & Associates LLP, a Philadelphia law firm closing December 2024. Its world holds 90 files from Clio, QuickBooks, Gusto, two bank accounts and a shared drive of workpapers.
We worked on that firm's ten open tasks: reconciliations, accruals, schedules and variance analysis, graded against 89 checklist items.
Accounting Operating Procedures
An Accounting Operating Procedure (AOP) is a package of instructions for one kind of work, like payroll reconciliation or AR aging. It combines plain-English instructions with deterministic scripts. Jiro executes the matching AOP on each task. AOPs are fully version controlled: a team can read and edit them, and every change is tracked.
ar-aging-analysis
scripts
payroll-journal-entries
scripts
--- name: ar-aging-analysis description: Calculate accounts receivable aging exposure percentages from aging workpapers or invoice detail, validating overdue dates, bucket assignments, balances, and the denominator. --- # Accounts receivable aging exposure An aging report can tie in total while assigning an invoice to the wrong bucket. Validate classifications before calculating exposure. 1. Establish the reporting date, requested source, aging basis, bucket boundaries, and treatment of credit balances. Inspect every relevant worksheet, including invoice detail and methodology. Stay within the requested workpaper unless a reconciliation requires additional support. 2. Read spreadsheets with formulas and cached values separately when necessary. An empty formula cache is missing data, not zero. Exclude titles, subtotals, and grand totals from detail aggregation; validate invoice uniqueness.
The loop
Each round, Jiro works every task, a grader marks what failed, and a Jiro agent rewrites the AOP where the method broke. The new AOPs are tested on the same tasks and kept only if the score goes up, so every round starts from the best version so far.
Jiro Optimization Loop
One round: rollout traces, failed checks, a coach and an AOP change, around the Jiro agent, which works from traces, evals, the repo and skills. A person reviews failed checks and can disregard them; a change is kept only if the score rose, and the next round starts from the best version.- Rollout trace. Jiro works all ten tasks, three attempts each, in isolated sandboxes holding the task files and the current AOPs. Every step it takes is recorded.
- Failed checks. A grader model scores every answer against the expert rubric, one check at a time, with a reason for each.
- Coach. A stronger model, at maximum reasoning, reads what failed with every expected amount masked, reworks the task in its files, and rewrites the AOP where the method broke. Three of these coaches work in parallel. Each edit is re-run on all ten tasks and graded again.
- AOP change. We pick the best edit and keep it if the score went up.
Results
After three rounds of coaching, Jiro’s average score across the ten tasks rose from 64.8% to 97.4%. Each task still took about two minutes: the AOPs made Jiro more accurate without making it slower.
- An accountant, per task
- 2.4 hours
- Jiro, per task
- under 5 minutes
Accuracy, round by round
All ten tasks, before and after
- Settlement holdback entry50% → 100%
- Payroll reconciliation50% → 100%
- Payroll and partner payment entries53% → 94%
- Year-end WIP roll-forward100% → 80%
- AR reconciliation, Clio to QBO100%
- Client cost advance reconciliation75% → 100%
- Payroll reconciliation by practice group72% → 100%
- Collections by practice group88% → 100%
- Partner guaranteed payment true-up0% → 100%
- AR aging exposure60% → 100%
| Round | Average score | What changed |
|---|---|---|
| Start | 64.8% | No AOPs. |
| Round 1 | 79.6% | Five AOPs written from scratch, two with scripts: payroll entries, payroll reconciliation, AR aging, client cost reconciliation and partner compensation. |
| Round 2 | 96.7% | Fixed the partner compensation cap and added AOPs for settlement entries and collections variance. |
| Round 3 | 97.4% | Final balances keep the posted accruals. Scored on 30 fresh attempts. |
What Jiro learned
By the end, Jiro had written seven AOPs of about 500 words each. They read like a senior's review notes: where the numbers come from, what to check, in what order, and the mistakes to avoid.
Partner compensation true-up
Translate the agreement into a rule table: eligible activity, measurement period, allocation order, cap unit, lookback comparator, thresholds, marginal rates, and effective dates. Obtain the agreement, compensation schedule, administration register, realization schedule, time, matter ownership, prior-year true-up, and actual postings.
Establish the compensation basis
Use effective dates inside admission/promotion records, which can differ from record dates. Ignore recruiting offers and unrelated events. Prorate guarantees when required; do not prorate annual performance thresholds unless the agreement permits it.
Identify the authoritative source for each input. A compensation-specific realization schedule can contain adjustments absent from a general invoice export. Reconcile it to approved time and invoice deductions, but do not silently replace its designated net-realized billing. Check period, version, approval, and adjustment coverage. If authority remains unresolved, label the primary basis and quantify the alternative. Collections, invoice values including costs, and approved time less contractual deductions are different measures.
One task, before and after
Task 26 is a year-end true-up of partner pay at the law firm. Each partner earns performance pay on the matters, or client cases, they bring in, based on what those matters billed after write-downs. When a second partner works enough hours on a matter, its credit is split 70/30. The partnership agreement caps the credit from any one matter at $400,000, shared by everyone on it.
Jiro has to work out each partner's pay from the agreement, the billing schedules and the time records, then propose the true-up entry. The right answer is $227,257.06 of performance pay.
Before coaching
$404,620.27 (wrong)
Split each matter between partners, then capped each share at $400,000, so one matter could credit more than $400,000.
After coaching
$227,257.06 (correct)
Capped each matter at $400,000, then split it between partners.
What changed was one rule in the AOP:
Then apply the cap at its stated unit. A cap on each partner's credit from a matter belongs after that partner's allocation, not automatically on gross matter billing before splitting.
Resolve the cap's unit, not just its amount. A matter-wide limit restricts combined credit derived by all recipients from that matter; splitting must not multiply the limit.
What's next
We're looking for accounting teams to deploy Jiro with. We'll map out your financial workflows and build evals from your own closed months, so you know how well AI does on your work before anyone relies on it. From there, the same loop captures your team's institutional knowledge in AOPs and keeps improving them.
Jiro is an applied research lab developing products to accelerate accounting workflows.
Sources: APEX-Accounting by Mercor and Ramp (open World 9 tasks, CC BY 4.0); Mercor APEX-Accounting leaderboard, checked October 2026; Jiro eval runs, October 2026.