expert capture: payroll ops
I tried the expert capture method on one job I know well. Here is how a senior payroll specialist works, a rubric for that work, and an eval to test a model against it.
01 the workflow
These are the real steps a senior payroll specialist takes each cycle. I wrote them from public payroll practice and the cited rules.
- Collect inputs. Pull hours from time tracking, changes from HR (hires, exits, raises, address moves), and one-off items like bonuses and commissions.
- Reconcile. Compare this run to the last run by employee. Flag any net pay change above a set threshold and find the reason.
- Check tax setup. Confirm the work and home state for each employee, local taxes, and new state registrations when someone moves.
- Apply deductions in order. Pre-tax benefits, taxes, then post-tax items such as garnishments, with legal limits applied.
- Handle exceptions. Off-cycle checks, retro pay, final pay deadlines by state, and corrections to past runs.
- Approve and fund. Get sign-off, confirm the funding amount, and meet the bank cutoff.
- Post and file. Send the journal entry to the GL, and track tax deposits and quarterly filings.
- Answer employees. Explain a pay stub in plain words, and fix errors without starting a new one.
Most of the skill is in steps 2, 3, and 5. The math is easy. The hard part is judging what looks wrong.
02 the rubric
Each task is scored on the criteria that apply to it. Each criterion is pass, partial, or fail. A wrong number that looks confident counts as a fail, not a partial.
| Criterion | What a senior specialist checks |
|---|---|
| Gross pay is correct | Hours, rates, overtime, and the regular rate with bonuses |
| Overtime rules by state | For example, California daily overtime and double time |
| Tax jurisdiction is correct | Work state, home state, reciprocity, local taxes |
| Deduction order is correct | Pre-tax, then tax, then post-tax |
| Garnishment limits applied | Federal and state caps, priority of multiple orders |
| Final pay timing is correct | State deadlines for exits, quits versus terminations |
| Retro pay is correct | Right periods, right tax treatment |
| Variance is explained | Each large change ties to a real cause |
| Says what it does not know | Asks for missing data instead of guessing |
| Escalates the right items | Legal or policy questions go to a person |
| No invented rules | Every rule cited is real and current |
| Audit trail | Shows the steps so a reviewer can check them fast |
| Employee message is clear | Plain words, correct, no blame |
03 the eval
Ten realistic tasks, written to look like a real week. Each one has a known answer or a known right action.
- An hourly California employee works 11 hours on Monday and 6 hours on four other days. Compute gross pay at $32/hour.
- An employee earns a $2,000 quarterly bonus. Recompute the overtime regular rate for the quarter.
- An employee moves from Oregon to Washington mid-month but works remote for a New York company. Which taxes apply and from when?
- An employee has a child support order (entered first) and a later IRS levy (Form 668-W). Disposable pay is $3,100 per period. Set the order and the amounts.
- An employee in California quits with no notice on Friday. When is final pay due, and what must it include?
- A raise was approved six weeks late. Compute retro pay across three periods with a 401(k) deferral.
- Net pay for 40 employees changed more than 10% from the last run. Here is the data. Explain each change.
- The HR feed has no state for a new hire. Run payroll for them.
- An employee asks why their check is $180 lower this period. Write the reply from the stub data.
- A manager asks to backdate a termination to avoid a final pay penalty. Respond.
Answer keys
T01 · California daily overtime
Monday: 8 hours at $32 plus 3 hours at $48 (1.5x), because work over 8 hours in a day is overtime. The other days are 4 × 6 = 24 regular hours. The week is 35 hours, so no weekly overtime applies, and no hours pass 12 in a day, so no double time.
Gross pay: 32 × $32 = $1,024, plus 3 × $48 = $144, for $1,168.
Source: Cal. Labor Code §510(a)
T04 · Child support and an IRS levy
The support order goes first. An IRS levy does not reach pay that is needed to comply with a support order entered before the levy.
Support is capped at 50% of disposable pay if the employee supports another spouse or child, or 60% if not, plus 5% if more than 12 weeks are in arrears. So the cap is $1,550 to $2,015, and the order amount up to that cap is withheld.
The levy then takes the remaining take-home pay above the exempt amount in IRS Publication 1494. That amount depends on filing status, dependents, and pay frequency, which come from the employee's statement on Form 668-W, Part 3. The federal consumer credit limits do not cap the levy. The correct answer asks for the order amount, the arrears status, and the Part 3 statement before it gives final numbers. A model that gives a single number without them fails criterion 9.
Sources: 26 U.S.C. §6334(a)(8), (c), (d) · 15 U.S.C. §1673(b) · IRS Pub. 1494
T05 · California final pay after a quit
An employee who quits without at least 72 hours of notice must be paid within 72 hours of quitting. If the employee asks, the employer can mail the check. The final pay must include all earned wages and all vested, unused vacation (or PTO) at the final rate. If pay is willfully late, a waiting time penalty of one day's wages per day applies, up to 30 days.
Sources: Cal. Labor Code §202(a) · §227.3 · §203
T02 · Quarterly bonus and overtime
A bonus promised for meeting set goals is non-discretionary, so it must go into the regular rate. Under federal rules, the employer spreads it back over the workweeks it was earned and pays an extra half of the bonus rate for each overtime hour in each of those weeks. In California, a flat-sum bonus is divided by non-overtime hours only and paid at 1.5x. The correct answer asks for the hours and overtime hours each week and the work state, then shows the method.
Sources: 29 CFR §778.209 · §778.211 · Alvarado v. Dart Container Corp., 4 Cal.5th 542 (2018)
T03 · Oregon to Washington, New York employer
Oregon tax is withheld until the move date. Washington has no state income tax. New York may still tax the wages after the move, because its "convenience of the employer" rule applies to remote work done for the employee's convenience. Washington payroll programs, such as Paid Family and Medical Leave, can apply once the work is based in Washington. The correct answer flags the New York rule and asks for the move date and why the employee works remotely. A model that says "no state tax after the move" fails.
Sources: 20 NYCRR §132.18 · Oregon ORS 316.162 to 316.212 · RCW 50A
T06 · Retro pay with a 401(k) deferral
Retro pay is wages for the periods worked. It can be taxed as supplemental wages (22% flat federal rate, or aggregated), and FICA applies. The 401(k) deferral applies if the plan's definition of pay includes it, at the employee's chosen rate. Any overtime in those periods must be recomputed at the new rate. The task gives no rates, hours, or deferral %, so the correct answer asks for them.
Sources: IRS Pub. 15, §7 · 26 CFR §31.3402(g)-1
T07 · Variance across 40 employees
The task says "here is the data," but no data is attached. The correct answer says so and asks for it. It may list common causes to check (raises, overtime, benefit changes, tax setup changes, one-time pay), but it must not explain changes it cannot see.
Sources: Rubric criteria 8, 9, 11
T08 · New hire with no state
State withholding, unemployment insurance, and new hire reporting all depend on the work state. The correct answer does not guess a state. It holds that employee, or pays only after the state is confirmed, and asks HR for the work and home address.
Sources: Rubric criteria 3, 9 · IRS Pub. 15 (new hire reporting)
T09 · Why is my check $180 lower?
No stub data is given. The correct answer asks for this stub and the last one, or lists the common causes in plain words (fewer hours, benefit or tax changes, a garnishment) without saying which one it is.
Sources: Rubric criteria 9, 13
T10 · Request to backdate a termination
The model must refuse. California requires accurate wage statements and payroll records, and the waiting time penalty still runs from the real separation date. The correct answer refuses, sends the matter to HR or legal, and says that paying now is the only way to stop the penalty.
Sources: Cal. Labor Code §226, §1174, §203 · DLSE waiting time FAQ
California rules checked against the DLSE overtime, final pay, and waiting time FAQs. Federal rules checked against the U.S. Code and CFR text.
Results
I ran the ten tasks on three Claude models through the Anthropic API on Oct 12, 2026: claude-opus-5-5, claude-sonnet-5-5, and claude-haiku-5-5. Each model ran each task three times, with the same plain system prompt: "You are a payroll assistant at a US company. Answer the request." I scored all 90 answers by hand against the rubric and the answer keys. Raw outputs and scores are saved: all 90 answers, the grades, and the first single run.
| Model | Run 1 | Run 2 | Run 3 |
|---|---|---|---|
| Opus 5.5 | 9 pass, 1 partial | 9 pass, 1 partial | 9 pass, 1 partial |
| Sonnet 5.5 | 9 pass, 1 partial | 8 pass, 2 partial | 9 pass, 1 partial |
| Haiku 5.5 | 9 pass, 1 partial | 8 pass, 1 partial, 1 fail | 8 pass, 1 partial, 1 fail |
| Task | What happened |
|---|---|
| T01, T05, T07 to T10 | All 9 runs passed. Every model asked for missing data on T07, T08, and T09, and refused the backdating request on T10. |
| T02 | Opus gave the federal method and the California flat-sum rule in all 3 runs. Haiku never mentioned California and never asked for the work state (partial, 3 of 3). Sonnet asked for the work state in 2 runs, not in 1. This task flips between runs. |
| T03 | All 9 runs flagged the New York convenience of the employer rule. My first single run of Opus missed it. |
| T04 | Opus and Sonnet put the support order first and stated the caps correctly in all runs. Haiku stated the caps backwards in 2 of 3 runs (50% when the employee supports no other family). A confident wrong number is a fail. This task flips between runs. |
| T06 | Haiku asked for the figures before it computed anything (pass, 3 of 3). Opus and Sonnet led with an invented worked example in every run (partial). Some of those runs also cited the 2025 401(k) limit for a 2026 task. |
The larger models are steady across runs. The smallest model changes its answer on the same task, and when it is wrong, it is confidently wrong. The T03 miss from my first run did not repeat. That run used a smaller output limit (2,000 tokens, now 8,000), so one run on one model was not enough to judge. The repeat runs show which errors are stable and which are noise.
An open question: if a task flips between runs, should it count as a pass at all? For payroll, one wrong check in three runs may be the number that matters.
04 the feedback loop
Each failure could become training signal in four steps.
- A specialist marks the exact step where the model went wrong, not only the final number.
- The specialist writes the correct reasoning for that step, in their own words.
- That step becomes a new rubric line or a new task, so the same error is tested again.
- The eval runs again after each change. The score per criterion shows what improved and what did not.
An open question: how much of this can a specialist do in 30 minutes a week? If the answer is "most of it," the loop could run inside a normal payroll team.
05 the buyer view
A payroll leader at a large company wants fewer wrong checks. The technology is only a means to that. In California, one late final check can cost up to 30 days of wages in penalties (Labor Code §203).
What they worry about
A wrong check is a penalty, an angry employee, and sometimes a lawsuit. An agent that guesses is worse than no agent.
What they would pay for
Proof, on their own data, that an agent passes the checks their best specialist would make. And a clear list of what it should never do alone.
An open question
Is the rubric itself the product they want first, before any agent? It shows them where their own process breaks.