Should an IT services firm build or buy an AI assistant for pay and HR questions?
Build it in-house, in three gated releases. Payback in about 19 months from kickoff, 22 including the Sprint fee. Target: 6 in 10 HR questions answered without HR. No employee pay data reaches a model provider.
I ran this Sprint in February 2026. The client isn't named here, so I have changed the numbers and some details to keep it that way. The findings and the decisions are as they were. It follows the same layout as the report the steering committee received. Amounts are in US dollars.
Contents
- Executive summary
- The question and how we answered it
- Where HR's time goes
- Which questions to automate first
- Build, buy or fix the process
- Business case and sensitivity
- Architecture, controls and data protection
- Risks and how they are handled
- Roadmap, governance and KPIs
- What we recommend against
- Decisions and next 30 days
- Appendix: method and assumptions
Build one in-house assistant for pay, policy, leave and salary advances. It pays back about 19 months from kickoff and keeps payroll data inside the firm.
- Situation
- 950 of 1,200 employees are consultants at client sites in 14 states. Every pay, leave and policy question goes to HR by email.
- Complication
- HR spends about 500 hours a month on 2,400 questions, answers take 1.5 days, and headcount grows 15% a year. Two vendors were pitching packaged assistants.
- Resolution
- Build one assistant on the firm's cloud, connected to Workday, ADP and one clean policy library. Release it in three phases, each gated on measured results.
The answer. Build, do not buy. $107,000 one-time, about $5,200 a month to run, first users in week 10, full rollout in 24 weeks. Payback about 19 months from kickoff, 22 months including the $25,000 Sprint fee, 13 months at full run rate.
Why
Most of what HR answers is already written down somewhere. Payslips, leave, policy and salary advances make up 8 in 10 questions, and each answer sits in Workday, ADP or a policy file. The plan targets 75% of those answered without HR, about 6 in 10 of all questions. The day 6 prototype got 92% of its answers right.
The biggest category is also the one nobody could sell us. A consultant's payslip question depends on their client, their rate, their timesheet and the state they worked in that week, and neither vendor could see that. Over three years from kickoff, building nets about $148,000. The service platform would have lost about $54,000.
The numbers hold when pushed: run-rate payback stays under 18 months whichever single assumption we stress. What worried us more was legal exposure on salary advances, and whether consultants would use the thing at all. Both now have an owner and a gate (Sections 7 and 8).
Decisions requested from the steering committee
- Approve phase 1 (pay and policy, through the week 14 decision): $74,000 of the $107,000.
- Approve the stop rule: continue only if run-rate payback, projected from 4 weeks of pilot use, is 18 months or less.
- Name an HR product owner (1 day a week) and commission a 14-state counsel review of advance rules.
- Decline both vendor proposals as they stand.
The payroll question that shaped the pilot
A representative case from the day 6 test set; wording and figures changed.
| Employee asked | Prototype answered | Payroll's approved answer |
|---|---|---|
| "My take-home dropped $166 this pay. I moved from the Dallas client to the Chicago one on the 14th. Why?" | "Changes in take-home pay are usually caused by changes to your federal withholding (W-4) or benefit deductions. Please check your W-4 elections." | "Your work state changed from Texas to Illinois on the 14th. Texas has no state income tax; Illinois withholds 4.95%. 8 of the 11 working days this period were in Illinois, so $166 of Illinois tax was withheld." |
| What broke | The prototype could not see ADP's work-location history or per-state tax lines, so the model filled the gap with a generic answer. No figure was wrong; the explanation was. |
| What engineering builds first | Ticket HR-PAY-01: map those ADP fields, compute the change in code, and send anything not fully explained to payroll. Owner: the firm's backend lead. Six acceptance tests. |
| What unlocks the next $33,000 | The week 14 scorecard. Targets: 95% correct (prototype observed 92%), zero wrong pay figures (prototype observed 0 of 300), 70% answered without HR, 60% pilot usage. |
See the engineering handoff: the ticket, its tests and the scorecard, as the firm's team received them.
What changed our plan. Going in, I assumed the model would be the weak point. It wasn't. Of the 24 answers the prototype got wrong, 19 came down to data or policy: an old policy version, an ADP field we hadn't mapped, a rule HR had never written down. So we changed the order of work. Field mapping became a condition for launch, the policy clean-up moved in front of the build, and anything the data can't fully explain now goes to payroll rather than getting a best guess.
| Why prototype answers failed (24 of 300) | Share | Fix before launch | Owner |
|---|---|---|---|
| Old policy version retrieved | 3% | One source per policy, with effective dates | HR product owner |
| ADP field not yet mapped (state tax codes, retro pay) | 2% | Complete the mapping; add cases to the test set | Tech lead, payroll lead |
| Question too vague to answer | 1.7% | Assistant asks one clarifying question | Tech lead |
| Policy silent or contradictory | 1.3% | HR decides the rule; it goes on the rules screen | CHRO |
| Total | 8% | Release bar: 95% correct |
What the client gets on day 10
| Deliverable | What it settles |
|---|---|
| Decision memo | Build or buy, the options scored, and the four decisions above |
| Business case model | Month-by-month cost and benefit from kickoff, editable by finance, with the stop rule built in (Exhibit 7) |
| Target architecture and data rules | What stays inside the network, masking, approved calculations, audit trail (Section 7) |
| Release bar and test-set design | 300 questions, 95% correct, zero wrong pay figures; why the prototype missed and who closes each gap (Section 1) |
| Policy rules and escalations | Which requests a person must decide (Exhibit 11) |
| Risks, gates and stop rule | Owners and pass conditions for each release (Sections 8 and 9) |
The firm's own team plans and builds from these. Sarthi reviews progress against the bar at each gate.
We agreed the decision criteria with the CFO on day 1, before looking at any technology.
| Criterion | Threshold | Set by |
|---|---|---|
| Payback | 18 months or less after full rollout | CFO |
| Employee data | No identifier or pay figure leaves the firm's cloud unmasked | CIO and IT security |
| Answer quality | Zero wrong pay figures; 95% correct on an HR-approved test set | CHRO |
| Speed | Employees using something within one quarter | CHRO |
Exhibit 1Five questions, each answered by a specific analysis
Source: Sprint workplan, agreed with the steering committee on day 1.
Ten days, one output per day
| Day | Work | Output |
|---|---|---|
| 1 | Interviews with CHRO, payroll lead, 3 HR generalists, IT director; half a day shadowing HR (41 requests in 4 hours) | 14 candidate problems |
| 2 | 3 months of HR inbox (7,100 messages), masked in the firm's cloud and classified; 8 consultant calls | Volume and time by question type |
| 3 | Where each answer lives, API access, data quality, owner | Data readiness map |
| 4 | Wage statement, deduction and advance rules in 14 states, with employment counsel | Legal limits per use case |
| 5 | Scored 6 use cases with the CHRO and payroll lead | Shortlist of 4 |
| 6 | Prototype on 300 masked real questions; 50 cross-access attempts | Accuracy and access results |
| 7 | Two vendor demos and proposals; four options scored | Build-or-buy call |
| 8 | Cost build-up with IT, benefits with finance, stress tests | Business case, stop rule |
| 9 | Architecture and security review; release plan with HR | Architecture, risks, roadmap |
| 10 | Read-out to the steering committee | This report, four decisions |
HR spends about 500 hours a month on questions whose answers already sit in Workday, ADP or a policy file.
Exhibit 2Six question types make up all 2,400 monthly questions
| Question type | Per month | Minutes each | HR hours | Answer already in a system? |
|---|---|---|---|---|
| Payslips and deductions | 820 | 10 | 137 | Yes, ADP |
| Salary advances | 220 | 25 | 92 | Yes, written rules applied by hand |
| HR policy (travel, benefits, overtime) | 410 | 12 | 82 | Yes, policy documents |
| Timesheets and client billing | 290 | 15 | 73 | Partly, nightly export |
| Leave balance and requests | 500 | 8 | 67 | Yes, Workday |
| Visa and employment letters | 160 | 20 | 53 | Partly, needs legal sign-off |
| Total | 2,400 | 503 |
Source: HR inbox, 3 months, 7,100 messages; time study of 41 requests. Note: minutes are median handling times.
What drives the volume
Two things surprised us. The load isn't spread evenly: most payslip questions land in the three days after a pay run, exactly when the same two payroll people are approving advances. And many "payslip" questions are really tax questions. Consultants move between client sites, sometimes across a state line, and "why did my take-home change?" can only be answered from that person's own pay history.
The rest is housekeeping. Some policies exist in three versions (intranet, shared drive, old emails), so answers drift. Slow replies make more work too: one consultant in five asks the same question twice.
Four question types clear the bar. Timesheets wait for better data; visa letters stay with counsel.
Exhibit 3Payslips, advances, policy and leave combine high value with ready data
Note: value is HR time at $48 an hour, with 75% of questions answered without HR. Build threshold: $25K a year.
| Use case | Value / year | Data readiness | Risk | Call |
|---|---|---|---|---|
| Payslips and deductions | ~$59K | High: ADP API | Figures must be exact | Phase 1 |
| HR policy | ~$35K | Medium: clean up first | Low | Phase 1 |
| Leave | ~$29K | High: Workday API | Low | Phase 2 |
| Salary advances | ~$40K | High: written rules | High: state wage law | Phase 3 |
| Timesheets | ~$31K | Low: nightly export | Low | Not yet |
| Visa letters | ~$23K | Partial | High: counsel sign-off | Never |
Advances are worth more than policy or leave but go last: they move money, so the assistant earns trust on read-only questions first.
Building returns the most over three years, because only a build can read each consultant's own pay data.
We looked at four ways forward, including not using AI at all.
- A. Clean up the policies and publish a searchable FAQ. Cheap, and worth doing whatever else happens, so it's folded into phase 1.
- B. Switch on Workday's built-in assistant. It handles leave and policy well. It can't read ADP, though, and ADP's own assistant only covers pay inside ADP, so consultants would have two assistants that each know half the story. Advances need both.
- C. Buy an employee service platform on top of both systems. Good with policy documents; the payroll connectors were quoted as custom work.
- D. Build on the firm's own cloud, reading Workday and ADP directly.
Microsoft's Employee Self-Service agent was on the market, but it needs Microsoft 365 Copilot licences the firm didn't have, so we didn't score it.
Exhibit 4The build scores highest on the criteria weighted most
| Criterion | Weight | A. Process | B. Workday | C. Platform | D. Build |
|---|---|---|---|---|---|
| Question volume covered | 30% | 1 | 2 | 3 | 5 |
| Employee data control | 25% | 5 | 4 | 2 | 5 |
| 3-year cost | 20% | 5 | 4 | 3 | 3 |
| Time to first value | 15% | 5 | 4 | 3 | 3 |
| No lock-in | 10% | 4 | 2 | 2 | 5 |
| Weighted score (of 5) | 3.7 | 3.2 | 2.7 | 4.3 |
Note: scores 1 to 5, weights set on day 1, before any vendor demo.
Exhibit 5Over three years, only the build clearly earns back more than it costs
| Option | Covers | 3-year cost | HR hours freed / month | 3-year net value |
|---|---|---|---|---|
| A. Fix process only | Policy FAQ | $5,000 | ~20 | +$29,000 |
| B. Workday's assistant | Leave, policy; no ADP pay data | $144,600 | ~110 | +$34,000 |
| C. Service platform | Policy, leave, advances; payslips at handbook level | $334,200 | ~180 | -$54,000 |
| D. Build in-house | All four, from each employee's own data | $281,200 | ~280 | +$148,000 |
Source: the two vendor proposals received in February 2026 (B: $15,000 setup, $3 per employee per month; C: $75,000 implementation and custom work, $6 per employee per month). Prices are rounded and changed like every figure here; they are not Workday or ADP list prices. Note: net value is HR time at $48 an hour minus all costs over 36 months from kickoff, using the same monthly model and rollout ramp as Exhibit 7. At 15% headcount growth, C's licences rise to about $375,000.
$107,000 one-time pays back about 19 months from kickoff, 13 months at full run rate. The share of questions answered without HR is the number to watch.
Exhibit 6Cost to build and run
| One-time | Amount |
|---|---|
| Engineering: 20 to 22 developer-weeks over 24 weeks, costed at 22 | $79,000 |
| Security review and penetration test | $15,000 |
| ADP API access and setup | $8,000 |
| Policy clean-up | $5,000 |
| Total | $107,000 |
| Per month | Amount |
|---|---|
| Upkeep: one developer, about 20% of their time | $3,100 |
| Hosting and vector database | $1,200 |
| Model usage, about 15 cents a conversation | $400 |
| Vault and key management | $300 |
| Monitoring and logs | $200 |
| Total | $5,200 |
Note: developer-weeks at $3,600 (about $185,000 a year, loaded). A small model routes each question; a larger one drafts the answer.
| Return | Assumption | Amount |
|---|---|---|
| HR time freed | 75% of covered questions answered without HR | 280 hours / month |
| HR capacity recovered | $48 per loaded HR hour, set by finance | $13,440 / month |
| Running cost | Exhibit 6 | $5,200 / month |
| Net benefit | $8,240 / month | |
| Payback at full run rate | $107,000 ÷ $8,240 | 13 months |
| Payback from kickoff | With the rollout ramp (Exhibit 7) | 19 months |
| All-in, with the Sprint fee | Adds $25,000 spent before the decision | 22 months |
Exhibit 7Modeled net value from kickoff turns positive in month 19, or month 22 counting the Sprint fee
| Month | 3 | 6 | 12 | 18 | 24 | 36 |
|---|---|---|---|---|---|---|
| Modeled net value | -$69K | -$101K | -$53K | -$3K | +$47K | +$148K |
| Cash view: two hires avoided | -$70K | -$125K | -$93K | -$39K | +$15K | +$123K |
Note: build costs from Exhibit 6, spread over weeks 1 to 24; running cost from week 10. Benefits ramp with the releases: pay and policy from the week 10 pilot, leave from month 5, advances from month 7; full run rate from month 9. Modeled value counts recovered HR hours at $48. The cash view counts only the two HR hires, assumed for months 7 and 10 (about $170,000 a year): cash payback in month 23, or 26 with the Sprint fee.
Exhibit 8Run-rate payback stays under 18 months across realistic ranges of all four drivers
Note: each bar moves one driver and holds the others at base case.
HR's hourly cost moves payback the most, but it isn't really in doubt: finance gave us that number on day 1. What nobody can know yet is how many questions the assistant will close on its own. That is what the pilot is for, before the rest of the budget is spent.
Exhibit 9The stop rule turns the pilot result into a decision
| Scenario | Answered without HR | Hours freed | Net / month | Run-rate payback | Decision |
|---|---|---|---|---|---|
| Expected | 75% | 280 | $8,240 | 13 months | Go |
| Pilot gate | 70% | 265 | $7,520 | 14 months | Go |
| Weak | 60% | 226 | $5,648 | 19 months | Release leave; defer advances |
| Stop | 50% | 189 | $3,872 | 28 months | Stop |
Note: run-rate payback, projected from 4 weeks of pilot use. Up to 18 months, go; 18 to 24, release leave but defer advances; over 24, stop. Agreed with the CFO.
For the CFO: the model values recovered HR capacity (280 hours a month, about 1.75 people). Cash shows up only in the hiring budget: the two HR generalists planned for next year's growth, about $170,000 a year, are not hired. That budget is larger than the capacity value, so the model is conservative. Faster answers and fewer pay errors are not counted.
No real employee identifier or pay figure ever reaches the model provider, and every action can be explained a year later.
Exhibit 10Real employee data stays inside the firm's network; the model sees only masked values
How one question is answered
- The employee signs in with SSO; the assistant can read only that person's records.
- Names, IDs and amounts are swapped for masks ("EMP-4821", "AMT-07") before anything leaves the network.
- The model drafts the answer from the question, the policy passage and the masks.
- Inside the network, real figures are put back from ADP. Every pay figure comes from ADP's payroll APIs or from deterministic, tested code; the model only explains the result.
- If no approved calculation covers the question, the assistant does not improvise: it routes the question to payroll with the records attached.
Policy rules HR can change, and when a person decides
Policies lived in documents and in HR's heads. The build turns the deciding rules into settings HR edits on a rules screen, with every change logged: who, when, old and new value, effective date. Each decision records the rule version it used. Anything outside the rules goes to a person; the assistant never approves an exception.
Exhibit 11Requests that break a rule go to a person, with the reason attached
| Trigger | Goes to |
|---|---|
| Advance above the limit (half of net monthly pay) | Payroll lead |
| Second advance while one is being repaid | Payroll lead |
| Employee in first 90 days or serving notice | Payroll lead |
| Leave beyond balance or in a client blackout period | Manager, then HR |
| Pay dispute, termination, harassment or visa status | HR the same day; the assistant does not answer |
| No approved policy source, or low confidence | HR ticket |
An audit trail for every action
Every action is logged with the request, records read, rule and policy versions, model and prompt version, decision and approver. HR and counsel can replay any decision from the last 12 months; older records follow the firm's retention policy.
What IT security asked
| Question | Answer |
|---|---|
| Does the provider train on or keep our data? | No: enterprise terms, zero retention, through the firm's cloud account. |
| Can someone see another person's pay? | No: lookups are tied to the signed-in employee; 50 attempts failed in testing. |
| Can a crafted message trick it? | It can only read the user's own records and file a request. It cannot change pay or approve. |
| Can the model run code? | Only for HR and payroll staff doing ad-hoc analysis, such as testing a policy change: an isolated sandbox, no network, read-only masked data, a time limit, every script logged. Employees never see a figure from model-written code. |
| Are we locked into one model? | No: one internal interface; switching is a config change and a test rerun. |
| How do we know answers are right? | 300 HR-approved questions before every release. Bar: 95% correct, zero wrong pay figures. |
The biggest risks are legal and adoption, not the model. Each has an owner.
Exhibit 12Risk register
| Risk | Likelihood | Impact | Mitigation | Owner |
|---|---|---|---|---|
| Advance rules breach a state's wage law | Medium | High | Counsel signs off all 14 states before phase 3; written consent; no fees | CHRO |
| Consultants don't use it | Medium | High | Live in Teams; HR answers covered questions with a link to it; weekly usage review | HR product owner |
| Wrong pay figure shown | Low | High | Figures only from ADP APIs or tested code; unsupported questions go to payroll; zero-tolerance release bar | Tech lead |
| A past decision is challenged | Medium | High | Full audit trail, replayable for 12 months | CHRO |
| Outdated policy answer | Medium | Medium | One source per policy; answers cite document and date | HR product owner |
| HR sees it as a threat | Medium | Medium | HR owns the approved answers; hiring avoided, no role cuts | CHRO |
| Data leak through crafted messages | Low | High | Per-user scoping, masking, red-team tests each release | IT security |
| Model price or quality changes | Medium | Low | Swappable model behind one interface | Tech lead |
Three releases over 24 weeks, each opened only when the previous gate is met.
Exhibit 13First users in week 10; advances only after counsel sign-off
| Gate | When | Pass condition | Decided by |
|---|---|---|---|
| 1. Pay and policy | Week 14 | 70% answered without HR; zero wrong pay figures in 300 answers; run-rate payback 18 months or less | Steering committee |
| 2. Leave | Week 16 | Leave emails halved among pilot users | CHRO |
| 3. Advances | Week 24 | Counsel sign-off in 14 states; 4 pay runs with no manual rework | CFO and CHRO |
What the week 14 decision rests on. The pilot measures pay and policy directly. Leave is estimated from the prototype and advances from the written rules, so the committee releases the remaining $33,000 only if the measured results hit every target and projected run-rate payback is 18 months or less. Full scorecard in the engineering handoff.
Decision rights
| Activity | Steering committee | HR product owner | Tech lead and 2 developers | IT security | Payroll lead |
|---|---|---|---|---|---|
| Scope and gates | A | R | C | C | C |
| Approved answers, policy rules | I | A, R | C | C | |
| Build and test set | I | C | A, R | C | C |
| Security review | I | R | A | ||
| Advance rules in code | I | C | R | A |
Note: R responsible, A accountable, C consulted, I informed. Steering committee meets every two weeks.
KPIs
| KPI | Baseline | Week 24 target | Measured by |
|---|---|---|---|
| Covered questions answered without HR | 0% | 75% | No HR email or repeat within 72 hours |
| HR hours on covered questions | 377 / month | under 100 | Tickets × handling time |
| Time to answer | 1.5 days | under 5 minutes | Conversation logs |
| Repeat questions | 1 in 5 | 1 in 20 | Same person, same topic, 7 days |
| Wrong pay figures | n/a | 0 | Weekly sample of 50 answers |
| Consultants using it monthly | 0 | 60% | SSO sign-ins |
Three things the firm was considering should not be done now.
Timesheet questions can wait. Client approvals arrive late and only in a nightly export, so any answer would be stale. Fix the feed first, then look again; it's worth about $31,000 a year once the data is current.
Visa letters should never be automated. Each one has to match what was filed with immigration and needs counsel to sign it. A library of pre-filled templates saves more time, with almost none of the risk.
And we advised declining both vendor proposals as they stood. The platform costs more over three years and can't see consultant pay. Workday's assistant deserves another look in a year, sooner if it learns to read ADP.
Approve phase 1 this month and consultants use the assistant before quarter end.
| Action | Owner | By |
|---|---|---|
| Approve phase 1 ($74,000) and the stop rule | CFO | Week 0 |
| Name the HR product owner | CHRO | Week 0 |
| Commission the 14-state counsel review | CHRO | Week 1 |
| Request ADP and Workday API access | IT director | Week 1 |
| Book security review for weeks 8 and 9 | IT security | Week 2 |
| One source for the 12 most-asked policies | HR product owner | Week 4 |
| 300-question test set with approved answers | HR product owner, tech lead | Week 4 |
| Pick 150 pilot consultants across 5+ states | CHRO | Week 6 |
Note: phase 1 covers engineering and pilot support to the week 14 decision, the security review, ADP API setup and policy clean-up.
Method, assumptions and lessons
A. Method
- Volume: 7,100 messages, masked in the firm's cloud, classified by a model; 400 checked by hand (94% agreement).
- Handling time: median of 41 timed requests, checked against HR estimates; the lower figure used.
- Feasibility: 300 masked questions on a day 6 prototype; 92% of payslip, policy and leave answers correct before clean-up; the 24 misses are broken down in Section 1.
B. Assumptions
| Assumption | Value | Source |
|---|---|---|
| Loaded HR cost | $48 / hour | Finance |
| Answered without HR | 75% | Feasibility test, adjusted for adoption |
| Questions | 2,400 / month | Inbox, held flat |
| Engineering effort | 20 to 22 developer-weeks | IT estimate; 22 used |
| Developer cost | $3,600 / week | Finance |
| Hiring budget avoided | about $170,000 / year | 2 HR generalists; timing assumed (months 7 and 10) |
| Sprint fee | $25,000 | All-in payback view only |
| Headcount growth | 15% / year | Firm's plan |
C. From the lead advisor
On a comparable system I led, the hardest question was not which model to use but when a person must decide. HR policies lived in documents and in the HR head's memory, so we turned the deciding rules into settings HR could change themselves, with every change logged. Anything unusual went to a person: a medical expense claim over €2,000, for example, needed further proof, so it went to HR instead of being paid. We also kept a full record of every action, so a year later we could see what the system did and why. With pay and HR, any mistake can become a legal question, and that record is how you answer it.