“Knowing is not enough; we must apply. Willing is not enough; we must do.” — Goethe
We recently started an AI experiment at Justworks International with an attempt to reimagine how our Employer of Record (EOR) systems should work in an AI-native world. We weren’t trying to add another automation layer. The goal was to see how far we could push toward a model where AI handles most of the mundane work and humans handle the strategic, high-leverage work.
As engineers, we love to build. But this project taught us that sometimes the hardest part isn’t building the tool — it’s realizing you’re building the one you know, and new opportunities have been created that are worth exploring.
The Chatbot Phase
Naturally, we started with the industry default: a chatbot. We dumped our SOPs (Standard Operating Procedures), policy docs, and compliance data into a vector database and built a retrieval system.
The results were technically excellent. In our initial internal tests, we enabled the chatbot to reliably provide answers to the questions based on our knowledge base. We had built a highly accurate, knowledgeable librarian.
But as we ran our evaluations and demoed the tool to our stakeholders, the perspective shifted. We realized that “knowing” was only a fraction of the value. If we shipped this, our operators would get the correct answer, but then immediately have to switch context to manually perform the task. It worked, but it didn’t add much business value. In a high-velocity Ops environment, knowing how to do a task is only part of the battle.
The Obvious Question: Why Not Just Script This?
We pivoted immediately. We put the chatbot aside and shifted toward building an agent that could understand user intent and actually perform the task, with a human in the loop for guidance and approval.
We selected a simple test case: updating an employee’s vacation policy. But as we started designing the solution, we hesitated. It felt wrong. We looked at the architecture and thought: Wait a minute. Why are we using a probabilistic, expensive LLM to call an API?
If we want to update a vacation policy, we can just write a script or build a dedicated UI form. It’s cheaper, faster, and 100% deterministic. At one point, it felt like we were over-engineering the whole thing. We almost convinced ourselves to scrap the AI and just build better internal tools.
The Epiphany: The Long Tail of Ops
But then we looked at the reality of our business.
Justworks EOR (Employer of Record) is currently in 30+ countries, expanding every quarter. Each country has unique labor laws, unique data fields, and unique exceptions. The onboarding documents required for Canada are not the same as Spain. Offboarding is a whole different set of complexities.
For a small engineering team, building custom UI forms and “deterministic scripts” for every single edge case in 30+ countries is itself a time-intensive battle, and we were intentionally exploring ways to accelerate country expansion.
This was our epiphany: The value of the AI agent is not found in executing complex logic, but rather in its ability to scale alongside the variance in EOR operations.
Our biggest realization was that SOPs are essentially prompts — detailed instructions that humans follow step-by-step, which can also be fed to the agent. Instead of hard-coding the logic for “Update the Marital Status for this employee in Mexico” we could point the agent to the Mexico SOP. The agent reads the manual to understand the rules, plans the workflow, and executes it.
The agent became a universal, semantic interface. We weren’t replacing Ops; we were giving them a dynamic gateway to the backend, bypassing the bottleneck of custom UI development while maintaining strict access controls and data validations.
We aren’t alone in this thinking. The team at Brex recently published similar findings(opens in a new tab) about using AI to address the “long tail” of internal software. Their conclusion mirrored ours: traditional engineering solves the “head” of the distribution (high volume, common tasks), but agents are a scalable way to solve the “tail” (endless custom workflows).
How We Actually Built This
Once we committed to this “Universal Interface” model, we moved fast to build an internal Proof-of-Concept. We avoided black-box systems in favor of something our engineers could debug and our Ops team could trust.
The Stack:
Mastra (TypeScript): We chose the open-source AI framework Mastra because it came bundled with all the tools and abstractions required for building AI agents like observability, evals, memory, tool calling, etc.
MCP (Model Context Protocol): We used MCP to give the agent structured access to our internal APIs. This was critical. We aren’t letting the AI “guess” the API calls; we are giving it a defined toolset.
Context-Aware UI: We didn’t want a generic chatbot window. We built a dedicated UI inspired by modern AI-assisted tools like Cursor. Ops users can search for employees or companies and “click-to-pin” them into the context. This ensures the Agent knows exactly who it is operating on, reducing hallucinations and eliminating the need for users to type out long IDs.
The Workflow in Action: We proved this out with a complex real-world scenario: Creating a custom PTO (Paid Time-off) policy.
Fetch & Identify: The agent pulls employee details from the pinned context and identifies the country and region (e.g., Canada — British Columbia).
SOP-to-Workflow: The agent pulls the relevant policy for that region and figures out the required rules from it. We didn’t hard-code “Canadian PTO logic”; the AI learned it from the doc. It’s scalable to multiple countries and regions instantly.
Cross-Platform Action: It fetches the employment agreement, reads the contract to find any additional PTO clauses, and proposes the new policy creation.
The Human “Kill Switch”: Before writing anything to the database or sending an email, the agent presents a summary. The Ops user reviews it. One click to approve.
It was quite impressive to see it perform this accurately without any explicit coding, but just simple instructions written in English.
We even added “ripple effect” awareness. For example, if a data update impacts benefits, the agent notices and suggests: “This update affects benefits. Should I create a Jira ticket for the Benefits team?”
Where This Leaves Us
We demoed this proof-of-concept, which we’ve recently dubbed internally “Ops Co-pilot”, and the reaction was positive. The demo sparked genuine excitement as the team saw a tangible solution to their operational bottlenecks and also the future potential of it to strengthen compliance.
By treating the agent as a dynamic interface between Ops and our Backend, we could effectively remove the “Product Bottleneck” for hundreds of edge-case tasks. We are still in the early days of moving this forward, but our mindset has shifted. We don’t have to chase feature requests anymore. Now, we can build more general capabilities that will let Ops scale at the speed of the business.
Do you want to build products that help entrepreneurs and small businesses grow with confidence? We’re hiring across our Technology teams. Come build with us! Check out our Careers page.
