Recently, one of our Associate Product Managers (Willy(opens in a new tab)!) spent two long weekends prompting AI agents and produced what looked like 70% of a native Applicant Tracking System (ATS). Work that normally takes months appeared in days. It was the kind of moment that makes you want to throw away everything you thought you knew about building software.
While we knew this was a moment we could seize to ship something new and useful to our customers, we also wanted to use it to help us quickly find and spotlight where our current way of building and changing our product needed to be rethought from the ground up.
So we ran an experiment where we brought together a small cross-functional team — product, design, research, engineering — and asked them two questions:
How far off this is from production-ready, in terms of what it does and how the code works?
What are the bottlenecks to building new products, if the tools the entire team reached for first were agents and models?
Here’s what we learned, and the path we took to launch Justworks ATS.
AI accelerates getting to a first answer
AI was excellent at producing what we started calling the median answer — comprehensive, technically reasonable, generically correct. But after a few quick rounds of testing the prototype, the team quickly uncovered that our Ideal Customer Profile (ICP) had distinct needs and priorities. But even when we passed the prototype through an AI tool we built (Justworks Pulse — more on that later, and where it’s been game changing) that parses the millions of interactions we have with our customers and all of our ICP research, it didn’t give us nearly the same amount of texture we got during a handful of conversations with real customers.
This is where I think the conversation around product judgement starts to take on a few different meanings. When people say AI cannot replace judgement, they’re referring to which ideas to solve, or how to solve them, such that what they ship isn’t a median product. In the case of ATS, the product judgement of the lead GPM really kicked in when she had to decide the difference between shipping a median product and an MVP, the difference being that one might be more full-featured but less well suited to our customers, and one that might be scoped down but could be quickly iterated on.
The judgement here was knowing that the risk of shipping something brand new and full featured but directionally off is customers won’t internalize it as, “oh, I can see how they’ll make this more robust,” and therefore a better fit for me over time. They think, “oh, this isn’t really for me, it’s for someone with different needs.” This is especially acute when building for small businesses, who have historically only had point solutions that were too expensive, too overbuilt, and not made for them. Shipping that median product creates a real risk not only for learning (people are more likely to give feedback on what they want added, not subtracted or changed), but also a long term risk to adoption of new things we offer.
Give everyone the same toolset
It was easier for the Engineers on the project to reach for agents and models as their toolset than it was for Product & Design, less because of technical skill but because of the way most organizations provision laptops, and make tools available to various functions. The result was that individual productivity gains didn’t compound into team-level speed — you can’t unlock the real acceleration of AI-native building when half the team is working around a tooling gap.
The speed gains were real, and uneven
Engineering was the clearest win. AI coding tools helped a small team — one senior engineer and an associate PM, Willy — maintain architectural patterns across multiple codebases, generate documentation, and compress work that would normally take a week into hours. Emotional attachment to AI-written code was lower, which made it easier to respond to new information.
Design saw similar gains. Rapid prototyping with our frontend and design system let the team iterate quickly enough that feature changes based on user feedback happened in minutes, not days.
While the work itself got faster, that didn’t necessarily mean the product was magically in the hands of customers faster — the acceleration of individual productivity was great, but the best part was it showed us where the systems around the teams building needed to be reworked to allow for that work to show up as value to customers immediately.
The question shifted from cost-to-ship to cost-to-carry
When building was the bottleneck, the main question was: how much will this cost to build? AI has largely retired that question. A small team can now ship something that looks and functions like a real product in days.
The question that replaced it is harder: what are we willing to maintain, and what are we willing to let occupy our org’s mindshare? Maintenance is not just engineering hours;it is security surface, regulatory exposure, customer support, roadmap attention, and the institutional obligations that come with being a system people rely on. Every product we ship makes a claim on all of that.
So when the team was deciding on if and how to take ATS beyond alpha, their conversations went beyond what’s the effort to build vs. the upside. Great products have everything you need and nothing you don’t. We’re seeing the best Tech teams build the products that compound. Where the maintenance burden reflects genuine, lasting value for customers, not just a feature we could ship because the cost to build got cheap.
Where we are now
Addressing the toolset gap became a product priority in its own right. Justworks Current, our homegrown tool that provisions every Justworker’s laptop with the AI tools, workflows, and skills tailored to their actual role and the stack they use, came directly out of this realization. Rather than leaving AI adoption to individual initiative, Current gives every function a purpose-built starting point. It has become one of the highest-leverage investments we have made in company-wide AI fluency, largely because broad AI adoption is more about making it the path of least resistance.
After the initial vibe-coded ATS, we had an org-wide Hackathon where a small group of people took what they learned from customers’ reaction to the first prototype and built a true MVP in under three days.
They had conviction on scope and approach because of that initial prototype and what we learned from customers, which was just as critical in accelerating time to learning as the tools they were using. And critically, they all had the latest tools and fluency in using them so they could work together across functions inside the same working product from day one, vs handing off between tools, but building in the same place.
Within a matter of weeks, that MVP became a beta in the hands of our customers, with a line of hundreds of customers ready to get hiring.
This effort surfaced more questions than answers, which is how we know it was the right thing to do. We haven’t cracked where synchronous collaboration lives in an AI-native workflow; it’s one of the more interesting org design questions we’re sitting with. The one we find even more compelling: what does GTM look like when your ship cycle is days, not quarters? Most go-to-market motions were designed for a world where building was the bottleneck. When it’s not, getting from live in prod to customers fully enabled and realizing value has to move at that same pace.
Do you want to build products that help entrepreneurs and small businesses grow with confidence? We’re hiring across our Technology teams. Come build with us! Check out our Careers page.
