Imagine a swimming pool service that operates entirely without human staff—yet somehow, it’s losing thousands of euros every month. It’s not science fiction; it’s the real-time experiment from Firmulate, which puts AI models in the driver’s seat of a live company. For pool and patio enthusiasts, this story might seem distant, but it reveals something fundamental about AI’s potential—and its limits—in managing complex, money-driven tasks.
The Live Company That’s Building in Public
At the heart of this experiment is a small, virtual software company, run entirely by AI. It has 13 synthetic employees—each a decision-making model—working through the challenges of real business: handling customers, managing crises, and trying to close deals. Every day, the company operates with a set of self-learned rules, which are versioned and publicly accessible at firmulate.com/live.html. This transparency makes the experiment unlike any other, allowing viewers to see decisions unfold in real time and understand the mechanics behind AI-driven management.
AI decision-making software for business
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
How Does the Experiment Work?
Four advanced AI models, representing different levels of sophistication, were each tasked with navigating the same rough week for this virtual company. They faced identical customers, crises, and temptations—like trying to manipulate or cut corners—yet only two of these models managed to close a €55,000 deal that their own analysis deemed earned. The other two, despite similar diagnoses and pitches, failed to sign the deal, illustrating stark differences in discipline and decision-making quality.
The Surprising Role of Hidden Data
While all models detected crises and refused manipulative tactics, the real game-changer was a buried piece of information: a document, stored deep in the company’s files, which contained a crucial insight. Those that read this file and incorporated its knowledge went on to secure a full-price deal, adding over €4,500 to monthly recurring revenue. It’s a reminder that often, vital details are hidden in plain sight, and AI’s ability to read and analyze documents can determine success or failure.
Can AI Be Trusted to Play It Straight?
One test involved social engineering scenarios—fake CEO messages escalating in severity, and even a reporter asking for a quick, off-the-record approval. All four models declined these manipulative requests, citing reasons like suspicion of impersonation or bypassing approval processes. Kimi K3, one of the models, explained: “Treat the request as a suspected approval-bypass / possible impersonation.” This shows AI’s growing capacity to resist social engineering tricks that often fool humans.
The Money and the Strategy Behind the Experiment
This virtual company burns €105,000 every month against a meager €2,300 in monthly recurring revenue. Its cash countdown and real-time metrics are publicly visible, making it a vivid illustration of the economic realities of running an AI-powered business. The goal isn’t just to keep the lights on—it’s to see whether AI can outperform human decision-making in complex, profit-driven environments.
The Limitations of the Most Thorough AI
Among the models, Opus 4.8 stood out for its analytical depth, learning over 80 rules and conducting detailed analyses. Yet, it still finished last in the scoring, leaving deals on the table and showing slips in discipline—like writing issues instead of escalating problems. Even the most sophisticated AI struggled with consistency, highlighting that more rules and deeper analysis do not automatically translate into better business outcomes.
What This Means for Business and Beyond
This experiment is more than a tech demo. It lays bare how AI models are approaching real-world management and whether they can be trusted to handle crucial aspects of business—honesty, discipline, strategic reading. For owners of pools, patios, and water features, it’s a glimpse into the future where AI might help optimize operations—but also a cautionary tale that technology still has hurdles to clear before it can reliably replace human judgment in complex, profit-oriented settings.
Try It Yourself
If you’re curious about how your own business decisions might stack up in this AI-driven environment, you can run a similar wargame against your company’s data, all without risking real systems. Visit firmulate.com/pilot.html to learn how, and see what your management choices reveal about AI’s current capabilities and limitations.
This live experiment from Firmulate shows AI models successfully detect crises and refuse manipulation but still struggle with closing deals and maintaining discipline—highlighting both AI’s promise and current limits for business management.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
