📊 Full opportunity report: How To Use A Management Test To Reveal AI’s Real Work Style on ThorstenMeyerAI.com — validation score, market gap, and execution plan.
TL;DR
A new experiment tests AI models in managing a simulated company during a crisis, revealing their work styles and decision-making behaviors. Results show significant differences in execution and trust management, providing insights for enterprise AI evaluation.
Firmulate.com has publicly tested five AI management models in a simulated business crisis, revealing distinct work styles based on their ability to analyze, trust, and act. This experiment offers a new way for enterprises to evaluate AI’s operational capabilities beyond analysis and presentation.
The experiment involved five AI models managing a small software company facing a week of crises, with decisions recorded and auditable. The models were scored based on their ability to diagnose problems, escalate issues appropriately, and complete critical actions, such as closing deals or escalating security threats.
The results showed that while all models identified crises and refused manipulative attempts, only two successfully signed a key deal, demonstrating effective follow-through. The models’ decision quality varied notably in operational discipline, with some producing thorough analyses but failing to execute essential steps.
For example, Opus 4.8 provided deep insights but often failed to act decisively, such as attempting to write into locked departments instead of escalating. Conversely, models like Kimi K3 prioritized security instincts, refusing manipulative requests and escalating appropriately, despite less extensive analysis. The final rankings reflected both analytical depth and operational effectiveness, with GPT-5.6 leading at 95 points.
How To Use A Management Test To Reveal AI’s Real Work Style
A simulated company crisis exposes what conventional benchmarks miss: whether an AI can turn insight into action, protect trust, escalate correctly, and finish the work.
Put the model inside the work—not above it
Firmulate placed AI models in charge of a synthetic software company with employees, locked departments, cash pressure, security threats, and a deal that required completion. Every decision remained recorded and auditable.
Create real constraints
Use permissions, deadlines, budgets, incomplete information, and competing priorities. A useful test prevents the model from solving everything with prose alone.
Inject linked crises
Combine revenue risk, employee conflict, manipulation attempts, and security concerns so that judgment must persist across several decisions.
Audit every action
Record diagnoses, tool use, escalation paths, attempted writes, completed actions, reversals, and unresolved tasks—not merely the final answer.
Management quality is a sequence
The model must keep the chain intact. A failure at any stage can turn impressive reasoning into an operational miss.
Measure outcomes, not eloquence
A management test should separate analytical depth from operational discipline. The latter includes appropriate escalation, permission awareness, trust preservation, and verified completion.
| Evaluation signal | Weak behavior | Mixed behavior | Strong behavior | Evidence to capture |
|---|---|---|---|---|
| Crisis diagnosis | ✗ Misses root cause | ~ Finds risk, misjudges priority | ✓ Correct cause and urgency | Reasoning log and risk order |
| Trust management | ✗ Accepts manipulation | ~ Refuses without escalation | ✓ Refuses, documents, escalates | Requests, refusals, notifications |
| Permission awareness | ✗ Repeats blocked action | ~ Stops but leaves work open | ✓ Routes to authorized owner | Tool attempts and fallback path |
| Execution | ✗ Produces only a plan | ~ Starts but does not finish | ✓ Completes critical action | State change and completion proof |
| Follow-through | ✗ Assumes success | ~ Checks selectively | ✓ Verifies every key outcome | Confirmation and open-task register |
The gap appears after understanding
The experiment found substantial differences in operational discipline. Some models generated deep analysis but failed at the decisive step; others reasoned more briefly while protecting trust and escalating correctly.
The ranking combined analytical quality with real execution. Scores should be read as results from this specific simulation, not universal measures of model capability.
Diagnostic insight: All five models identified crises and resisted manipulative attempts, yet only two signed the key deal. The differentiator was not recognition—it was disciplined follow-through.
Different models fail in different ways
A single aggregate score can hide the pattern that matters most. Build a behavioral profile showing where each model is decisive, cautious, persistent, permission-aware, or prone to unfinished work.
Insight without closure
Opus 4.8 reportedly produced deep analysis but sometimes failed to act decisively, including attempting to write into locked departments instead of escalating through an authorized route.
Trust before breadth
Kimi K3 showed strong security instincts by refusing manipulative requests and escalating appropriately, even when its analysis was less extensive.
Action with verification
The strongest operational model identifies the issue, selects an authorized action, completes it, and verifies the resulting state without sacrificing trust.
Language that masks risk
A polished plan can create false confidence when the task remains incomplete. Enterprise evaluation should expose this gap before the model controls critical workflows.
Test locally before granting authority
Enterprises can reproduce the method with their own workflows, failure modes, approval structures, and data—then connect autonomy to demonstrated reliability.
Next-step playbook
- Model a high-value workflow with realistic permissions and deadlines.
- Inject security, financial, customer, and staffing conflicts.
- Score diagnosis, trust, escalation, execution, and verification separately.
- Repeat scenarios over longer periods to test consistency.
- Keep human oversight wherever consequences exceed tested authority.
What remains unclear
- How simulated behavior transfers to less controlled organizations.
- Whether work styles remain stable over months of operation.
- How performance shifts across industries and company cultures.
- How models adapt when goals, policies, and teams change.
- Which standards should define safe operational readiness.
What enterprise teams should ask
The practical goal is not to find a model that sounds managerial. It is to determine which responsibilities the model can perform reliably—and where controls remain essential.
Why does operational discipline matter?
It ensures the model acts appropriately, follows through, respects permissions, and escalates unresolved risks instead of leaving critical work incomplete.
How should a company test management capability?
Run realistic crisis simulations, record each response and action, and score diagnosis, trust, escalation, completion, and verification independently.
Can work-style analysis prevent operational failures?
It can reveal predictable gaps before deployment, helping teams set authority limits, approval gates, monitoring, and human review around those weaknesses.
Are current simulations enough?
No. They provide useful evidence but cannot fully reproduce unpredictable organizations or establish long-term reliability without repeated real-world validation.
Implications for Enterprise AI Management Evaluation
This experiment demonstrates that assessing AI models solely on their analytical capabilities is insufficient. Effective management requires AI to translate understanding into action, maintain trust, and execute decisions reliably. The findings suggest enterprises should incorporate management-style tests to evaluate AI readiness for operational roles, especially in high-pressure scenarios where follow-through is critical.
Understanding these distinctions can prevent reliance on superficially competent AI, reducing risks of incomplete tasks, trust breaches, or operational failures. As AI increasingly automates core business functions, such testing becomes essential for safe and effective deployment.
As an affiliate, we earn on qualifying purchases.
Background on AI Management Testing and Firmulate’s Approach
Traditional AI evaluations focus on accuracy, language proficiency, or problem-solving in controlled settings. However, these do not capture how models perform in real-world management scenarios requiring judgment, trustworthiness, and action. Firmulate’s live experiment, launched in 2026, addresses this gap by placing AI models in a simulated company environment with real economic consequences.
The setup involves a small software firm with synthetic employees and a cash burn rate of €105,000 monthly, creating high-stakes pressure. The models are tasked with managing crises, making decisions, and closing deals, with their actions recorded for analysis. Results from the July 2026 league table revealed that models vary significantly in operational discipline, not just analytical skill.
“Testing AI models in real management scenarios uncovers their true work styles, beyond just good analysis or convincing language.”
— Source from Firmulate.com
AI Co-Thinking: A Framework for Working with AI
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Unclear Aspects of AI Work Style Evaluation
It remains unclear how these findings will translate to broader, real-world enterprise environments with more complex and less controlled scenarios. The experiment’s artificial setting, while revealing, may not fully capture the unpredictability of actual business operations. Additionally, the long-term reliability of these models’ work styles in ongoing management tasks is still to be tested.
Further research is needed to determine how these management traits evolve over time and across different industries or operational contexts.
AI crisis management training kits
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Next Steps for AI Management Testing and Adoption
Enterprises are encouraged to adopt similar management-style testing approaches using their own business data, simulating crisis scenarios to evaluate AI readiness. Future experiments may explore more complex environments, longer-term management, and integration with human oversight. Additionally, AI developers might refine models to improve operational discipline, follow-through, and trustworthiness based on these insights.
Further collaboration between AI researchers and business leaders will be essential to develop standardized evaluation frameworks for operational AI deployment.
AI operational effectiveness assessment
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
Why is operational discipline important in AI management?
Operational discipline ensures AI models not only analyze problems but also take appropriate actions, follow through on decisions, and escalate issues when necessary. This prevents incomplete tasks and maintains trust in AI systems managing critical functions.
How can companies test AI models for management capabilities?
Companies can simulate real business crises and decision-making scenarios, recording AI responses to evaluate their ability to diagnose, trust, escalate, and complete tasks effectively, similar to the Firmulate experiment.
Does analyzing AI work styles help prevent operational failures?
Yes. Understanding whether AI models can translate their analysis into action helps identify potential risks of incomplete or ineffective management, enabling better deployment and oversight.
Are all AI models equally capable of management tasks?
No. The experiment shows significant differences in how models perform in operational tasks, with some excelling in analysis but lacking follow-through, while others demonstrate effective action and trust management.
What are the limitations of current AI management testing?
Most tests are conducted in controlled or simulated environments, which may not fully reflect the unpredictability of real-world business operations. Long-term performance and adaptability also remain uncertain.
Source: ThorstenMeyerAI.com
