Your company’s AI needs a scoreboard

10.05.2026

Author - Enrique Dans

I’m talking to many companies these days that are trying to incorporate AI into their business practices. Most of them are still in what I call “the administrative phase,” when they just try modest automations that allow some workers to draft documents or presentations, answer emails, or do research on certain topics, but given my recent articles arguing for companies to go much farther, I’m starting to get more and more questions from enterprises looking to go way beyond that, and get AI to become more “strategic.”

For these companies, the main concern is expressed as a question: Will AI be smart enough? But I think a better and more useful question would be: How does the AI know whether what it did actually improved the business?

The examples are well known and not theoretical: from optimizing a salesperson for revenue, margin, or retention, to a factory in which we can improve throughput, but we could also improve quality, or reduce defects. In business, “optimization” rarely means just one thing: improving one metric can easily damage another. So, in the same way management already works through objectives and feedback, AI agents need to do exactly that. In fact, OpenAI now explicitly frames enterprise evaluation as “specify → measure → improve, and in doing so, it can turn what would be vague business goals into actionable and measurable expectations.

Current AI metrics often measure variables such as answer quality, task completion, latency, cost, etc. However, what businesses actually care about are things like customer churn, margin, conversion, claims accuracy, delivery times, or customer lifetime value. 

An agent could be completing its task perfectly as instructed, and still hurt the company by doing so. Businesses do not ultimately care about generated outputs: they care about outcomes. It is worth repeating, because it changes the entire conversation: businesses do not ultimately care about outputs; they care about outcomes. What we demand in corporate implementations of AI is completely different from what we need in our individual use. 

The managerial point is simple: task completion is not the same as business success. 

Think about AlphaZero, the brilliant algorithm designed by DeepMind become strong because it improved, it saw the consequences of its actions and it had an unambiguous objective: to win. 

DeepMind says it learned through repeated trial and error, to favor moves that could increase its chances of winning. Companies, obviously, have goals that are much messier than chess, but the principle survives: learning requires feedback tied to an objective. 

The hard part is choosing the scoreboard: “increase sales” is clearly not enough. What about margin, or churn, or returns, or compliance, or brand image? When establishing goals, multiple objectives can (and do) conflict. And those conflicts are not engineering problems alone: they are management problems. Therefore, boards and executives eventually have to decide what should improve, what must never be sacrificed under any circumstance, or what trade-offs could be allowed. Once systems can continually optimize, the objective itself becomes a governance decision.

Previous
Previous

When AI runs the company, what exactly is leadership for?

Next
Next

Enterprise AI impact isn’t about the most powerful model, but about the smartest steering.