When industrial AI should not ship
Thesis
An AI model can produce a useful result and still be the wrong production decision.
The real question is whether the capability can operate reliably, economically and responsibly inside an existing product and business system. A successful prototype says little about permissions, invalid inputs, deterministic constraints, downstream execution, long-term cost or who owns the failure when the output is wrong.
For an industrial or B2B software company, production readiness therefore requires more than model quality. Executives must decide which outcomes may remain probabilistic, whether the data is decision-ready, whether the capability can preserve authorization and workflow boundaries, whether failure modes are engineered, whether the economics are sustainable and whether accountable owners exist. If those conditions are absent, AI should not ship yet.
Begin with the cost of being wrong
The first production question is not How accurate is the model? It is What happens when it is wrong?
A recommendation that saves a user several minutes and can be rejected without consequence may tolerate uncertainty. A result that affects physical compatibility, a binding price, a financial posting, a quality approval or a compliance gate has a different error economy. If the cost of one wrong result is many times higher than the cost of expert review, automation should stop before final authority.
Accuracy averages can hide this distinction. A system may perform well across a test set while failing on precisely the rare condition that matters most. Executives should examine the distribution of consequences, not only the distribution of predictions.
That leads to a practical boundary: probabilistic output is suitable where it creates options, prioritizes attention or reduces search effort. It is not automatically suitable where correctness is non-negotiable.
Some decisions should remain deterministic
AI recommendation and deterministic validation are not competing philosophies. In many industrial products, the strongest architecture uses both.
Product configuration
In complex product configuration, AI can interpret an incomplete request, propose variants and help a user navigate a large option space. But mechanical, electrical or thermal compatibility is governed by product rules and physical constraints. AI may generate a possibility; a deterministic system must still decide whether that possibility is valid.
This division creates a useful product boundary. The probabilistic layer improves access to complexity. The deterministic layer protects validity. Asking the model to replace both layers often makes the demonstration simpler and the production product weaker.
Pricing, finance and hard business rules
The same principle applies to pricing. AI may classify a request, extract commercial context or structure an input. It should not invent a legally binding price, discount, currency conversion or margin calculation. Final calculation belongs in deterministic pricing and business logic.
Accounting, tax treatment, posting logic and comparable compliance decisions also require hard validation. AI can assist people in finding and interpreting relevant information, but assistance does not transfer final authority to a probabilistic output.
The executive decision is therefore not whether to use AI or rules. It is where each belongs. Hybrid systems are often more capable than ideological “AI everywhere” architectures because they use uncertainty where it creates value and determinism where it protects the company.
Explainability is not the same as model transparency
Explainability is often framed as a demand to expose model internals. That is rarely what a B2B user needs.
For a low-risk recommendation, trust may come from context: which source information was used, why the suggestion is plausible and how confident the system is. For a quality, approval or compliance gate, contextual plausibility is not enough. The product must expose the actual condition, rule or constraint that was violated.
The distinction matters because the user’s task is not to understand the AI system. The product should reduce the amount of complexity the user must understand. If people have to learn how to prompt, interrogate and second-guess the model before they can act, the interface may have moved work rather than removed it.
A conversational interface can be useful, but conversation alone is not product value. The system must still help the user reach a valid decision, with the right evidence and the right boundary between recommendation and approval.
Bad data cannot be rescued by model selection
Many AI initiatives should stop before teams compare models. The relevant question is whether the organization possesses data from which a dependable decision can be learned or produced.
No trustworthy ground truth
Historical records are not automatically training truth. They may contain old decisions without any reliable indication that those decisions were correct. A larger dataset can then scale inherited inconsistency rather than improve judgment.
Broken entity identity
ERP, CRM and legacy systems often describe the same customer, product or asset differently. If no stable identifiers connect those records, the AI problem is preceded by an entity-resolution problem. A model cannot compensate reliably for an organization that cannot determine whether two records represent the same thing.
Process change without structured history
Products, commercial rules and operating processes change. If those changes are not captured as structured metadata, historical relationships become misleading. The model learns a process that no longer exists and presents obsolete patterns with new confidence.
Data that cannot be used together
Even technically available data may not be legally, contractually or operationally usable for the intended purpose. Access rights, purpose restrictions and organizational ownership can prevent a theoretically attractive dataset from becoming a legitimate production input.
An unstructured spreadsheet with handwritten commentary is therefore not automatically an AI-ready forecasting system. The information first has to be structured, reconciled, validated and made decision-ready. That work may create more value than premature model selection.
The model is often the smaller part of the work
In production, model development is often easier than integrating the capability into the existing business system.
Authorization-aware retrieval
Connecting a retrieval system to ERP, PDM or PLM content can be technically straightforward. The difficult question is whether the requesting user is entitled to see every piece of information contained in the generated answer.
Authorization cannot be checked only when documents enter an index. Source-system permissions may change by user, role, project, customer or lifecycle state. The AI layer must preserve those boundaries dynamically. A technically correct RAG answer is still a product failure if the user was not authorized to see the underlying information.
This is both an integration and governance problem. It cannot be solved by a disclaimer placed beneath the answer.
Legacy systems and downstream continuity
Older systems may lack events, modern APIs or integration semantics suitable for near-real-time workflows. Polling and brittle interfaces can become the real bottleneck even when model latency is excellent.
The same constraint exists downstream. An AI-generated “optimal configuration” has little operating value if it cannot become a valid bill of materials, order, production instruction or other executable business object. AI value ends where the surrounding system cannot consume the result safely.
Executives should therefore evaluate the complete decision path: source data, identity, authorization, model behavior, validation and downstream execution. Optimizing only the model improves one component while leaving the product unfinished.
A demo proves the happy path
A demonstration is designed around favorable inputs and visible success. Production capability is revealed by abnormal conditions: missing data, ambiguous intent, unavailable dependencies, authorization changes, hostile behavior and results that should never be accepted.
Four criteria expose the difference.
Graceful degradation and fallback
What happens when the input is invalid, incomplete or unexpected? What happens when the model or a source system is unavailable? A production design needs a safe path to deterministic rules, expert review or a controlled failure state. Silence, fabricated certainty and an unhandled exception are not acceptable fallbacks.
Deterministic safety nets
Critical output should be checked against hard constraints before it reaches customers or downstream systems. The model can propose; the safety net decides whether the proposal may proceed.
End-to-end observability
The operating team needs visibility into latency, failures, cost and quality signals that matter for the use case. Monitoring the model endpoint alone is insufficient when the failure occurs in identity resolution, retrieval, permissions, validation or downstream integration.
Context and authorization integrity
The capability must remain inside the user’s data, permission and domain context. Production systems also have to reflect how people actually behave—including mistakes, shortcuts and attempts to manipulate the workflow—not only the intended path shown in a demonstration.
Production readiness is therefore visible in failure handling, not presentation quality. The stronger the consequence of an output, the more important the exceptional states become.
Model quality is only part of total cost
A probabilistic system can be technically feasible and economically wrong.
API, token, compute and GPU costs may grow with usage. But inference is only one part of total cost. A production capability also requires monitoring, evaluation, re-labelling where applicable, failure handling, integration maintenance, support and clear operational ownership. Those costs continue after the initial project team has moved on.
The relevant comparison is not AI versus doing nothing. It may be AI versus a simpler deterministic design that solves most of the business problem with less operating complexity. The simpler system is not automatically less ambitious; it may be the better product decision when the additional probabilistic capability does not justify its lifecycle cost.
Executives should connect architecture to unit economics. How does cost change with transaction volume? Which steps require expert review? What happens when source systems or models change? What permanent capability must the organization fund? An attractive prototype can become an unattractive product once these questions are priced honestly.
Maintainability and operating economics can matter as much as model quality. A system the company cannot afford to evaluate, support and evolve should not enter production merely because it performs well today.
The production gate has three dimensions
Before release, I would require evidence across technical, economic and organizational dimensions.
Technical evidence
The capability should pass realistic edge cases, preserve authorization boundaries, degrade safely and perform predictably under representative load. Critical outputs need deterministic validation or an explicit human approval path. The complete workflow—not only the model—must be testable and observable.
Economic evidence
The company should understand process-level value and realistic total cost: infrastructure and API use, monitoring, evaluation, integration, support and continuing ownership. Feasibility without sustainable economics is not production readiness.
Organizational evidence
The operating model needs named owners for the process, product and data; required privacy, security or compliance approval; and an escalation path for failure. These are not administrative tasks to add after deployment. They are production dependencies.
No system can be made completely safe. Technology controls reduce risk, but accountability cannot be delegated to a model. Someone must own the domain, the data, the decision boundary and the consequences.
The strongest architecture is often hybrid
The right conclusion is not that industrial companies should avoid AI. It is that AI should be used where probabilistic reasoning creates real product value.
Let AI interpret ambiguity, rank possibilities and reduce the effort required to navigate complexity. Keep deterministic systems where physical, legal, financial or product correctness is non-negotiable. Keep humans where judgment and accountability cannot be delegated.
This architecture is less ideologically pure than “AI everywhere.” It is also more likely to survive contact with real products, real users and real economics.
Conclusion
AI should not ship because the demonstration is impressive. It should ship when the company can own the outcome, the failure mode, the economics, the data and the operating responsibility.
Until then, delaying production is not a failure of innovation. It is executive judgment: protecting customers and the business while doing the work required to turn probabilistic capability into a dependable product.
About Andrei Lisikov · Related experience in industrial software and applied AI