What happens when a working AI prototype becomes a workflow that other people depend on?
Many teams can build an AI prototype that works. A trigger arrives, the model produces a plausible response, and the demo ends with the expected file or message. But what happens when that prototype becomes a workflow that other people depend on?
Imagine an AI-assisted process that prepares a recurring report. The notification arrives on time, but the report is blank. On another run, the research is useful but lands in the wrong template. A retry then replaces a version that someone had already reviewed. In each case, the model may have responded normally while the application still delivered the wrong operational result.
I encountered versions of these failures while operating Localizeflow, an independent translation orchestration project built around Co-op Translator. A translation could appear to finish without producing a complete, reviewable document. A credential failure could look like a missing result. Overlapping requests could place too much work and state inside one long-running worker.
Those experiences changed the question I ask after an AI prototype succeeds. The next question is not simply, “Can the model do the task?” It is, “Can the workflow record the request, survive failure, validate the final artifact, and return something another person can trust?”
This article uses translation as one worked example, but the lessons apply to any long-running AI workflow that produces a report, document, batch, or repository change. We will follow three failure classes from prototype to production, then add one operating requirement that makes the workflow's progress visible.
A working prototype proves only one thing
The translation prototype was deliberately simple. A trigger started the process, the process downloaded the source, called the model, wrote the translated files, and uploaded the result. It proved that the model could translate a document.
It did not prove that the workflow had used the intended input revision, preserved a partial result, rejected stale output, or left enough evidence to explain a failure. It also did not prove that the resulting document was ready for someone else to use.
That distinction became visible in public repository work. MCP for Beginners pull request #926 shows an incomplete translation batch that stopped with two of 19 translation tasks complete and was closed without merge. The screen records the observable state, but it does not establish the root cause or prove that every generated file was blank.
Figure 1. Public failure evidence from MCP for Beginners pull request #926. The Localizeflow progress record showed two of 19 translation tasks complete, and the pull request was closed without merge. This screen establishes the observable outcome, not the root cause or the contents of every generated file.
By contrast, AI Agents for Beginners pull request #790 connects a completed translation update to a visible diff, review findings, follow-up fixes, and a repository decision. The Localizeflow bot opened the update, and the repository merged it on October 30, 2026. The merge proves that the repository accepted the change. It does not prove translation accuracy, time saved, or a separate deployment. We will return to this accepted result at the end.
The two records illustrate the central distinction in this article:
A model response, a completed process, a delivered artifact, and an accepted result are different outcomes.
Translation exposed that distinction, but it is not unique to translation. A report generator can return text while omitting a required section. A post-processor can damage a valid response. An uploader can publish the wrong revision. A retry can replace work that a person had already approved.
What the workflow had to add
The production-oriented design starts by treating each request as durable work rather than a long-running model call. Figure 2 applies that pattern to an independent translation example. The sections that follow explain the responsibility at each handoff.
Figure 2. Independent reference architecture for hosting Co-op Translator on Azure. A GitHub documentation change enters an orchestrator hosted in Azure Container Apps, which records the request and starts an Azure Container Apps Job. The job runs Co-op Translator and calls Azure OpenAI to produce a candidate translation. Deterministic checks gate the pull request, and a person decides whether to merge it. This is an independent reference design, not a description of an existing project's internal implementation.
Note: Azure Container Apps is suited to service-style workloads such as HTTP APIs, while Container Apps Jobs runs tasks that start, perform a finite unit of work, and stop. In this design, the Container App hosts the request-facing API and orchestrator, while each accepted translation attempt runs as a separate Job.
The container app receives the GitHub change, validates the scope, records the logical job, and decides when an attempt may start. The job runs Co-op Translator, calls the deployed Azure OpenAI model, writes an attempt-scoped artifact, reports its outcome, and exits. Azure OpenAI performs model inference, but it does not own workflow state, validation policy, or merge authority.
First, accept the work. The application checks that the caller, input, scope, and policy are allowed. In this worked example, the accepted request connects a GitHub documentation change to the repository, source revision, file scope, and target language. In a report workflow, it may mean the data source, reporting period, template, and recipient.
Next, record and orchestrate it. The orchestrator hosted in Azure Container Apps saves the logical job before starting execution. The record connects the request to the input revision, policy version, attempt history, and eventual decision. If the process stops, the work still has an identity and a state. In this design, the application layer detects work, records the logical job, and coordinates bounded executions.
Then, execute a bounded attempt. The orchestrator remains the request-facing service boundary in Azure Container Apps while an Azure Container Apps Job runs Co-op Translator for the finite unit of work. The worker receives the recorded job ID, writes to an attempt-specific workspace, and reports its outcome.
Finally, validate and decide. The job's output remains a candidate. Application checks establish whether the artifact is complete, current, and within the allowed scope. Only then does the application prepare a reviewable change. In this worked example, that change is a pull request, and a person decides whether to merge it.
The model call remains important, but it is no longer asked to carry workflow state, validation policy, and delivery authority by itself.
What changes after the prototype works?
Starting with the prototype was the right decision. A first version should answer the smallest useful question: can the model perform the task on one known input? Building job records, isolated workers, recovery logic, validation rules, and monitoring before that question is answered can become overengineering.
The mistake is not starting simple. It is treating the simple version as production-ready. Reliability features should be added when the workflow crosses a real boundary: more inputs, overlapping requests, external dependencies, consequential delivery, or users who need to understand and recover from failure.
The original prototype connected a trigger directly to a model call and treated the returned output as success. That was enough to prove the basic capability. Most state lived inside the process, however, so a stopped run or incomplete artifact left little durable evidence.
The production-oriented workflow keeps the model call but changes the boundaries around it. The trigger records a job, each execution becomes an attempt, generated files remain separate from accepted state, and a retry cannot silently erase the failed attempt.
Start with the simplest workflow that can prove the task. Add complexity only when a real failure boundary justifies it.
Figure 3. The model call remains. Production adds durable state, bounded execution, artifact checks, and human review.
This change matters because real operation adds failure surfaces that a successful demo does not exercise. A trigger can arrive twice. A credential can be invalid. The source can change while work is running. A model can return only part of a document. Application code can damage a valid response. A validator can reject acceptable variation. An external system can complete an action while the workflow loses the response.
While writing this article, I went back through every Co-op Translator release, from v0.2.1 to v0.23.3. Reading them in order felt a little like replaying the project from the first prototype to the workflow it became. I had forgotten how many of the harder problems appeared only after the model was already returning translations. Credentials failed, chunks were reassembled incorrectly, retries needed boundaries, and valid output could still be rejected.
Looking back helped me choose the cases below. I wanted to share the problems that became visible only after the prototype worked, because they are the kinds of problems other teams may meet when a proof of concept becomes an automation people depend on.
Three failures changed the design most.
Lesson 1: A successful model call can still produce a failed artifact
In early runs, a translation could appear to finish while omitting source lines or changing protected Markdown structure. The provider request had completed, but the document was not complete enough to review.
Co-op Translator pull request #568 records a reproducible version of this behavior. Azure OpenAI returned HTTP 200 with finish_reason='stop', but a response could still omit source lines or lose, duplicate, or reorder protected Markdown placeholders. The affected responses were below the provider token limit and did not report a length or content-filter stop.
The recovery logic therefore stopped treating a normal provider completion as proof of a valid artifact. It validated each translated chunk, preserved the verified prefix, and retried only the missing or damaged portion when possible.
I described how that review path evolved, including marker checks, bounded retries, preserved human edits, and recorded failures, in What production taught us about reviewing AI translations.
The model is not the only place where an artifact can break. Issue #505 showed a translated Markdown table that rendered correctly for the first four rows and then collapsed into unformatted text.
Figure 4. Cropped evidence from Co-op Translator issue #505. Application reassembly inserted an extra newline at the chunk boundary, so the final Markdown table broke after row 03 even though translated chunks were available. Pull request #508 documents the fix. This is not a model-quality failure.
The one-line repair was conceptually simple: join the translated chunks without injecting a new newline. The lesson was larger. Valid model outputs can become an invalid document after the response, so validation must inspect the final assembled artifact.
The earlier article Fixing Broken Markdown in AI Translation: Hardening a Production Pipeline examines the wider class of structural failures, including damaged code fences, links, lists, images, anchors, and CJK formatting.
Guardrails can also be wrong. Pull request #581 documents a valid Italian translation that preserved the protected structure but moved ordinary prose naturally. The validator retried the valid result and eventually failed, so the fix narrowed the invariant to the structure that actually needed protection.
These cases led to one operating rule: treat every generated response as a candidate. Keep the logical job, attempt, candidate, and accepted result separate. Pull request #493 applied the same principle by preserving the last usable translation when a replacement failed.
Where policy permits, quarantine a rejected candidate with its validation report. Where the content cannot be retained, preserve safe structural diagnostics such as the file, attempt ID, failed check, and expected and actual counts. Do not copy sensitive source content into routine logs.
The same rule applies to a report workflow. A report with a missing required section is not a completed report, even if the model returned a normal response. A template populated with damaged fields is not safe to deliver, even if the original content was correct.
Try this before production: remove one required section or protected marker from an otherwise normal candidate. The attempt should fail without replacing the previous accepted result, and the failure record should explain which check rejected it.
Lesson 2: A missing result may be a dependency failure
Another failure occurred before translation quality could matter. A credential problem prevented the provider from producing a usable response. In the early workflow, the expected artifact simply did not appear, while the underlying authentication problem surfaced only as a generic process failure.
A blind retry would repeat a request that could not succeed. The workflow needed to distinguish “the model produced a bad candidate” from “a required dependency was unavailable.”
Co-op Translator pull request #211 added a lightweight connectivity check that exercises the real provider data path before translation begins. It checks the endpoint, API version, credential, and deployment. Authentication failures such as 401 or 403 stop the run with actionable feedback instead of appearing after a long batch.
Figure 5. Authentication failure becomes failed with a stable reason and does not publish an empty artifact. After repair, the workflow starts a new attempt under the same logical job.
A startup check is valuable, but it cannot guarantee that a token or credential will remain valid throughout a long execution. A 401 or 403 received during execution should enter the same stable failure category. The job can become failed with a reason such as provider_authentication, while an operator repairs the endpoint, permission, or credential and starts a new attempt without losing the earlier history.
The same pattern applies to retrieval indexes, databases, document stores, and delivery APIs. Report the dependency failure instead of allowing it to masquerade as missing or low-quality AI output.
For Azure resources that support Microsoft Entra authentication, managed identities reduce the secrets that an application must store and rotate. Any remaining credentials belong in the execution environment or a secret store, never in prompts, repository content, or job payloads.
Try this before production: use an invalid endpoint or unavailable credential. The workflow should stop before expensive work, record a stable dependency reason, and publish no empty artifact. After repair, the new attempt should remain connected to the original request.
Lesson 3: Long-running work needs a bounded execution
The first hosting path ran translation work through the same long-lived Container App process that served the application. When requests overlapped, the process had to coordinate more than one workspace, more than one translation state, and more memory-intensive work at the same time. Cleanup and failure isolation became harder to reason about.
This was not evidence that Azure Container Apps inherently leaks memory. Container Apps can scale continuously available services horizontally. The problem was the execution model: a translation batch has a beginning, an end, and an artifact, while the worker behaved like a continuously running service.
Figure 6. Keep the continuously available API in Azure Container Apps and give each finite attempt its own Azure Container Apps Job execution. Retries, cleanup, and admission control still require application policy.
Microsoft describes a container app as a continuously running service and a Container Apps Job as a task that starts, performs a finite unit of work, and stops. That made the split clearer. The API and dispatcher could remain in a container app; Figure 2 groups those continuously available application responsibilities as the orchestrator. Each accepted unit of work could start a separate job execution linked to the logical job and attempt IDs.
The job definition supplies the container image, resource allocation, timeout, and retry policy. Each execution has its own finite process lifetime, execution identity, status, logs, and an attempt-scoped workspace when the application creates one. A finite process lifetime does not remove the need to control memory or clean temporary files, but it limits how far one failed attempt can spread. Data written to persistent storage still needs an explicit retention and cleanup policy. Bound cost and permissions the same way: cap model usage per attempt, and give the dispatcher, worker, validator, and publisher only the access each one needs.
The dispatcher can admit independent requests up to an application-defined limit and deliberately serialize work that might conflict, such as simultaneous updates to the same repository. Container Apps Jobs parallelism controls replicas within one execution. It does not replace request-level admission control.
The application should not rely on external side effects happening exactly once. A retry, restart, or lost response can still occur. Attempt ownership, idempotent state transitions, and reconciliation of an existing pull request or downstream record remain application responsibilities.
Try this before production: submit two independent requests together. Each should retain its own execution ID, workspace, result, logs, and retry path. Then submit two requests that would update the same downstream resource and confirm that the dispatcher applies the intended serialization policy.
Then test the handoffs that fail quietly: submit the same request twice, change the source while an attempt is running, and lose the response after the downstream record is created. The recorded job, input revision, and current attempt owner should lead the workflow to one logical job, no stale output, and no duplicated downstream record.
Lesson 4: Make progress visible before the final artifact appears
A long-running AI workflow should not force a developer to infer its health from whether a file eventually appears. Without searching raw console logs, a person should be able to answer four questions: Was my request accepted? Is the work still advancing? Where did it stop? What result is waiting for review? The workflow should expose the current attempt, how many units are running, complete, queued, or failed, when progress last changed, and whether a reviewable artifact is ready.
Those answers come from a small set of job states. Status should describe evidence, not optimism, so each state names what must be on record before a job can enter it. The following is a reference state model for this design, not a description of Localizeflow's current internal state machine. Here, accepted means the orchestrator has durably recorded the request and accepted it for processing. It does not mean that execution has completed.
| State | What it means | Evidence on record |
|---|---|---|
accepted | The request is stored for processing. | Request ID, input revision, scope, policy |
running | A current attempt owns the job. | Attempt ID, ownership expiry |
validating | An artifact exists and checks are running. | Candidate artifact, checks run so far |
ready_for_review | Validation passed and a reviewable change is available. | Validation result, artifact diff |
failed | The latest attempt ended without a valid artifact. | Failing stage, stable reason, safe diagnostics |
superseded | A newer request or input revision replaced this job. | Link to the replacement |
approved | A person accepted the reviewable change. | Reviewer identity, downstream decision record |
rejected | A person declined the reviewable change. | Reviewer identity, recorded reason |
approved and rejected come from the downstream decision, not from the workflow finishing. For failed, name the stage, such as dependency, generation, assembly, validation, or delivery, so the workflow never retries blindly.
The Localizeflow progress records in the public pull requests make that distinction concrete. The failed record in Figure 1 exposes partial completion and failed jobs instead of presenting an empty destination as the only symptom. The completed record shown at the end exposes a different state using the same vocabulary. A developer can see that the automation is advancing, stopped, or ready for the next decision without treating a model response as the only signal.
Progress is not quality. 19 / 19 completed means that the execution units reported completion; it does not mean every translation is accurate. Operational visibility must connect to artifact validation, a reviewable diff, and the final human decision. The dashboard explains what the workflow is doing. The checks and reviewer decide whether its result is acceptable. Whether the workflow saves time or improves quality needs its own measurements: review time, retries, dependency failures, superseded results, human corrections, and the model, container, and storage resources each run consumes. Collect them before claiming improvement.
Try this before production: start a multi-unit request, then fail one worker or make a dependency unavailable. The progress view should stop advancing, identify the failed or blocked unit, preserve the earlier attempt, and lead an operator to the relevant evidence. After a retry, the new attempt should be visible without erasing the old one.
Build for the moment after the demo
The prototype answered whether a model could perform one task. Production asked a harder question: could the surrounding application explain failure, preserve usable work, reject unsafe output, and return something a person could trust?
Translation revealed three failure classes, but the design lesson is broader. A normal model response can still produce a failed artifact. A missing result can be a dependency problem. A finite unit of work becomes easier to isolate when its process lifetime is finite too.
Most reliability concerns in an AI workflow are familiar automation concerns: state, retries, dependencies, concurrency, and delivery. What AI adds is a probabilistic boundary. Even when the model call succeeds, its response may be incomplete, structurally invalid, or simply unsuitable for the intended use. A reliable design must therefore validate the artifact, preserve rejected candidates safely, and keep consequential acceptance outside the model.
An AI workflow must handle ordinary automation failures and also assume that a successful model call can produce an unacceptable result.
A reliable AI workflow connects the accepted request, input version, execution attempt, candidate artifact, validation result, and human decision. When a run fails or the input changes, that chain should explain what happened and what should happen next.
Figure 7. The public Localizeflow run record linked from AI Agents for Beginners pull request #790 shows the overall state, all 19 work units complete, no active work, no recovery attempts, and no terminal failures. These fields make execution observable. Review findings still led to follow-up changes before the repository merged the pull request on October 30, 2026. Neither completed progress nor the merge alone proves translation accuracy, time saved, or deployment.
That is the purpose of the progress view: make the AI and automation workflow inspectable while it is running, then connect completion to validation and a visible repository decision. A green progress bar is useful evidence, but it becomes trustworthy only as part of that larger chain.
Start with one input, one expected artifact, and one downstream decision: in the translation example, one document, one language, and one test repository. Prove that one valid run stops at ready_for_review and waits for a person, then widen the scope. Track the work, preserve the evidence, validate the result, and return something a person can inspect.