Probabilistic Reasoning Deterministic Output

I built Kofax Clear to remove repetitive browser work from an application-support queue. I recently extended it with a narrow AI-assisted bug-correlation step: it can take the context of a failed event, search existing Azure DevOps work items, and return the most likely matching bug for me to review.

The useful part is not that an LLM is involved. It is the boundary around it: probabilistic reasoning inside, deterministic interface outside.

The repetitive problem

While working in enterprise application support, I deal with failed events that sometimes persist after automatic retries. When that happens, a support engineer has to inspect the failure and determine whether the underlying problem is already being tracked.

The manual workflow is familiar: open an internal exception portal, inspect the failed event, copy the useful error context, open Azure DevOps, try several searches, compare candidate bugs, and bring the matching work-item number back to the support workflow. None of those steps is individually difficult, but the context switching adds up.

Kofax Clear already automated the browser-heavy portion of that work. The new feature focuses on the fuzzy part: deciding which existing bug, if any, best corresponds to the failure.

Automating the browser work

The original automation is Python and Selenium. It navigates the internal web application, inspects queued items, extracts the useful context, and performs repetitive queue-clearing interactions.

That part benefits from normal deterministic software. A selector points to a known element. A parser extracts a known field. The automation can validate that a page reached the expected state before moving on.

Selenium can retrieve an error and an entity identifier, but it cannot decide that one loosely worded bug description refers to the same underlying problem. Hard-coded string matching is brittle when descriptions use different wording or include different slices of context.

The part Selenium could not solve

Bug correlation is a reasoning problem. The exact error may not appear in the bug title. An identifier might be present in a comment instead of the description. Several search results can share the same general symptoms while only one belongs to the relevant application domain.

I did not want to bury a growing collection of search heuristics inside the scraper. I wanted to hand a bounded task to a component that could interpret context:

Here is a failed event and its identifying information. Search the existing work items and determine whether one is a likely match.

That is where Claude Code fits into the pipeline.

Adding Claude Code as a component

The Python automation invokes the Claude Code CLI locally in a headless workflow. It is currently configured with Claude Sonnet 5. Claude is not the application and it is not chatting with me; it is a reasoning and orchestration layer called by the larger program.

The input is deliberately limited to the sanitized failure context needed for the search. The output returns to the Python process, which validates and consumes it like any other dependency.

This arrangement let me add contextual reasoning without moving the rest of the automation into an agent framework. Selenium still does browser automation. Python still owns control flow, parsing, and validation. Claude handles the one step where flexible interpretation is useful.

Giving the model tools through MCP

Claude Code connects to Azure DevOps through a Model Context Protocol server. MCP exposes the relevant search tools through a structured interface, so Claude can query existing work items and inspect likely candidates.

The tool access is narrow by design. This workflow retrieves and correlates information; it does not autonomously edit Azure DevOps, update a bug, or create a new one. Restricting the available operations makes the agent’s job clearer and limits the consequences of a bad judgment.

After comparing the failure context with candidate work items, Claude returns the likely matching identifier. If the search is ambiguous or does not find a good match, that result can be represented too.

Making an LLM safe for a software pipeline

Normal model output is a terrible interface for another program. The same answer might arrive as a sentence, a Markdown list, or a paragraph with extra explanation. All of those are readable by a person and brittle for a parser.

I use a YAML-defined response contract to constrain the result. Claude must return JSON with the expected shape: whether a likely match was found, the work-item identifier when appropriate, and only the limited metadata the calling automation needs. The exact internal fields are intentionally not part of this public writeup.

The program then validates the response before using it. If the output does not satisfy the contract, it is an integration failure—not an invitation to guess what the model meant.

This is the key architectural idea:

Let the model reason probabilistically, but require it to cross a deterministic boundary.

The contract makes the LLM feel more like a normal software dependency. Its internal path can vary, while the surrounding program sees a stable machine-readable interface.

Keeping a human in the loop

Kofax Clear returns a likely bug number to me. It does not make the final operational decision.

I review the result, decide whether the match makes sense, and choose the next action. If no existing bug is appropriate, creating one remains a human step. Ambiguous and unusual cases also stay with the engineer instead of being forced through the happy path.

That is not a temporary limitation I am racing to remove. The lookup is repetitive and well scoped; modifying the issue tracker carries more consequence and benefits from review. Automating the first part delivers value without granting the model unnecessary permissions.

What I learned

The strongest automation did not come from replacing the whole workflow. It came from giving each part the job it handles best:

• Python and Selenium perform predictable browser work and data extraction.
• Claude Code interprets context and orchestrates a bounded search through MCP.
• A schema-constrained JSON response connects probabilistic reasoning to deterministic software.
• A support engineer reviews the result and controls any write action.

MCP also changed the cost of this kind of experiment. Instead of building custom one-off issue-tracker orchestration into the scraper, I could expose an existing tool through a standard interface and focus on the correlation task.

Most importantly, useful enterprise AI does not have to be an autonomous employee. A narrow reasoning component, placed inside a conventional program and surrounded by strict interfaces, can remove a frustrating step without taking over the entire process.

Tech used

Python, Selenium, browser automation, Claude Code CLI, Claude Sonnet 5, Model Context Protocol, Azure DevOps, YAML contracts, JSON, schema validation, and human-in-the-loop workflow design.