Bounded Local Agentic Patterns for Clinical RAG

Calling a workflow “agentic” can hide the most important architectural question: who has authority to do what? In a health-record system, an unrestricted model that repeatedly selects tools, changes plans, and executes queries is a poor default. It enlarges the attack surface, makes latency unpredictable, and makes failures difficult to explain.

This synthetic prototype takes a narrower approach. Its agents are fixed dispatch and orchestration paths inside a Rust Rocket server. Local models can classify a request or propose a typed plan, but Rust chooses capabilities, validates every plan, executes data operations, and follows a finite fallback graph. There is no cloud Foundry Agent Service, no cloud model fallback, and no general ReAct-style or multi-agent loop.

The result is still useful agentic behavior: task-aware routing, local planning, structured execution, semantic retrieval, and grounded narration. The difference is that autonomy is treated as a budget rather than a goal.

This article describes both what runs today and the accepted target design specified in plans/new/ - service-line agents, a single structured executor, bounded clarification, typed conversation focus, and pre-bound suggestions. Where a pattern is design rather than code, the text says so.

Rust powers this series implementation, but the same bounded pattern can use the Foundry Local SDK Reference in C#, JavaScript, Python, or Rust; see Part 2 for language links.

Bounding autonomy also keeps the system describable to a regulator. Data-protection regimes generally require an impact assessment before high-risk processing of health data begins, and regulators treat AI-powered clinical systems as high-risk. An impact assessment has to state what data the system reads and where each operation happens. A finite graph can answer that. An open-ended loop that selects its own tools until it decides it is finished cannot, because the set of data operations is not enumerable in advance. Part 1 sets out the residency boundary every one of these paths runs inside.

Prototype scope: all examples describe a synthetic on-premises prototype. They are not claims of clinical suitability, regulatory compliance, or production readiness. The service-line agents, StructuredExecutor, ConversationFocus, clarify branch, and suggestions are the design direction , not shipped behavior.

Problem statement

An unrestricted tool-using loop gives a probabilistic model expanding authority: it can select new capabilities, revise plans repeatedly, and continue executing until an informal stopping condition is met. In a health-record system, that increases data access, latency, resource consumption, and the number of paths that must be explained after a failure.

Local inference does not remove those risks. Accelerator capacity is finite, model output remains untrusted, and an open-ended agent can still issue unsafe queries or produce unpredictable fallback behavior. Prompted tool rules are not equivalent to server-enforced authorization.

Finite means smaller than it sounds. The measured configuration ran generation on an integrated graphics processor inside a 32 GB envelope, sharing a machine with an EMR. The router mitigates this by treating device placement as a per-role, ordered preference - an NPU-first role falls back to CPU rather than failing, and a GPU-resident LRU unloads the oldest idle model before admitting another - but no scheduling policy recovers the capacity an unbounded tool loop spends on its own deliberation.

The agentic pattern must therefore use a fixed, finite graph. Models may classify intent or propose typed data, but Rust must select capabilities, validate plans, execute bounded operations, and decide transitions. Each stage must narrow authority and terminate in a known result or fallback. The set of data operations any path can perform must also be enumerable, because that list is an input to the impact assessment the deployment requires.

Success criteria

• Every agent path is a finite server-controlled graph with named transitions.
• Model outputs deserialize into closed types or fail to a bounded fallback.
• Rust retains sole authority to authorize and execute data operations.
• Tool scope, retries, rows, deadlines, and concurrency are explicitly capped.
• Planning and narration are separated so narration cannot invoke more tools.
• Agent scope is a data-bound allow-list, enforced in code at every hook, never a prompt instruction.
• Clarification, follow-up resolution, and suggestions are bounded and model-free wherever possible.
• Traces identify the route, plan, validation, execution, and terminal outcome.

The architecture: local models inside a server-controlled graph

The Desktop App and Mobile App share a Tauri implementation. Their web layer invokes a Rust bridge, which holds the server URL and JWT. The bridge calls the on-premises Rocket API and relays server-sent events back to the client. Operational data remains in external clinical systems; copied retrieval data and application state live in a separate internal store.

 

Foundry Local is embedded through its device SDK in the server process. It performs local generation for routing, rewriting, planning, narration, optional extraction, and optional verification. BGE-M3 embeddings and the BGE reranker run separately through fastembed. Prompts and record content are not sent to a cloud model.

That topology deserves a production caveat. Microsoft positions Foundry Local for on-device, hardware-constrained scenarios where one user accesses a model at a time; it is not a server inference stack with continuous batching and efficient multi-user GPU sharing. This prototype adds admission control around its central server, but a real multi-user deployment must re-evaluate the inference runtime under measured concurrency. Foundry Local on Azure Local is a distinct enterprise option built around Arc-enabled Kubernetes; it should not be confused with the in-process device SDK used here.

Pattern 1: route with the cheapest sufficient mechanism

Routing is tiered. Each tier earns additional flexibility only when the preceding tier cannot decide safely. The implemented router has three tiers; the target design (router v3) inserts two model-free steps and one bounded escape.

1. Focus resolution (planned). Before any classification, typed conversation state resolves cheap references - “her”, “that ward”, “same period” - and turns bare follow-ups such as “and by gender?” into a mutation of the last structured query. No model is involved (Pattern 8).
2. Tier 0: conversation and metadata rules. Anchored rules catch greetings, thanks, capability questions, off-topic prompts, and questions about the current conversation. These avoid record retrieval entirely.
3. Tier 1.5: deterministic parse (planned). The question is offered to the deterministic QuerySpec grammar from Part 7 before any lexical or model classification. A successful, fully bound parse is itself the routing decision - structured, live-source SQL - and skips the model entirely. A parse with a missing required slot becomes a clarify candidate.
4. Tier 1: lexical intent. Deterministic markers identify aggregation, trend, enumeration, lookup, narrative, and hybrid shapes. Clear structured questions go to exact execution; clear narrative questions go to semantic retrieval.
5. Tier 2: local classification. Only ambiguous or mixed questions reach the local classifier. Its output is mapped into closed route and intent enums. In the target design its prompt lists only the service lines that are usable on a connected source, and its cache key includes the binding version. Invalid or inconsistent output falls open to semantic retrieval.
6. Clarify (planned). When intent is clearly structured but a required slot is unbound and focus cannot fill it, the router emits one templated clarifying question (Pattern 7).

A bounded LRU caches normalized Tier-2 decisions. Tier 0 and Tier 1 are already inexpensive. Catalog replacement clears the cache because a changed schema can change whether a question is structurally answerable.

focus resolution (model-free)
  -> conversation/meta rules
    -> deterministic QuerySpec parse
      -> lexical intent
        -> local classifier only if needed
          -> one clarifying question, or semantic fallback

The target router also decides where a structured question runs from binding coverage, not from string shape: live-source SQL only when a connected source has tables bound to the concepts the question mentions; otherwise aggregation over ingested copies; otherwise semantic retrieval.

“Fail open” here does not mean unrestricted access. It means falling back to the grounded semantic path, which retrieves a small evidence set and can refuse when scores are weak. The router never turns classifier uncertainty into broader tool authority.

One current limitation is worth preserving in documentation: the general /chat path routes with working-memory awareness, while the Auto agent endpoint currently calls the router with has_history=false before loading recent history. Conversation-meta behavior is therefore not identical across those two paths. The target design removes the divergence by having both endpoints call one router and one executor.

Pattern 2: deterministic first, model planner second

Structured questions should not automatically become text-to-SQL prompts. The server first tries a deterministic compiler. Today that is a set of templates for known patterns - exact counts, bounded listings, grouped counts, trends, patient overviews, identifier follow-ups, and selected healthcare joins - keyed on the development seed’s table names. The target design (Part 7) replaces them with a grammar that parses into a typed QuerySpec, binds it to the source’s schema binding by column role, and compiles per dialect, so the deterministic path contains no physical names.

A deterministic match is not trusted merely because it came from code. It passes through the same SQL safety envelope as model output:

• exactly one read-only query;
• linked-table allowlists;
• prohibited locks, DDL, DML, dynamic execution, file operations, and delay functions;
• an outer LIMIT or TOP clamp;
• a second connector-side row cap;
• optional cost preflight;
• read-only connector behavior; and
• database and Tokio timeouts.

If deterministic matching returns no plan, the local planner may propose one. Today it proposes a SQL statement; in the target design it fills the same QuerySpec the grammar produces, so a proposal naming an unbound concept or role is rejected by the binder before any SQL exists. The initial proposal and at most one repair share one deadline. There is no “keep trying until it works” loop.

The source linker also limits scope. It ranks local schema cards, chooses one source, takes bounded seed tables, and adds only outgoing one-hop foreign-key targets. It does not authorize cross-source SQL or recursively expand the schema graph. In the target design the candidate cards are first filtered to the asking service line’s allow-list (Pattern 4), so an explicit mention of an unowned table cannot pull it into a join.

Pattern 3: models propose data, not executable narration

A useful separation is planning versus narration.

Planning outputs are treated as untrusted data:

Constrained decoding can improve output shape, but it is not the security boundary. Some local ONNX variants do not reliably compile the desired grammar, so several planners request JSON content and then deserialize it. SQL is requested as plain text. In both cases, mandatory Rust validation is the control that matters.

After execution, narration runs separately with tools disabled. It receives only validated rows or numbered passages. Deterministic SQL results can use deterministic summaries without a second model call. The narrator cannot decide to issue another query because the capability is absent from that stage.

A practical Rust-like shape is intentionally boring:

let proposal = local_planner(question, linked_schema).await?;
let validated = validate_plan(proposal, &active_catalog)?;
let rows = execute_bounded(validated, &auth_user).await?;
let answer = narrate_without_tools(question, rows).await?;

The important property is not the syntax. It is that each transition narrows authority.

Pattern 4: fixed agents are policies, not personalities

The current build exposes Auto, Health Query, Trends, Patient Lookup, and Summarize. These names describe dispatch policies:

• Auto uses the tiered router and then a fixed path.
• Health Query and Trends try deterministic live SQL, then constrained DocumentDB aggregation.
• Patient Lookup tries a narrow deterministic patient-overview query, then semantic retrieval.
• Summarize tries a narrow recent-record query, then semantic retrieval.
• Chat handles conversation and grounded semantic generation.

Those names are honest about mechanism and useless to a ward clerk. The target design (plan 05, data map) keeps the “policies, not personalities” stance but changes what the policy is keyed on.

Service-line agents

The roster becomes Ask - the router-driven entry point that replaces Auto - plus thirteen service lines a hospital actually staffs: Patient Chart, Front Desk, Ward Board, Emergency, Maternity, Theatre, Pharmacy, Diagnostics, Revenue, Quality & Safety, Workforce, Chronic Care, Facilities. Trends and Handover stop being agents and become modes any service line can run in: Trends forces a time-bucketed shape and chronological narration; Handover forces a cohort-plus-narrative synthesis with SBAR headings and required citations.

An agent’s identity is a scope, and the scope is data, not prose. Each service line owns entity concepts (Admission, Delivery, Bill); a per-source SchemaBinding decides which physical tables supply those concepts. The agent’s allow-list is tables_for(line) plus the shared concepts every agent may read. That allow-list is enforced at four code hooks:

The scope is a read allow-list that narrows the search space and makes “what can you do?” answerable without a model. It is not authorization; RBAC stays in the auth layer.

Personas generated from the binding

Each agent’s system prompt is composed, not authored: a fixed role line, the bound concepts with the hospital’s own table names and observed enum vocabulary (“Deliveries (deliveries): delivery_mode ∈ spontaneous_vaginal, caesarean, vacuum_assisted…”), fixed grounding rules, a mode block, and the current focus. PII columns are never listed. The same Maternity agent installed at a hospital whose table is OB_Delivery speaks that hospital’s vocabulary because the prompt is derived from that hospital’s binding. The persona carries no authority - it cannot widen scope - but it makes the model’s proposals land inside it more often.

A MultiHop model role exists, including configuration that could support tools, but a general iterative executor does not, and none is planned. Likewise, the router can label a request Hybrid, but the full cohort-conditioned path - exact cohort keys applied as a filter to semantic retrieval - is not wired today; it is specified in the executor design below. Hybrid currently falls back to ordinary semantic RAG.

These non-claims matter. A role enum or tool schema is not evidence that an autonomous workflow exists.

Pattern 5: make fallback a closed graph

Fallback should reduce risk or capability, not silently expand it. For structured /chat requests, the intended chain is:

1. deterministic SQL against one linked external source;
2. local SQL planning with validation and one repair;
3. constrained aggregation over active ingested records in DocumentDB; and
4. semantic hybrid retrieval with score-gated generation.

Dedicated agents use narrower subsets of this graph today. Health Query and Trends do not currently invoke the general live-source model planner after deterministic SQL misses; they move to DocumentDB planning. Patient Lookup and Summarize move from narrow deterministic SQL to semantic retrieval. The ladder is also implemented three times, once per call site, with slightly different fallbacks and slightly different events, and the client infers provenance from which events happened to arrive.

One executor, one ladder, one provenance record

The target design (plan 04) collapses those into a single StructuredExecutor called identically by /chat and every /agents/<line>. Its rungs are fixed and ordered:

1 Link scope-filtered cards for the chosen source                                  miss -> 4
2 Det. SQL parse -> bind -> compile                                                      miss -> 3
3 Model SQL plan QuerySpec -> bind -> compile; raw SQL once          miss -> 4
  (2 and 3 both flow through validate -> cost preflight -> execute)
4 Aggregation scoped DocumentDB spec, from the IR when possible    miss -> 5 or 6
5 List scoped DocumentDB listing (enumeration / lookup)                      miss -> 6
6 Retrieval filtered semantic RAG; relax an inferred filter once

“Miss” is defined, not felt: parse miss, bind error, validation error, connector error, timeout, cost rejection, or an empty result on an existence question. An empty list or grouping is a hit with an honest “none” - falling back there would swap an exact answer for a fuzzy one. Each rung has its own deadline from a per-request budget; when the total is exhausted the executor skips to retrieval and records why.

The path actually taken is emitted as a first-class provenance SSE event before the first token - an ordered list such as [Link(Hit), DeterministicSql(Miss("bind error: no EventTime on Bill")), ModelSql(Hit), Validate(Hit), Execute(Hit)], the backend, the service line, the scope, and per-rung timings, all PHI-free - and persisted with the message. The client renders provenance; it no longer guesses it.

Hybrid becomes real inside the same ladder: rungs 1–4 run the cohort as a list-shaped QuerySpec, the row keys (capped) become an explicit retrieval filter for rung 6, and narration receives both the cohort line and the citations.

This distinction prevents a friendly product label from implying broader database powers than the endpoint actually has - and, once the executor is shared, prevents two endpoints from quietly disagreeing about what “fallback” means.

Pattern 6: bind concurrency and streams to runs

Local inference is a scarce resource. The server uses timed semaphores for active generations, retrievals, global ingestions, and per-source ingestions. Saturation produces 429 with a retry hint rather than an unbounded queue.

On the client side, every submission receives a UUID run ID. Tauri events carry {run_id, data}, and application-lifetime listeners route tokens into per-run registries. Normal prompts queue behind active work in the same conversation; different conversations can run concurrently. “Send now” deliberately bypasses the queue, with the tradeoff that unfinished output is not yet part of server-loaded history.

Cancellation aborts the bridge HTTP/SSE future, and logout aborts all bridge runs. This is implemented at the transport and run-registry layer. It has not been proven to interrupt every blocking embedding, reranking, source-query, or already-running model operation end to end.

Pattern 7: clarify once, never twice

An agent that guesses is dangerous; an agent that interrogates is unusable. The target design (plan 02 §5, plan 06 §4) allows a single clarifying question, under strict conditions:

• the router is confident the intent is structured, but the deterministic parse has an unbound required slot - subject, patient, time range, metric, or dimension - and conversation focus cannot fill it (“how many were cancelled?” with no prior subject);
• or the Tier-2 classifier reports low confidence on a very short question with no focus.

The question is templated, not generated: “Which records do you mean - appointments, admissions, or surgeries?” lists the top bound concepts in scope; an ambiguous dimension lists the candidate columns. It is emitted as a clarify event, and the pending slot is recorded in focus so the next short reply (“appointments”, “last quarter”, PT-00042) is merged into the partial spec and re-routed rather than parsed from scratch.

The bound is the point. At most one clarification per turn; never two in a row. If the user’s answer does not fill the slot, the system treats the reply as a fresh question and falls through to semantic retrieval. A clarify is a rung with a fixed exit, not a dialogue loop.

Pattern 8: typed focus instead of model-resolved pronouns

The implemented memory is a rolling summary plus a verbatim tail, consumed by model prompts. The target design (plan 06) adds a second, orthogonal store: ConversationFocus, a small typed record of what is being discussed - patient, provider, place, last concept, service line, time range, the last QuerySpec, a digest of the last result, and any pending clarify slot. It is written by the executor after each turn and costs zero model calls.

Focus is authoritative for reference; the summary is context for style and long-range recall. That split lets follow-ups resolve deterministically before routing:

Every substitution is recorded in the routing decision and shown to the user (“using: Jane Chebet · Aug 2026”). The query-rewrite model role, when it runs at all, receives the already-resolved question, so it never has to guess a pronoun. “New topic” clears focus. A spec mutation is a normal QuerySpec: it binds, compiles, validates, and executes like any other, and the provenance records which message it was derived from.

Pattern 9: suggestions are pre-bound, never generated

After each answer the target design offers up to four follow-up chips. They are not produced by a model. Each is generated from a mutation of the last QuerySpec that has already passed bind on the current binding - “by ward”, “trend per month”, “only caesareans”, “drill into Medical Ward B”, or, after a patient lookup, the chart set of allergies, active prescriptions, last vitals, open bills, and upcoming appointments, filtered to the concepts this hospital actually has.

Because every suggestion is pre-bound, clicking one can never produce a clarify or a miss; it is a question the system already knows it can answer, and the click carries the spec so the router skips parsing. Ranking prefers a different mutation kind from the last turn, stays inside the current agent’s scope, and favours enum dimensions with two to eight values. Model-generated suggestions are explicitly excluded: they cannot be guaranteed answerable, and they cost a generation.

Suggestions are a small illustration of the whole stance. The system proposes only what Rust has already validated, and the model is never the thing deciding what the user can do next.

Anti-patterns to avoid

“Give the model every tool and add a system prompt”

A prompt is not authorization. Select capabilities in server code, scope them to a route, and validate outputs independently.

“Retry until the model produces valid SQL”

Open-ended repair hides instability and consumes scarce local capacity. Use one shared deadline and a fixed retry count, then follow an explicit fallback.

“Let the narrator keep using tools”

Combining execution and explanation permits accidental extra queries and makes provenance ambiguous. Narrate only after execution, with tools disabled.

“Call semantic top-k an exact analytics engine”

Retrieval returns a bounded evidence sample. Counts, rankings, and trends require a structured executor or an honest refusal.

“Name the agent after the query mechanic and hardcode its tables”

Users do not think in aggregations, and a table name is a property of one deployment. Key agents on hospital functions, bind functions to tables per source, and enforce the scope in code at every hook.

“Let the model write the follow-up questions”

A generated suggestion may be unanswerable, out of scope, or subtly leading, and it costs a generation. Offer only pre-bound queries the system can already run.

“A local model makes a central server production-ready”

Locality addresses data movement, not batching, tenancy, throughput, availability, or authorization. The Foundry Local device SDK is officially oriented toward single-user device scenarios.

Tradeoffs

Bounded orchestration gives up some flexibility. Conservative deterministic matchers will miss valid phrasings. One-hop schema linking can omit a required reverse relationship. A fixed fallback can produce a semantic response where a sophisticated autonomous system might discover a better query plan. A single clarifying question will sometimes be one too few, and a model-free focus resolver will not handle every reference a person can make. Service-line scope means a question that genuinely spans two departments is answered by redirect rather than by a cross-scope join.

In return, the system gains finite execution, testable branches, explicit provenance, stable cost ceilings, and clearer authorization points. In a clinical data context, a well-explained miss is often preferable to an inventive success whose path cannot be reconstructed.

Practical review checklist

• ☐ Is every route selected by server code from a closed taxonomy?
• ☐ Do deterministic handlers run before model planners - and before the model classifier?
• ☐ Is model output parsed into a typed structure or a single constrained statement?
• ☐ Are catalogs, fields, tables, operators, rows, cost, and time independently bounded?
• ☐ Is repair finite and under one deadline?
• ☐ Are planning tools unavailable during narration?
• ☐ Does every failure follow a documented fallback or refusal, and is the path taken emitted as provenance?
• ☐ Is agent scope a data-bound allow-list enforced at the linker, planner, retrieval, and identity hooks?
• ☐ Is clarification bounded to one question, never two in a row?
• ☐ Are pronouns and ellipsis resolved from typed focus before any model sees the question?
• ☐ Are suggested follow-ups pre-bound rather than model-generated?
• ☐ Are streams keyed by run ID and protected from token interleaving?
• ☐ Does admission reject saturation instead of accumulating unlimited work?
• ☐ Are incomplete features - such as general MultiHop and cohort-conditioned hybrid retrieval - named as incomplete?
• ☐ Has the inference runtime been load-tested for the actual central multi-user hardware profile?

Prototype status

Implemented today: tiered routing, Tier-2 cache, task-aware role specifications, deterministic SQL templates, local SQL planning, one repair, SQL and DocumentDB validation, fixed agent dispatch (Auto, Health Query, Trends, Patient Lookup, Summarize, Chat), grounded narration, run-scoped streaming, persistence, and admission control.

Planned and specified in plans/new/, not yet in code: service-line agents with Ask/Trends/Handover modes and binding-generated personas; router v3 with focus resolution, deterministic-first parse, backend from binding coverage, and the bounded clarify branch; the single StructuredExecutor with a provenance event and a real hybrid cohort path; ConversationFocus with model-free anaphora and spec mutation; and pre-bound suggestions.

Partial or open: Auto conversation-meta parity, agent use of persisted rolling summaries, full hybrid cohort execution, complete cancellation propagation, and production multi-user capacity evidence. MultiHop is a role definition only; no general ReAct or multi-agent loop is implemented or planned.

Key takeaways

1. Agentic behavior does not require an unconstrained loop.
2. Use deterministic logic before local model judgment - for parsing, routing, reference resolution, and suggestions alike.
3. Treat plans as untrusted data and keep execution in Rust.
4. Separate planning from narration and remove tools from the narration stage.
5. Make fallback finite, visible, and progressively safer, and emit the path taken.
6. Key agents on hospital functions and bind them to tables per source; enforce scope in code, not in the prompt.
7. Bound clarification to one question and make every suggestion something the system can already answer.
8. Local inference preserves data locality, but central multi-user operation still requires a deliberate runtime and capacity decision.

Source deep dives

• Agentic patterns
• Routing, agents, and structured query
• Hospital agents and their data map
• Deterministic query handling reference
• Models, accelerators, and lifecycle
• Retrieval, chat memory, and concurrency
• Implementation plans: router v3, structured execution and fallbacks, hospital agents, conversation memory and suggestions
• Code: onprem-rag-server/src/router/mod.rs, onprem-rag-server/src/agents/routes.rs, onprem-rag-server/src/nl2sql/, onprem-rag-server/src/aggregation/, and onprem-rag-server/src/admission.rs

Learn more on Microsoft Learn

• What is Foundry Local?
• Get started with Foundry Local
• Foundry Local SDK Reference - Rust on Windows
• Design and develop a RAG solution
• Foundry Local on Azure Local overview - the enterprise Arc/Kubernetes option, distinct from this prototype’s in-process device SDK.

 

Continue the series

Previous: Part 7 - Deterministic SQL Before an LLM

Next: Part 9 - Hardening On-Premises RAG

 

The code behind this series is open source: github.com/kevin-gatimu/onprem-health-rag. It is a synthetic-data research prototype, not a clinical or compliance-ready system - issues and corrections are welcome.