Idea #1
Multi-source evidence reconciliation is the practical path to autonomous confidence in ambiguous classification tasks. A single LLM extraction gives you a probabilistic guess with no way to verify it without a human check — but if you can find a second independent data source that either confirms or contradicts that guess, you get a convergence signal that is far more trustworthy. The mechanism is simple: only act autonomously on cases where both sources agree; route everything else to human review. This lets you grow the autonomous share of work without accepting the tail risk of a single-model error going undetected. In the oncology prior-authorization workflow described, the LLM alone classified drugs as 'no auth required' or 'auth on file', but the output was too unreliable to act on without review. Adding authorization letters (historical proof that a drug was already approved) as a second source for the 'auth on file' case, and building a payer-rule SQL knowledge base (built from portal checks and LLM extractions) as a second source for the 'no auth required' case, allowed the system to confidently route those cases without human touch wherever both sources agreed. The confidence was structural, not probabilistic.
[from: Can Oncology Workflows Run Without Human Touch? – Anant Shankhdhar, Risa Labs — YouTube · AI Engineer · https://youtu.be/_cVfz88_j7A]
Idea #2
Self-healing RPA loops, not perfect upfront integration, are the scalable answer to automating access across dozens of heterogeneous portals. Building bespoke custom integrations for each payer portal is not scalable — the portals change, break, and differ in structure. The alternative is to build a large library of reusable portal actions, then use an LLM to generate the configuration that composes those actions into a working automation for any given portal. This collapses integration development time dramatically. The remaining fragility problem — automations that worked at build time break in production when a portal updates its DOM or flow — is handled by a self-healing loop that detects failures at runtime and mitigates them automatically, preventing silent failures from blocking live orders. In the Risa system, the Coverage Orchestrator decides per-payer whether to call a structured API or execute an RPA path; the RPA path uses the LLM-generated config over the shared action repository, and the self-healing loop monitors production runs and patches breakages during operating hours. The key insight is that fragility is inevitable and should be designed for explicitly, not treated as a deployment failure.
[from: Can Oncology Workflows Run Without Human Touch? – Anant Shankhdhar, Risa Labs — YouTube · AI Engineer · https://youtu.be/_cVfz88_j7A]
Idea #3
Layer your automation: run the cheapest, most deterministic checks first and reserve agents for only the residual cases that rules genuinely cannot resolve. This is not just an efficiency pattern — it is a correctness strategy. Deterministic checks have no hallucination risk and impose zero cognitive load on downstream agents; if you can rule something in or out mechanically, do it before any model sees the case. The agent's job shrinks to the genuinely ambiguous residue, which is also the set where its reasoning capability actually adds value. At Risa Labs, the pipeline starts with a deterministic eligibility decision engine (is coverage active? yes/no) that kills ineligible orders before any LLM runs. Then a rule-based drug categorization pass eliminates orders whose drugs are all either already authorized or exempt. The medical necessity agent — the most expensive and latency-sensitive component — only runs on orders that survive both earlier layers. This staged architecture is also what makes no-touch share a metric that can be tracked and grown incrementally: each layer that expands its deterministic reach directly grows the autonomous share without touching agent code.
[from: Can Oncology Workflows Run Without Human Touch? – Anant Shankhdhar, Risa Labs — YouTube · AI Engineer · https://youtu.be/_cVfz88_j7A]
Idea #4
For clinical or regulatory reasoning tasks, the right model output is not just an answer but an answer plus its supporting and contradicting evidence, with a confidence score that gates autonomous action. The confidence score is only meaningful if the agent has genuinely queried all the data sources that bear on the question — so the architecture must include structured patient data retrieval as a first-class step, not just raw text. At Risa Labs, the medical necessity agent processes each prior-auth case by reading three things in parallel: the patient's clinical notes, the payer's policy criteria document, and a patient medical graph (a structured biomarker graph extracted from records). The agent queries the graph using the policy criteria to find which biomarkers are present and what the patient's clinical condition actually is, then passes all of this to an LLM to produce an answered questionnaire with both supporting and contradictory facts. If confidence is high enough, the case proceeds autonomously; otherwise it is escalated to a human clinician. The practical lesson here is that routing to humans should be a designed, explicit outcome — not a fallback — and the confidence threshold is the dial that balances throughput against risk tolerance.
[from: Can Oncology Workflows Run Without Human Touch? – Anant Shankhdhar, Risa Labs — YouTube · AI Engineer · https://youtu.be/_cVfz88_j7A]
Source: Can Oncology Workflows Run Without Human Touch? – Anant Shankhdhar, Risa Labs (AI Engineer)
Text summarized and optimized using Anthropic’s models and reviewed by a human.