Clinical and scientific decision-making

Overview

The lab’s other four areas are methodological; this one supplies the problems they address. Clinical care exhibits every structural difficulty at once: the recorded state is a fraction of the patient’s state, the log was generated by clinicians conditioning on information that was never written down, outcomes resolve over days, some trajectories cannot be recovered from, and a single decision fixes several interacting quantities. Automated experimentation presents the same structure under a different cost model — select the next configuration under uncertainty while earlier attempts remain unresolved, against a budget that makes exploration materially expensive. The two settings share enough structure for methods to transfer and differ enough that the transfer is diagnostic.

Themes

• Partial observability and state construction — a policy over clinical time series acts on a learned summary of an irregularly sampled history, and the summary is a modeling decision with measurable consequences for the resulting policy (An Empirical Study of Representation Learning for Reinforcement Learning in Healthcare)
• Confounded observational records — treatment decisions in a medical record respond to unobserved covariates, which makes the identifying assumptions rather than the estimator the substantive part of any claim; see offline RL
• Transfer across populations and sites — grounding transfer in counterfactual reasoning rather than distributional similarity, and encoding the invariances that make it possible as inductive bias (Counterfactual Transfer via Inductive Bias in Clinical Settings)
• Unrecoverable clinical states — negative outcomes in a record identify states and treatments to avoid even where the data cannot support an optimal policy (Medical Dead-ends and Learning to Identify High-Risk States and Treatments), which is developed further under irreversibility and risk
• Composed interventions — clinical and experimental actions set multiple interacting quantities per decision, so the faithful formulation is combinatorial; see structured action spaces
• Person-centered policies — the target is not a policy that is optimal in expectation over a population but one that adapts to the individual and the context, holds up outside its training distribution, and respects the preferences and constraints of its intended use
• Sequential experimental design — formulating automated experimentation as a sequential decision problem in which the readout is a delayed proxy and prior decisions are still pending; see delayed and unreliable feedback

Open questions

• Which assumptions a decision-support claim rests on, and how to state them so that a clinician or experimentalist can judge whether they hold in a given case
• How to distinguish variation in observed treatment attributable to physiology from variation attributable to clinician judgment or resource constraint, and how to keep a learned policy from reproducing the latter
• What evidence is required before a policy learned offline is worth evaluating prospectively, and what an evaluation protocol short of deployment can establish
• How much of the clinical formulation carries over to experimental pipelines, where the horizon is set by an experiment cycle rather than by physiology

Also relevant: Clinically Motivated Sequential Decision Making Under Uncertainty in Offline Settings, which sets out the modeling decisions that anchor method development in the intended use case rather than in a benchmark.