When to Abort a Stuck Bulk Export
2Min to read 0 View

Every FHIR client that runs bulk exports at scale eventually has an export that hangs. The polling loop returns the same progress hint hour after hour, the manifest never populates, and the client is left wondering whether to wait or cancel. Naming the abort criteria before the incident happens is the discipline that keeps the decision from becoming a judgment call at 3am.

The trick is to write the criteria down. For related walkthroughs, more FHIR implementation patterns collects the surrounding material.

What Stuck Actually Looks Like

Three symptoms suggest an export is stuck rather than slow:

  1. The polling response returns identical progress across multiple poll intervals.
  2. The manifest file count has not grown for a defined window.
  3. The server-side utilization for the export has dropped to near zero.

Any one of these can mean a slow export. Two of them at once usually mean something is wrong. All three together is a strong signal to abort. A pass through the site's export duration estimator provides the bounded ETA that turns "slow" into a numeric threshold.

The Timeout That Matters

Every scheduled export should carry a maximum duration set at kickoff. When the export exceeds the duration by a defined multiple, the client aborts. The multiple should be more than one but less than three; two is a reasonable default.

Timeouts that fire too eagerly waste work; timeouts that fire too late waste time. Naming the timeout up front is what turns the decision from a debate into a rule.

Server-Side Symptoms Worth Watching

If the client has access to server-side observability, three signals are the most useful indicators of a stuck export:

  • No CPU or I/O on the worker that owns the export.
  • No entries in the export's progress log for the last N minutes.
  • A pool slot held by the export with no active database work.

Any of these persistently means the export is not making progress. For the underlying driver framing, the four things that dominate $export duration covers why exports slow down; the observability signals catch when slow has become stuck.

The Restart-vs-Resume Question

Aborting an export raises the question of whether to restart from scratch or resume from a checkpoint. The Bulk Data spec does not define resume semantics; some servers support it as an extension.

Restarting from scratch is safe and expensive. Resuming from a checkpoint is cheap and only works when the server supports it. Deployments that plan for restart at abort time avoid the disappointment of discovering resume support does not exist.

Communicating the Abort

Aborted exports should generate a summary the same way completed exports do. Duration to abort, reason for abort, mitigations attempted, and the plan for the next attempt. The summary becomes the input to the next export planning.

Silent aborts leave the operational team wondering what happened; documented aborts let the next attempt be smarter. For the client-side monitoring framing that catches stuck exports early, monitoring $export progress from the client side is the accompanying reference.

Publishing the Abort Policy

Abort policies that live in tribal knowledge produce inconsistent decisions across operators. Policies that are documented produce consistent decisions and give the on-call escalation a clear rubric to reference.

Every bulk-export operation that lands well shares the same posture: predict the ETA, monitor for stuck symptoms, apply the timeout, document the abort. The next export benefits from the discipline.

Retro-crt-grain diagram of an abort decision tree with symptoms on the left branching into wait or abort paths through warm burnt-orange scanline arrows

Sources