The client is where the FHIR bulk-export progress conversation actually lives. The server publishes a poll endpoint and, sometimes, a progress hint header. Everything else is client-side inference. Every bulk-data pipeline that survives at scale carries a monitoring layer that turns sparse server signals into a useful dashboard the operations team can read.
Naming the signals is the discipline. For related walkthroughs, more on FHIR bulk data collects the surrounding material.
Poll Interval Matters
The polling loop is the heartbeat of client-side monitoring. Poll too frequently and the client wastes network cycles; poll too rarely and the client discovers stuck exports late.
The spec's Retry-After header hints at a reasonable interval. When the header is present, honor it. When it is absent, a common pattern is exponential backoff starting at fifteen seconds and capping at three minutes. A pass through the site's export duration estimator helps you pick a starting interval against the projected duration.
Manifest-Level Progress
The most reliable progress signal is the manifest. As the server writes NDJSON files, the manifest lists them. Comparing manifest file count against the expected count gives a percentage that matches what the server has actually produced.
Manifest-based progress requires the server to publish the manifest incrementally. Some servers publish only at the end; others publish as files are written. Prefer servers that publish incrementally; the operational visibility is worth it.
X-Progress Header Handling
Some servers include an X-Progress header on the polling response. The header format is not standardized; it might carry a percentage, a resource count, or a free-form status message.
Parse the header defensively. Some clients treat unrecognized formats as opaque status strings and display them in the dashboard; others ignore the header entirely. Both patterns work. For the wider gap the spec leaves, why the FHIR bulk data spec cannot tell you the ETA covers the shape.
Reconciling With the Predicted ETA
The prediction from before the export started should tighten as monitoring proceeds. Each poll returns real progress; the ETA range narrows toward the actual completion time.
Dashboards that show predicted-vs-actual make the tightening visible. Dashboards that show only current status hide it and lose the calibration signal. For the prediction framing, predicting how long a bulk data $export will actually take covers how to bracket the initial estimate.
Alerting on Stuck Symptoms
The monitoring layer should carry alerts on stuck-export symptoms: no manifest growth for N minutes, no progress-header movement for N polls, no server-side activity indicators. For the specific abort criteria, when to abort a stuck bulk export covers the thresholds.
Alerts that fire too eagerly waste attention; alerts that fire too late waste time. Naming the thresholds up front is what turns the alerts into a useful operational signal.
Delivery-Side Monitoring
The monitoring loop does not end when the manifest completes. The delivery step (direct download, presigned URLs, SFTP, streaming) has its own progress to track. For the delivery framing, delivery method and its impact on end-to-end ETA covers what to watch.
Every bulk-data client that runs at scale carries end-to-end monitoring, not just kickoff-to-manifest monitoring. The end-to-end view is what the operations team actually reads.
Publishing the Dashboard
Client-side monitoring that lives in a script is invisible to the rest of the team. A dashboard that shows active exports, predicted ETAs, actual progress, and alert status is the artifact operations reads.
The dashboard does not need to be elaborate. A single-page view with the four metrics above is enough for most deployments.
Sources
- HL7 FHIR Bulk Data Access IG - HL7 FHIR Bulk Data Access IG, canonical spec covering polling and manifest semantics