The FHIR Bulk Data spec finishes at the manifest URL. Everything after that is delivery, and every delivery method has its own throughput signature, its own security model, and its own contribution to the end-to-end ETA. Sizing plans that skip the delivery step end up under-estimating the total time by an amount that ranges from minutes to hours.
Naming the delivery method up front is the discipline. For related walkthroughs, the FHIR learning path collects the surrounding material.
Direct Download From the Server
The simplest delivery pattern is direct HTTPS download from the server's file endpoint. The client fetches each file listed in the manifest and stores it locally.
Direct download is cheap to implement and expensive at scale. Every file transits the server's egress bandwidth, and large exports produce many gigabytes of NDJSON. Servers with limited egress become the bottleneck. A pass through the site's export duration estimator surfaces the delivery contribution when the method is entered explicitly.
Presigned Cloud Storage URLs
A more scalable pattern is for the server to place the export files on cloud storage (S3, GCS, Azure Blob) and return presigned URLs in the manifest. The client fetches directly from cloud storage.
Presigned URLs move the egress cost off the FHIR server and onto the cloud storage tier. Throughput scales with the cloud storage's per-object bandwidth, which is usually generous. Setup is more complex; delivery is faster once the setup is done.
SFTP or File Drop
Some regulated environments require file delivery via SFTP or a shared file drop. The server exports files locally and pushes them to the destination.
SFTP is slower than HTTPS or object storage but sometimes non-negotiable. The delivery-side ETA depends on the SFTP throughput and the retry behavior. Plans that use SFTP should budget for it as a distinct step.
Streaming to Kafka or a Queue
Analytics-heavy workloads sometimes stream export data to Kafka or a message queue rather than delivering as files. The server publishes NDJSON records as messages; consumers subscribe and process them in real time.
Streaming shifts the ETA question from "when does the last file land" to "when does the last message finish processing." The two metrics can differ substantially. For the client-side monitoring shape, monitoring $export progress from the client side covers the pattern.
Bandwidth Between Server and Delivery
Every delivery method has a bandwidth ceiling somewhere. Direct download is bounded by server egress. Presigned URLs are bounded by cloud storage tier. SFTP is bounded by the SFTP negotiation. Streaming is bounded by broker throughput.
Sizing plans should name the bandwidth ceiling per method and multiply through the payload size. For the payload-size framing that this multiplies against, chunking a large $export into manageable jobs covers the strategy.
Retention on the Delivery Side
Files delivered via cloud storage or SFTP usually have a retention window on the delivery side. Files that expire before the client picks them up produce a re-export loop that doubles the effective ETA.
Every delivery method should include a defined retention window and a client-side pickup deadline. For the wider prediction framing, predicting how long a bulk data $export will actually take covers how delivery composes with the earlier steps.
Naming the delivery method as its own line item is the shortcut that keeps the ETA honest all the way to the client's local disk.
Sources
- HL7 FHIR Bulk Data Access IG - HL7 FHIR Bulk Data Access IG, canonical reference for manifest URL delivery patterns