Headless data importer: send files to an API, get validated JSON back

How the headless data importer works

Dromo Headless is for files that arrive with no one there to click through an importer: machine-to-machine transfers, bulk migrations, and data your backend receives from other systems. You send the file to an API, Dromo maps and validates it against your schema, and you get clean JSON back. Headless is available on the Enterprise plan.

An import is two requests. Create the import record with the schema to use and the file's name, then upload the file to the URL Dromo returns:

const res = await fetch("https://app.dromo.io/api/v1/headless/imports/", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "X-Dromo-License-Key": process.env.DROMO_BACKEND_KEY,
  },
  body: JSON.stringify({ schema_id: SCHEMA_ID, original_filename: "orders.csv" }),
});
const { id, upload } = await res.json();

await fetch(upload, { method: "PUT", body: fileBuffer });
// Store `id`. The import starts on its own once the upload completes.

There is no job to trigger and no worker to run. Requests use a backend license key, which is separate from the frontend key the embedded importer uses and should only ever live on your server. If the data is already structured records rather than a file, skip the upload and send the rows as a JSON array in initial_data. Dromo runs the same matching, validation, and hooks either way.

Every import reports one of six statuses. Three are in progress: AWAITING_UPLOAD, PENDING, and RUNNING. Three are outcomes. SUCCESSFUL means the data is clean and ready. NEEDS_REVIEW means a person has to look at it, and a review link is attached. FAILED means the file could not be processed at all. Those three outcomes are the only branches your code has to handle.

What a homegrown file pipeline misses

A script that reads a CSV from a bucket and inserts rows works for the first partner. It starts to break when the second partner sends a file with a title row, renamed columns, dates in a different order, and three duplicates. Each fix becomes another special case in code that nobody wants to own.

The difference that matters most is the fourth row. Every automated pipeline eventually gets a file it cannot handle on its own. The question is whether that file becomes an engineering ticket or a two-minute fix by the person who understands the data.

Send a person a link, not a stack trace

Most file pipelines fail in one of two ways. They reject any file that is not perfect, which means constant manual rework, or they accept bad data quietly and let it spread downstream. Headless is built around a third outcome.

When Dromo cannot finish an import on its own, the import stops in NEEDS_REVIEW instead of failing. That happens when the header row cannot be determined, when columns cannot be matched automatically, or when rows fail validation and your schema is set to block invalid submissions.

A NEEDS_REVIEW import includes a review_url, which you send to whoever should resolve it. Opening it in a browser loads that exact import in Dromo's importer, with the unmatched columns and failing rows ready to fix, using the same guided steps as an embedded user. You can see that review experience in the live demo. When they finish, the import moves to SUCCESSFUL and your pipeline continues as if nothing happened.

FAILED is reserved for problems no person can fix in an interface: a corrupt file that cannot be parsed, a completely empty file, or a custom hook that throws an unhandled exception. A failed import has no results and no review link, so it goes to whoever owns the integration.

Define the schema once, or per import

The schema describes what the finished data must look like: the fields, their types, and the validators each one must pass. There are two ways to supply it.

A saved schema is built in Schema Studio and referenced by schema_id. It suits feeds whose shape is stable, and it lets a product manager change a validation rule without a code change.

An inline schema is passed in the request as fields, with optional settings and hooks. It suits schemas generated at runtime, such as a platform where each customer defines their own columns. An import uses one approach or the other, never both.

You can also keep a saved schema and override its settings or hooks for a single import. The overrides are merged on top of the saved schema for that import only, and every other import that uses the schema is unaffected:

{
  "schema_id": "YOUR_SCHEMA_ID",
  "settings": { "allowInvalidSubmit": true },
  "original_filename": "data.csv"
}

Get clean JSON back the moment an import finishes

Configure a webhook in the Dromo dashboard and Dromo notifies you when each import completes. Dashboard webhooks retry on delivery failure. The import_completed event carries the import id, the filename, the final status, and num_data_rows, so you know what arrived before you fetch the data.

With the ID, request a presigned URL for the results:

curl -H "X-Dromo-License-Key: your-backend-key" \
  'https://app.dromo.io/api/v1/headless/imports/{id}/url/'

The data behind that URL is validated JSON that matches your schema field for field, ready to upsert. If webhooks do not suit your setup, you can poll the import's status endpoint instead, though webhooks are the better choice at any real volume.

Because there is no browser involved, headless imports are processed on Dromo's servers rather than in Private Mode. That processing is covered by the same controls as the rest of the platform: a completed SOC 2 Type II audit, Business Associate Agreements for HIPAA, Standard Contractual Clauses for GDPR, and, on Enterprise, retention periods you set.

Automate recurring partner files over SFTP

Some files never pass through an API call. Banks, insurers, healthcare systems, and older enterprise software often only deliver files by SFTP, on a schedule, in a layout that drifts a little each time.

On the Enterprise plan, headless imports can arrive over SFTP as well as the API, and those files are checked against the same schemas, validators, and hooks as any other headless import. The rules you wrote once apply to every channel files come in on.

Because every headless import runs against a schema, a renamed column becomes a matching question for a person rather than values landing in the wrong field. This matters most for teams loading recurring vendor files into an ERP. For more on designing these pipelines, see automating CSV import over API and SFTP.

Headless or embedded?

Headless is for files with no person present when they arrive: scheduled feeds, backend integrations, migrations, and anything triggered by another system.

The embedded importer is for a user uploading a file inside your product, who can match columns and fix errors as part of the flow.

They share saved schemas, validators, and hooks, and NEEDS_REVIEW is where they meet: a headless import that needs a person is finished in the same interface embedded users see. With the schema saved in Schema Studio, a team can add headless to an existing embedded setup without redefining its rules.