Document Ingestion

The ingestion API lets you programmatically add, list, and remove documents in the Hadiq.io index. It is useful for custom pipelines where you already control document extraction and want to push content into Hadiq.io without configuring a connector. All ingestion endpoints require admin or curator role and are served under the /api/Hadiq.io-api prefix.

Documents ingested through this API are tracked separately from connector-synced documents and can only be deleted via this API.

Ingest a document

POST /api/Hadiq.io-api/ingestion

Indexes a single document. If a document with the same id already exists, it is updated in place.

Request body

object

required

The document to ingest.

Show document fields

string

A unique identifier for the document. If omitted, Hadiq.io derives one from the content. Use a stable identifier (e.g. a URL or database primary key) so that updates replace the existing document rather than creating a duplicate.

string

required

Human-readable title displayed in the Hadiq.io UI and citations (e.g. "Q4 OKR Planning Doc").

array

required

One or more content sections that make up the document.

Show section fields

string

required

The text content of this section.

string

URL anchor for this specific section. Used in citations so users can jump directly to the relevant part of the source document.

string

Source type identifier. Defaults to file. Examples: file, web, confluence, notion.

object

Key-value pairs of arbitrary metadata. Values must be strings or arrays of strings.

{
  "department": "engineering",
  "tags": ["onboarding", "policy"],
  "author": "Jane Smith"
}

string (ISO 8601)

When the source document was last modified. Defaults to the current time if omitted.

string

Title used for search ranking. Defaults to semantic_identifier if not provided.

array

Document owners or authors.

Show owner fields

string

Full display name.

string

Email address.

array

Secondary owners (assignees, space owners). Same shape as primary_owners.

integer

Connector-credential pair to associate the document with. Defaults to the system ingestion CC pair if omitted.

Response

string

The ID of the document that was indexed (either the id you provided or the derived ID).

boolean

true if the document was newly created, false if it replaced an existing document.

curl -X POST https://cloud.hadiq.io/api/onyx-api/ingestion \
  -H "Authorization: Bearer hadiqk-..." \
  -H "Content-Type: application/json" \
  -d '{
    "document": {
      "id": "hr-policy-pto-v3",
      "semantic_identifier": "PTO Policy — Employee Handbook",
      "title": "PTO Policy",
      "source": "file",
      "sections": [
        {
          "text": "Full-time employees accrue 15 days of PTO per year, prorated in the first year based on start date.",
          "link": "https://wiki.example.com/hr/pto#accrual"
        },
        {
          "text": "PTO requests must be submitted at least 5 business days in advance via the HR portal.",
          "link": "https://wiki.example.com/hr/pto#requests"
        }
      ],
      "metadata": {
        "department": "hr",
        "tags": ["policy", "benefits", "pto"],
        "author": "People Operations"
      },
      "primary_owners": [
        {
          "display_name": "People Operations",
          "email": "hr@example.com"
        }
      ],
      "doc_updated_at": "2025-01-15T10:00:00Z"
    }
  }'

import requests

resp = requests.post(
    "https://cloud.hadiq.io/api/onyx-api/ingestion",
    headers={"Authorization": "Bearer hadiqk-..."},
    json={
        "document": {
            "id": "hr-policy-pto-v3",
            "semantic_identifier": "PTO Policy — Employee Handbook",
            "title": "PTO Policy",
            "source": "file",
            "sections": [
                {
                    "text": (
                        "Full-time employees accrue 15 days of PTO per year, "
                        "prorated in the first year based on start date."
                    ),
                    "link": "https://wiki.example.com/hr/pto#accrual",
                },
                {
                    "text": (
                        "PTO requests must be submitted at least 5 business days "
                        "in advance via the HR portal."
                    ),
                    "link": "https://wiki.example.com/hr/pto#requests",
                },
            ],
            "metadata": {
                "department": "hr",
                "tags": ["policy", "benefits", "pto"],
                "author": "People Operations",
            },
            "primary_owners": [
                {"display_name": "People Operations", "email": "hr@example.com"}
            ],
            "doc_updated_at": "2025-01-15T10:00:00Z",
        }
    },
)
result = resp.json()
print("document_id:", result["document_id"])
print("already_existed:", result["already_existed"])

List ingested documents

GET /api/onyx-api/ingestion

Returns a minimal list of all documents that were ingested via the ingestion API.

Response

An array of DocMinimalInfo objects.

string

Document identifier.

string

Human-readable title (the semantic_identifier set at ingestion time).

string

Link to the source document, if provided.

curl https://cloud.hadiq.io/api/hadiq.io-api/ingestion \
  -H "Authorization: Bearer hadiqk-..."

import requests

resp = requests.get(
    "https://cloud.hadiq.io/api/hadiq.io-api/ingestion",
    headers={"Authorization": "Bearer hadiqk-..."},
)
for doc in resp.json():
    print(doc["document_id"], doc["semantic_id"])

Delete an ingested document

DELETE /api/hadiq.io-api/ingestion/{document_id}

Removes a document from the Hadiq.io index and database. Only documents originally ingested via the ingestion API can be deleted through this endpoint.

Path parameters

string

required

The document_id returned when the document was ingested (or the id field you provided).

Returns 204 No Content on success.

Deletion is permanent. The document is removed from both the vector index and the database. You must re-ingest the document if you need it back.

curl -X DELETE \
  "https://cloud.hadiq.io/api/hadiq.io-api/ingestion/hr-policy-pto-v3" \
  -H "Authorization: Bearer hadiqk-..."

import requests

requests.delete(
    "https://cloud.hadiq.io/api/hadiq.io-api/ingestion/hr-policy-pto-v3",
    headers={"Authorization": "Bearer onyxk-..."},
)

List documents by CC pair

GET /api/hadiq.io-api/connector-docs/{cc_pair_id}

Returns minimal info for all documents associated with a specific connector-credential pair.

Path parameters

integer

required

The CC pair whose documents to list.

Response

Same as list ingested documents — an array of DocMinimalInfo objects.

curl https://cloud.hadiq.io/api/hadiq.io-api/connector-docs/7 \
  -H "Authorization: Bearer hadiqk-..."

import requests

resp = requests.get(
    "https://cloud.hadiq/api/hadiq.io-api/connector-docs/7",
    headers={"Authorization": "Bearer onyxk-..."},
)
for doc in resp.json():
    print(doc["document_id"], doc["link"])

Bulk ingestion pattern

To ingest many documents efficiently, loop over your documents and call the ingestion endpoint for each one. The endpoint is idempotent — re-ingesting a document with the same id updates it in place.

Python

import requests
import time

BASE_URL = "https://cloud.hadiq.io"
HEADERS = {
    "Authorization": "Bearer hadiqk-...",
    "Content-Type": "application/json",
}

documents = [
    {
        "id": "wiki-page-1",
        "semantic_identifier": "Engineering Onboarding Guide",
        "source": "file",
        "sections": [{"text": "Welcome to engineering..."}],
        "metadata": {"department": "engineering"},
    },
    {
        "id": "wiki-page-2",
        "semantic_identifier": "Security Policy",
        "source": "file",
        "sections": [{"text": "All data must be encrypted at rest..."}],
        "metadata": {"department": "security"},
    },
]

for doc in documents:
    resp = requests.post(
        f"{BASE_URL}/api/hadiq.io-api/ingestion",
        headers=HEADERS,
        json={"document": doc},
    )
    resp.raise_for_status()
    result = resp.json()
    print(f"Indexed {result['document_id']}")
    time.sleep(0.1)  # be respectful of rate limits

Error codes

Status	Cause
`400`	The specified `cc_pair_id` does not exist.
`400`	Attempting to delete a document not created via the ingestion API.
`403`	Insufficient role (requires admin or curator).
`404`	Document not found.

Overview

Endpoints

Ingest a document

Request body

Response

List ingested documents

Response

Delete an ingested document

Path parameters

List documents by CC pair

Path parameters

Response

Bulk ingestion pattern

Error codes

​Ingest a document

​Request body

​Response

​List ingested documents

​Response

​Delete an ingested document

​Path parameters

​List documents by CC pair

​Path parameters

​Response

​Bulk ingestion pattern

​Error codes

Ingest a document

Request body

Response

List ingested documents

Response

Delete an ingested document

Path parameters

List documents by CC pair

Path parameters

Response

Bulk ingestion pattern

Error codes