Uploading & managing files

Before you can search, your documents need to be in LightOn. Uploading a file triggers an ingestion pipeline that parses the content, splits it into chunks, generates embeddings, and indexes everything. The whole process typically takes a few seconds for a standard PDF. Ingestion is asynchronous: the upload returns immediately with a pending status, and you poll GET /api/v3/files/{id} for completion.

This tutorial covers POST /api/v3/files and GET /api/v3/files. The full schema for every endpoint and parameter lives in the API reference.

Upload a file

Send the file as multipart/form-data with the destination workspace_id:

from lighton import File, LightOn, Workspace

workspace_id = 42  # replace with your workspace ID

with LightOn() as client:  # reads LIGHTON_API_KEY from the environment
    workspace = Workspace.get(client, workspace_id)
    doc = workspace.ingest(File(path="handbook.pdf"))
    print(doc.id, doc.status)

import { File, LightOn, Workspace } from "@lighton-ai/sdk"

const workspaceId = 42 // replace with your workspace ID

const client = new LightOn() // reads LIGHTON_API_KEY from the environment
const workspace = await Workspace.get(client, workspaceId)
const doc = await workspace.ingest(new File({ path: "handbook.pdf" }))
console.log(doc.id, doc.status)

import os
import requests

workspace_id = 42  # replace with your workspace ID
headers = {"Authorization": f"Bearer {os.environ['LIGHTON_API_KEY']}"}

with open("handbook.pdf", "rb") as f:
    response = requests.post(
        "https://api.lighton.ai/api/v3/files",
        headers=headers,
        data={"workspace_id": workspace_id},
        files={"file": f},
    )

file = response.json()
print(file["id"], file["status"], file["upload_session_uuid"])

The response is a 201 with the new file record, including an upload_session_uuid you can use later to find every file uploaded in the same batch.

Wait for indexing to complete

Poll GET /api/v3/files/{id} until status reaches embedded:

from lighton import File, FileStatus, LightOn

file_id = 1234  # replace with your file ID

with LightOn() as client:
    doc = File.get(client, file_id)
    doc.wait()  # polls refresh() until ingestion reaches a terminal status

    if doc.status == FileStatus.embedded:
        print("Ready to search")
    else:
        print("Ingestion failed:", doc.status, doc.status_detail)

import { File, FileStatus, LightOn } from "@lighton-ai/sdk"

const fileId = 1234 // replace with your file ID

const client = new LightOn()
const doc = await File.get(client, fileId)
await doc.wait() // polls refresh() until ingestion reaches a terminal status

if (doc.status === FileStatus.embedded) {
  console.log("Ready to search")
} else {
  console.log("Ingestion failed:", doc.status, doc.statusDetail)
}

import os
import time
import requests

file_id = 1234  # replace with your file ID
headers = {"Authorization": f"Bearer {os.environ['LIGHTON_API_KEY']}"}

for _ in range(60):  # give up after ~2 minutes
    response = requests.get(
        f"https://api.lighton.ai/api/v3/files/{file_id}",
        headers=headers,
    )
    body = response.json()
    if body["status"] == "embedded":
        print("Ready to search")
        break
    if body["status"] in ("parsing_failed", "embedding_failed", "fail"):
        print("Ingestion failed:", body.get("status_detail"))
        break
    time.sleep(2)

The status field moves through these stages:

Status	What’s happening
`pending`	Queued for processing
`parsing`	Extracting text from the document
`parsing_failed`	Parsing failed, see `status_detail`
`embedding`	Generating vector embeddings
`embedding_failed`	Embedding failed, see `status_detail`
`embedded`	Indexed and ready to search
`updating`	Re-indexing in progress
`fail`	Generic failure, see `status_detail`

status_vision tracks the same lifecycle for vision/image embeddings: pending, processing, embedded, fail, or - (not available for this file). One caveat: status describes the run that has already happened. If reprocessing is queued but hasn’t started, pending_reprocess is non-null and status still reports the previous run. That only comes up after a file replacement, where trusting status alone would report success against the old file.

Organising documents with tags and titles

Add a human-readable title and assign tag IDs at upload time. Tags can be sent as a JSON-encoded array string or as repeated form fields with the same name.

from lighton import File, LightOn, Workspace

workspace_id = 42  # replace with your workspace ID

with LightOn() as client:
    workspace = Workspace.get(client, workspace_id)
    doc = workspace.ingest(
        File(path="q4-report.pdf", title="Q4 Financial Report"),
        tags=[1, 2],  # tag IDs from Tag.list(client)
    )
    print(doc.id, doc.title)

import { File, LightOn, Workspace } from "@lighton-ai/sdk"

const workspaceId = 42 // replace with your workspace ID

const client = new LightOn()
const workspace = await Workspace.get(client, workspaceId)
const doc = await workspace.ingest(
  new File({ path: "q4-report.pdf", title: "Q4 Financial Report" }),
  { tags: [1, 2] }, // tag IDs from Tag.list(client)
)
console.log(doc.id, doc.title)

import os
import requests

workspace_id = 42  # replace with your workspace ID
headers = {"Authorization": f"Bearer {os.environ['LIGHTON_API_KEY']}"}

with open("q4-report.pdf", "rb") as f:
    response = requests.post(
        "https://api.lighton.ai/api/v3/files",
        headers=headers,
        data={
            "workspace_id": workspace_id,
            "title": "Q4 Financial Report",
            "tags": [3, 7],  # list of tag IDs
        },
        files={"file": f},
    )
print(response.json())

If a tag ID is invalid, the file is still created but the response is a 207 (multi-status) with a message explaining which tags were rejected. To replace tags after upload, PATCH /api/v3/files/{id} with a new tags array. It replaces all existing tags, manual and auto-assigned. Send [0] (sentinel) to remove every tag when using multipart format. To add tags without touching existing ones, POST /api/v3/files/{id}/tags.

Tracking documents from external systems

If you’re ingesting documents from a third-party system (ServiceNow, Confluence, SharePoint, etc.), store the source identifier in external_metadata. This lets you find the LightOn file later given only the external ID, and surface the original URL in your UI.

from lighton import ExternalMetadata, File, LightOn

workspace_id = 42  # replace with your workspace ID

with LightOn() as client:  # reads LIGHTON_API_KEY from the environment
    doc = File(
        path="srv-456789.pdf",
        workspace_id=workspace_id,
        external_metadata=ExternalMetadata(
            external_id="SRV-456789",
            doc_type="incident",
            additional_metadata={
                "external_url": "https://servicenow.example.com/incident/SRV-456789",
            },
        ),
    ).create(client)
    print(doc.id, doc.external_metadata)

import { File, LightOn } from "@lighton-ai/sdk"

const workspaceId = 42 // replace with your workspace ID

const client = new LightOn() // reads LIGHTON_API_KEY from the environment
const doc = await new File({
  path: "srv-456789.pdf",
  workspaceId,
  externalMetadata: {
    externalId: "SRV-456789",
    docType: "incident",
    additionalMetadata: {
      external_url: "https://servicenow.example.com/incident/SRV-456789",
    },
  },
}).create(client)
console.log(doc.id, doc.externalMetadata)

import json
import os
import requests

workspace_id = 42  # replace with your workspace ID
headers = {"Authorization": f"Bearer {os.environ['LIGHTON_API_KEY']}"}

with open("srv-456789.pdf", "rb") as f:
    response = requests.post(
        "https://api.lighton.ai/api/v3/files",
        headers=headers,
        data={
            "workspace_id": workspace_id,
            "external_metadata": json.dumps({
                "external_id": "SRV-456789",
                "doc_type": "incident",
                "additional_metadata": {
                    "external_url": "https://servicenow.example.com/incident/SRV-456789",
                },
            }),
        },
        files={"file": f},
    )
print(response.json())

external_id is required when creating; doc_type and additional_metadata are optional. When sent via multipart/form-data, the whole external_metadata value must be a JSON string. Updates merge rather than replace, including inside additional_metadata, so you can patch one key and leave the rest alone. external_id cannot be changed on a document imported from a datasource: the sync matches its documents on it, so a rename would detach the document from its source. Retrieve it later by external ID:

GET /api/v3/files?external_metadata__external_id=SRV-456789

Listing and filtering your documents

GET /api/v3/files supports rich filtering. A few common patterns:

from lighton import File, LightOn

with LightOn() as client:
    # File.list passes filters through as query params and follows pagination
    File.list(client, workspace_id=42)
    File.list(client, search="security policy", search_details=True)
    File.list(client, tag_id=3, extension="pdf", ordering="-created_at")

    docs = File.list(client, total_pages_min=10, total_pages_max=50)
    for doc in docs:
        print(doc.id, doc.title, doc.total_pages)

import { File, LightOn } from "@lighton-ai/sdk"

const client = new LightOn()
// File.list passes filters through as query params and follows pagination
await File.list(client, { workspaceId: 42 })
await File.list(client, { filters: { search: "security policy", search_details: true } })
await File.list(client, { filters: { tag_id: "3", extension: "pdf", ordering: "-created_at" } })

const docs = await File.list(client, { filters: { total_pages_min: 10, total_pages_max: 50 } })
for (const doc of docs) {
  console.log(doc.id, doc.title, doc.totalPages)
}

import os
import requests

url = "https://api.lighton.ai/api/v3/files"
headers = {"Authorization": f"Bearer {os.environ['LIGHTON_API_KEY']}"}

# All files in a workspace
requests.get(url, headers=headers, params={"workspace_id": "42"})

# Semantic search across filenames and titles, with the top chunk inline
requests.get(url, headers=headers, params={"search": "security policy", "search_details": True})

# PDFs tagged 'legal', most recent first
requests.get(url, headers=headers, params={"tag_id": "3", "extension": "pdf", "ordering": "-created_at"})

# Files in a 10–50 page window
response = requests.get(url, headers=headers, params={"total_pages_min": 10, "total_pages_max": 50})
print(response.json())

Set include_details=true to receive the signature (TLSH hash for duplicate detection) and parser fields on each result. For advanced metadata filtering, see the Facets tutorials: classify files by type, set custom attributes, then filter with operators. See the filter reference for the full DSL.

Downloading a document

Once a file is ingested, GET /api/v3/files/{id}/download gives you the stored bytes back, so your app can offer the original document next to the answer it grounded.

from pathlib import Path

from lighton import File, LightOn

file_id = 1234  # replace with your file ID

with LightOn() as client:  # reads LIGHTON_API_KEY from the environment
    doc = File.get(client, file_id)
    content = doc.download()  # or download("rendered_pdf") / download("transcript")
    Path("downloaded.pdf").write_bytes(content)
    print(len(content))

import { writeFile } from "node:fs/promises"
import { File, LightOn } from "@lighton-ai/sdk"

const fileId = 1234 // replace with your file ID

const client = new LightOn() // reads LIGHTON_API_KEY from the environment
const doc = await File.get(client, fileId)
const content = await doc.download() // or download("rendered_pdf") / download("transcript")
await writeFile("downloaded.pdf", content)
console.log(content.length)

import os
import requests

file_id = 1234  # replace with your file ID
headers = {"Authorization": f"Bearer {os.environ['LIGHTON_API_KEY']}"}

response = requests.get(
    f"https://api.lighton.ai/api/v3/files/{file_id}/download",
    headers=headers,
    params={"purpose": "original"},  # or "rendered_pdf", "transcript"
)
with open("downloaded.pdf", "wb") as f:
    f.write(response.content)
print(response.status_code, len(response.content))

The purpose parameter picks which stored version you get:

Purpose	What you get
`original` (default)	The exact bytes you uploaded
`rendered_pdf`	A PDF rendition, for in-browser viewers that only handle PDF
`transcript`	Sanitized markdown text, for audio and video sources

If the requested purpose has no associated file, the API falls back to original rather than returning a 404.

Thumbnails

Every document also gets a 256x256 WebP thumbnail, handy for a document picker or a result list. Generation runs asynchronously and separately from ingestion, so an embedded file may still have none. Check the thumbnail.status field on the file before fetching, or GET /api/v3/files/{id}/thumbnail returns a 404:

from pathlib import Path

from lighton import File, LightOn, ThumbnailStatus

file_id = 1234  # replace with your file ID

with LightOn() as client:  # reads LIGHTON_API_KEY from the environment
    doc = File.get(client, file_id)
    if doc.thumbnail and doc.thumbnail.status is ThumbnailStatus.READY:
        Path("thumbnail.webp").write_bytes(doc.download_thumbnail())
        print(doc.thumbnail.url)

import { writeFile } from "node:fs/promises"
import { File, LightOn, ThumbnailStatus } from "@lighton-ai/sdk"

const fileId = 1234 // replace with your file ID

const client = new LightOn() // reads LIGHTON_API_KEY from the environment
const doc = await File.get(client, fileId)
if (doc.thumbnail?.status === ThumbnailStatus.READY) {
  await writeFile("thumbnail.webp", await doc.downloadThumbnail())
  console.log(doc.thumbnail.url)
}

import os
import requests

file_id = 1234  # replace with your file ID
headers = {"Authorization": f"Bearer {os.environ['LIGHTON_API_KEY']}"}

file = requests.get(
    f"https://api.lighton.ai/api/v3/files/{file_id}",
    headers=headers,
).json()

if file["thumbnail"]["status"] == "READY":
    thumbnail = requests.get(
        f"https://api.lighton.ai/api/v3/files/{file_id}/thumbnail",
        headers=headers,
    )
    with open("thumbnail.webp", "wb") as f:
        f.write(thumbnail.content)
    print(thumbnail.status_code, len(thumbnail.content))

status is MISSING, PROCESSING, or READY. url is populated only once it reads READY.

Reading a document’s parsed text

The platform already parsed your document at ingestion, so you don’t need to re-upload it to /parse to read the text. Pass include_content=true to the file detail endpoint and you get the same {index, markdown} page shape /parse returns:

from lighton import File, LightOn

file_id = 1234  # replace with your file ID

with LightOn() as client:  # reads LIGHTON_API_KEY from the environment
    doc = File.get(client, file_id)
    for page in doc.pages():  # same {index, markdown} shape `parse` returns
        print(f"--- page {page.index} ---")
        print(page.markdown[:200])

import { File, LightOn } from "@lighton-ai/sdk"

const fileId = 1234 // replace with your file ID

const client = new LightOn() // reads LIGHTON_API_KEY from the environment
const doc = await File.get(client, fileId)
for (const page of await doc.pages()) { // same {index, markdown} shape `parse` returns
  console.log(`--- page ${page.index} ---`)
  console.log(page.markdown.slice(0, 200))
}

import os
import requests

file_id = 1234  # replace with your file ID
headers = {"Authorization": f"Bearer {os.environ['LIGHTON_API_KEY']}"}

response = requests.get(
    f"https://api.lighton.ai/api/v3/files/{file_id}",
    headers=headers,
    params={"include_content": True},
)
for page in response.json().get("pages", []):
    print(f"--- page {page['index']} ---")
    print(page["markdown"][:200])

{
  "id": 1234,
  "title": "Acme NDA 2025",
  "pages": [
    {"index": 1, "markdown": "# Non-Disclosure Agreement\n\nThis NDA is entered into..."},
    {"index": 2, "markdown": "## 3. Term\n\nThe obligations survive for three years..."}
  ]
}

The text can be large, so include_content is off by default. Ask for it only when you need the content, not on every list call.

Replacing a document’s file

When the source document changes, you don’t have to delete and re-upload: send the new file as the file part of PATCH /api/v3/files/{id} and the document is re-ingested in place. It keeps its id, title, manual tags, content types and attribute values, so every reference you stored still resolves.

from lighton import File, LightOn

file_id = 1234  # replace with your file ID

with LightOn() as client:  # reads LIGHTON_API_KEY from the environment
    doc = File.get(client, file_id)
    doc.replace("q4-report.pdf", wait=True)  # wait=True also waits out pending_reprocess
    print(doc.id, doc.status, doc.filename)

import { File, LightOn } from "@lighton-ai/sdk"

const fileId = 1234 // replace with your file ID

const client = new LightOn() // reads LIGHTON_API_KEY from the environment
const doc = await File.get(client, fileId)
await doc.replace("q4-report.pdf", { wait: true }) // wait: true also waits out pendingReprocess
console.log(doc.id, doc.status, doc.filename)

import os
import requests

file_id = 1234  # replace with your file ID
headers = {"Authorization": f"Bearer {os.environ['LIGHTON_API_KEY']}"}

with open("q4-report.pdf", "rb") as f:
    response = requests.patch(
        f"https://api.lighton.ai/api/v3/files/{file_id}",
        headers=headers,
        files={"file": f},
    )
body = response.json()
print(body["status"], body["pending_reprocess"])  # 'embedded' 'update': the new file is queued

Only the file and what is derived from it (text, chunks, embeddings, auto-assigned tags, summaries) is rebuilt. The new file may be of a different type than the one it replaces. You can send title, tags or external_metadata in the same request: the metadata changes apply immediately, the file is queued.

The response describes the document as it still is. status typically reads embedded (the previous run), and filename, size, total_pages and summaries still describe the old file. The one field that tells you a replacement is pending is pending_reprocess, which reads update until the pipeline picks the document up.So poll until pending_reprocess is null and status is terminal. Treating a terminal status on its own as completion reports success against the previous file. The SDK’s wait() already accounts for this, which is what replace(..., wait=True) uses.

The document keeps serving its previous content until processing actually starts, so search, extract and download stay consistent in the meantime. Once conversion begins, a document whose new file needs rendering has no rendered PDF yet, and download?purpose=rendered_pdf serves the original instead. Gate an in-browser viewer on status, not on the download succeeding. Replacing a file needs an owner or editor role in the document’s workspace, and is rejected in workspaces configured for synced documents. The storage quota is checked on the size difference, not the full new size.

Deleting files

Single delete:

from lighton import File, LightOn

file_id = 1234  # replace with your file ID

with LightOn() as client:
    File.get(client, file_id).delete()
    print(f"deleted file {file_id}")

import { File, LightOn } from "@lighton-ai/sdk"

const fileId = 1234 // replace with your file ID

const client = new LightOn()
await (await File.get(client, fileId)).delete()
console.log(`deleted file ${fileId}`)

import os
import requests

file_id = 1234  # replace with your file ID
headers = {"Authorization": f"Bearer {os.environ['LIGHTON_API_KEY']}"}

response = requests.delete(
    f"https://api.lighton.ai/api/v3/files/{file_id}",
    headers=headers,
)
print(response.status_code)

Bulk delete:

from lighton import File, LightOn

file_ids = [123, 124, 125]  # replace with your file IDs

with LightOn() as client:  # reads LIGHTON_API_KEY from the environment
    File.delete_many(client, file_ids)
    print(f"deleted {len(file_ids)} files")

import { File, LightOn } from "@lighton-ai/sdk"

const fileIds = [123, 124, 125] // replace with your file IDs

const client = new LightOn() // reads LIGHTON_API_KEY from the environment
await File.deleteMany(client, fileIds)
console.log(`deleted ${fileIds.length} files`)

import os
import requests

headers = {"Authorization": f"Bearer {os.environ['LIGHTON_API_KEY']}"}

response = requests.post(
    "https://api.lighton.ai/api/v3/files/bulk-delete",
    headers=headers,
    json={"ids": [123, 124, 125]},
)
print(response.json())

Both return 204 No Content on success. Files in synced (datasource-managed) workspaces cannot be deleted manually. The API returns 400.

Common errors

Status	Cause
`400`	Validation error, unsupported file type, or synced-workspace constraint
`401`	Missing or invalid API key
`403`	Permission denied (no upload/delete rights)
`404`	File does not exist or is not accessible
`429`	Too many concurrent uploads for this session

Overview

Build a searchable knowledge base

Classify and organise documents

Process documents on the fly

Upload a file

Wait for indexing to complete

Organising documents with tags and titles

Tracking documents from external systems

Listing and filtering your documents

Downloading a document

Thumbnails

Reading a document’s parsed text

Replacing a document’s file

Deleting files

Common errors

​Upload a file

​Wait for indexing to complete

​Organising documents with tags and titles

​Tracking documents from external systems

​Listing and filtering your documents

​Downloading a document

​Thumbnails

​Reading a document’s parsed text

​Replacing a document’s file

​Deleting files

​Common errors

Upload a file

Wait for indexing to complete

Organising documents with tags and titles

Tracking documents from external systems

Listing and filtering your documents

Downloading a document

Thumbnails

Reading a document’s parsed text

Replacing a document’s file

Deleting files

Common errors