Uploading & managing files

Before you can search, your documents need to be in LightOn. Uploading a file triggers an ingestion pipeline that parses the content, splits it into chunks, generates embeddings, and indexes everything. The whole process typically takes a few seconds for a standard PDF. Ingestion is asynchronous: the upload returns immediately with a pending status, and you poll GET /api/v3/files/{id} for completion.

This tutorial covers POST /api/v3/files and GET /api/v3/files. The full schema for every endpoint and parameter lives in the API reference.

Upload a file

Send the file as multipart/form-data with the destination workspace_id:

from lighton import File, LightOn, Workspace

workspace_id = 42  # replace with your workspace ID

with LightOn() as client:  # reads LIGHTON_API_KEY from the environment
    workspace = Workspace.get(client, workspace_id)
    doc = workspace.ingest(File(path="handbook.pdf"))
    print(doc.id, doc.status)

import os
import requests

workspace_id = 42  # replace with your workspace ID
headers = {"Authorization": f"Bearer {os.environ['LIGHTON_API_KEY']}"}

with open("handbook.pdf", "rb") as f:
    response = requests.post(
        "https://api.lighton.ai/api/v3/files",
        headers=headers,
        data={"workspace_id": workspace_id},
        files={"file": f},
    )

file = response.json()
print(file["id"], file["status"], file["upload_session_uuid"])

The response is a 201 with the new file record, including an upload_session_uuid you can use later to find every file uploaded in the same batch.

Wait for indexing to complete

Poll GET /api/v3/files/{id} until status reaches embedded:

from lighton import File, FileStatus, LightOn

file_id = 1234  # replace with your file ID

with LightOn() as client:
    doc = File.get(client, file_id)
    doc.wait()  # polls refresh() until ingestion reaches a terminal status

    if doc.status == FileStatus.embedded:
        print("Ready to search")
    else:
        print("Ingestion failed:", doc.status, doc.status_detail)

import os
import time
import requests

file_id = 1234  # replace with your file ID
headers = {"Authorization": f"Bearer {os.environ['LIGHTON_API_KEY']}"}

for _ in range(60):  # give up after ~2 minutes
    response = requests.get(
        f"https://api.lighton.ai/api/v3/files/{file_id}",
        headers=headers,
    )
    body = response.json()
    if body["status"] == "embedded":
        print("Ready to search")
        break
    if body["status"] in ("parsing_failed", "embedding_failed", "fail"):
        print("Ingestion failed:", body.get("status_detail"))
        break
    time.sleep(2)

The status field moves through these stages:

Status	What’s happening
`pending`	Queued for processing
`parsing`	Extracting text from the document
`parsing_failed`	Parsing failed, see `status_detail`
`embedding`	Generating vector embeddings
`embedding_failed`	Embedding failed, see `status_detail`
`embedded`	Indexed and ready to search
`updating`	Re-indexing in progress
`fail`	Generic failure, see `status_detail`

status_vision tracks the same lifecycle for vision/image embeddings: pending, processing, embedded, fail, or - (not available for this file).

Organising documents with tags and titles

Add a human-readable title and assign tag IDs at upload time. Tags can be sent as a JSON-encoded array string or as repeated form fields with the same name.

from lighton import File, LightOn, Workspace

workspace_id = 42  # replace with your workspace ID

with LightOn() as client:
    workspace = Workspace.get(client, workspace_id)
    doc = workspace.ingest(
        File(path="q4-report.pdf", title="Q4 Financial Report"),
        tags=[1, 2],  # tag IDs from Tag.list(client)
    )
    print(doc.id, doc.title)

import os
import requests

workspace_id = 42  # replace with your workspace ID
headers = {"Authorization": f"Bearer {os.environ['LIGHTON_API_KEY']}"}

with open("q4-report.pdf", "rb") as f:
    response = requests.post(
        "https://api.lighton.ai/api/v3/files",
        headers=headers,
        data={
            "workspace_id": workspace_id,
            "title": "Q4 Financial Report",
            "tags": "[1, 2]",  # JSON-encoded list of tag IDs
        },
        files={"file": f},
    )
print(response.json())

If a tag ID is invalid, the file is still created but the response is a 207 (multi-status) with a message explaining which tags were rejected. To replace tags after upload, PATCH /api/v3/files/{id} with a new tags array. It replaces all existing tags, manual and auto-assigned. Send [0] (sentinel) to remove every tag when using multipart format. To add tags without touching existing ones, POST /api/v3/files/{id}/tags.

Tracking documents from external systems

If you’re ingesting documents from a third-party system (ServiceNow, Confluence, SharePoint, etc.), store the source identifier in external_metadata. This lets you find the LightOn file later given only the external ID, and surface the original URL in your UI.

upload_with_external_metadata.py

import json
import os
import requests

workspace_id = 42  # replace with your workspace ID
headers = {"Authorization": f"Bearer {os.environ['LIGHTON_API_KEY']}"}

with open("srv-456789.pdf", "rb") as f:
    response = requests.post(
        "https://api.lighton.ai/api/v3/files",
        headers=headers,
        data={
            "workspace_id": workspace_id,
            "external_metadata": json.dumps({
                "external_id": "SRV-456789",
                "doc_type": "incident",
                "additional_metadata": {
                    "external_url": "https://servicenow.example.com/incident/SRV-456789",
                },
            }),
        },
        files={"file": f},
    )
print(response.json())

The Python SDK doesn’t expose external_metadata on File yet, so this one is plain HTTP only.

external_id is required when creating; doc_type and additional_metadata are optional. When sent via multipart/form-data, the whole external_metadata value must be a JSON string. Retrieve it later by external ID:

GET /api/v3/files?external_metadata__external_id=SRV-456789

Listing and filtering your documents

GET /api/v3/files supports rich filtering. A few common patterns:

from lighton import File, LightOn

with LightOn() as client:
    # File.list passes filters through as query params and follows pagination
    File.list(client, workspace_id=42)
    File.list(client, search="security policy", search_details=True)
    File.list(client, tag_id=3, extension="pdf", ordering="-created_at")

    docs = File.list(client, total_pages_min=10, total_pages_max=50)
    for doc in docs:
        print(doc.id, doc.title, doc.total_pages)

import os
import requests

url = "https://api.lighton.ai/api/v3/files"
headers = {"Authorization": f"Bearer {os.environ['LIGHTON_API_KEY']}"}

# All files in a workspace
requests.get(url, headers=headers, params={"workspace_id": "42"})

# Semantic search across filenames and titles, with the top chunk inline
requests.get(url, headers=headers, params={"search": "security policy", "search_details": True})

# PDFs tagged 'legal', most recent first
requests.get(url, headers=headers, params={"tag_id": "3", "extension": "pdf", "ordering": "-created_at"})

# Files in a 10–50 page window
response = requests.get(url, headers=headers, params={"total_pages_min": 10, "total_pages_max": 50})
print(response.json())

Set include_details=true to receive the signature (TLSH hash for duplicate detection) and parser fields on each result. For advanced metadata filtering, see the Facets tutorials: classify files by type, set custom attributes, then filter with operators. See the filter reference for the full DSL.

Deleting files

Single delete:

from lighton import File, LightOn

file_id = 1234  # replace with your file ID

with LightOn() as client:
    File.get(client, file_id).delete()
    print(f"deleted file {file_id}")

import os
import requests

file_id = 1234  # replace with your file ID
headers = {"Authorization": f"Bearer {os.environ['LIGHTON_API_KEY']}"}

response = requests.delete(
    f"https://api.lighton.ai/api/v3/files/{file_id}",
    headers=headers,
)
print(response.status_code)

Bulk delete:

bulk_delete_files.py

import os
import requests

headers = {"Authorization": f"Bearer {os.environ['LIGHTON_API_KEY']}"}

response = requests.post(
    "https://api.lighton.ai/api/v3/files/bulk-delete",
    headers=headers,
    json={"ids": [123, 124, 125]},
)
print(response.json())

The Python SDK has no bulk-delete helper yet: call File.get(client, id).delete() per file, or use the endpoint above.

Both return 204 No Content on success. Files in synced (datasource-managed) workspaces cannot be deleted manually. The API returns 400.

Common errors

Status	Cause
`400`	Validation error, unsupported file type, or synced-workspace constraint
`401`	Missing or invalid API key
`403`	Permission denied (no upload/delete rights)
`404`	File does not exist or is not accessible
`429`	Too many concurrent uploads for this session

Overview

Build a searchable knowledge base

Classify and organise documents

Process documents on the fly

Upload a file

Wait for indexing to complete

Organising documents with tags and titles

Tracking documents from external systems

Listing and filtering your documents

Deleting files

Common errors

​Upload a file

​Wait for indexing to complete

​Organising documents with tags and titles

​Tracking documents from external systems

​Listing and filtering your documents

​Deleting files

​Common errors

Upload a file

Wait for indexing to complete

Organising documents with tags and titles

Tracking documents from external systems

Listing and filtering your documents

Deleting files

Common errors