Document processing lifecycle

Ingest or import the document

Documents enter through either inline text ingestion or file import. Inline ingestion accepts title, content, optional metadata, and an optional external document id. File import accepts a multipart upload and preserves source file details.

Queue background work

The API stores the document record and queues a processing job. The initial response is 202 Accepted, not a completed indexing result.

Materialize source content

For uploaded files, Radioso reads the stored source file and extracts normalized markdown or text content before chunking. Inline text documents skip this storage round-trip.

Chunk the content

The worker loads workspace ingestion settings and applies the configured chunking strategy. Fixed-window, semantic, and recursive text chunking all map to workspace-scoped settings.

Fixed-window chunking always uses fixed-size overlapping text windows. Semantic and recursive text chunking first handle Markdown or HTML tables with repeated headers, then use code-aware chunking for fenced code blocks and source-code documents when parser support is available. Code-aware chunking is attempted for common shell, C-family, CSS, Go, Java, JavaScript, Kotlin, PHP, Python, Ruby, Rust, Scala, SQL, Swift, TOML, TypeScript, XML, and YAML inputs. If parser support is unavailable, processing falls back to text chunking.

Embed and publish chunks

Each chunk gets search text from its base document metadata and an embedding. Chunks are then published for the specific document revision so stale work does not overwrite newer content.

Mark the document ready for retrieval

Once the worker publishes the chunks, the document becomes usable in document search and grounded chat. This happens without waiting for metadata extraction.

Extract metadata asynchronously when enabled

Metadata extraction is optional and disabled by default. When enabled by the workspace setting, a source override, or a reprocess override, the worker queues a separate, lower-priority job after the document is already searchable.

That enrich job makes one model call for the document. The call understands the document type and extracts supported structured tags such as event dates. Event dates are written to the document metadata and patched onto the overlapping chunks as dateFrom and dateTo. Article publication dates can be attached at document level.

Patching chunk metadata recomputes the stored date_from and date_to columns that temporal retrieval reads. No re-embedding is needed, because those columns are structured and not part of the vector. If validation or the provider call fails, the document stays searchable without extracted tags; a failed enrich job never changes the document’s ready status.