Skip to main content

​
Row lifecycle

Three step kinds cover the id-keyed lifecycle, all through POST /v1/namespaces/{ns}/query:
  • UpsertN — full-row replace keyed on the external id. Existing rows with that id are deleted, then the new row is inserted. Replace, not merge: a vectorless upsert of a vectored row drops the vector.
  • SetProps — overwrite specific keys on a matched stream (start the query with NWhere); a null value removes the key. id itself can’t be changed — that’s an UpsertN. Right for occasional attribute rewrites, wrong for per-query counters.
  • DeleteN — delete a matched stream (NWhere → DeleteN), or pass ids for direct external-id deletes.

​
Vector encodings

Per row, one of two wire encodings (sending both is a 400):
vector_b64 is base64 of the vector as a little-endian f32 array — ~4× smaller on the wire and decoded with a memcpy instead of JSON number parsing. Python encoder:
Both encodings produce bit-identical stored vectors and identical search results. Dimension is fixed by the first vectored write; a mismatch is a 400.

​
Bulk ingest: POST /v1/namespaces/{ns}/bulk-vectors

The high-throughput seed path — one binary frame per request (Content-Type: application/octet-stream): a small JSON header (dims, label, ids, shared props) followed by row-major L2-normalized fp16 values. Compared to JSON+base64, this avoids re-encoding multi-GB payloads several times over.

​
Restart-safe loading

Resumable loaders pass ?expected_generation=<G>&expected_rows=<N>. The frame is accepted only when the namespace is exactly at that state:
  • Same frame already committed → 200 with "replayed": true and the matching request_fingerprint — no duplicate rows.
  • Different payload at the same position → 409 write_conflict.
  • Successful conditional writes return request_fingerprint, rows_before, rows_after — checkpoint your batch only after receiving one of these receipts.
This makes “crash, restart, re-send” the entire resume story: no import jobs to babysit.

​
Compaction

Auto-compaction is geometric, so total compaction work stays linear in ingested bytes. Finish any bulk load with an explicit compaction — send "compact": true at the top level of the last write request:
The response reports "compacted": true/false. Compaction also builds the ANN index once the table is large enough; rows written after the index snapshot are brute-force merged into results, so freshness is never sacrificed.

​
Deduplicating a bulk source

Bulk frames append blindly by design — a source that carries duplicate ids lands them silently, and duplicates burn top-k slots. {"DedupN": {}} reconciles a store to the external-id uniqueness contract (keeps the last-written row per id), and {"DedupN": {"dry_run": true}} is the census-only audit any bulk load should run when its source can’t prove id uniqueness — a matching total row count cannot reveal duplicates hiding inside it.

​
Retry rules

Reads can always be retried after a transport failure. For writes, only conditional bulk frames are automatically retryable (identical frame + CAS params → original receipt or replayed: true). Never blindly replay an unconditional write after a lost response — reconcile state first.