Google Cloud Storage

A Google Cloud Storage source keeps a bucket, or prefixes of it, in one namespace. Each file is read as an upload is, and a file deleted from the bucket loses its documents.

Grant access

Conare reads buckets as one Google service account, so no key changes hands. Grant it Storage Object Viewer and create the empty object .conare/<organization id>, which shows the bucket is yours:

gcloud

gcloud storage buckets add-iam-policy-binding gs://acme-reports \
  --member=serviceAccount:conare-ingest@strong-ward-488314-c4.iam.gserviceaccount.com \
  --role=roles/storage.objectViewer
gcloud storage cp /dev/null gs://acme-reports/.conare/ORG_ID

Preview the bucket

curl

curl https://api.conare.ai/sources/gcs/preview \
  -H "Authorization: Bearer $CONARE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"bucket": "acme-reports", "prefixes": ["2024/"]}'

Connect it

curl

curl https://api.conare.ai/sources \
  -H "Authorization: Bearer $CONARE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"type": "gcs", "namespace": "reports", "bucket": "acme-reports", "prefixes": ["2024/"]}'

The account

GET /sources/gcs

Response

{
  "configured": true,
  "role": "roles/storage.objectViewer",
  "service_account": "conare-ingest@strong-ward-488314-c4.iam.gserviceaccount.com"
}

service_account, role: the account, and the role to grant it on your bucket.
A source stops syncing when .conare/<organization id> is removed; gcs_not_verified names the object to create.

Full schema: The account to grant a bucket to.

Preview a bucket

POST /sources/gcs/preview Lists what a source of the bucket would read; nothing is stored.

Response

{
  "bucket": "acme-reports",
  "bytes": 74,
  "complete": true,
  "objects": 2,
  "prefixes": ["2024/"],
  "sample": ["2024/prices.csv", "2024/site-7.txt"],
  "service_account": "conare-ingest@strong-ward-488314-c4.iam.gserviceaccount.com",
  "skipped": 0,
  "too_large": 0,
  "types": { "csv": 1, "txt": 1 },
  "unsupported": 0
}

Option	Type	What it does
`bucket`	string, required	The bucket’s name
`prefixes`	array	Up to 20 prefixes, each up to 1,024 characters; none reads the whole bucket

Field	Meaning
`objects`, `bytes`	Files a source would read, and their size
`types`	Files by type; `other` for those only their bytes will tell
`skipped`, `unsupported`, `too_large`	Folders and empty objects; files Conare doesn’t read; files over 512 MiB
`sample`	Up to 20 file names
`complete`	`false` when the bucket held over 10,000 objects or took over 10 seconds to list

Full schema: Preview a bucket.

Connect a bucket

Send "type": "gcs" to POST /sources; the first sync starts at once, and a key scoped to the namespace can connect it.

Response

{
  "id": "src_4e4abfd7ac248dce3668ef76",
  "kind": "gcs",
  "type": "gcs",
  "namespace": "reports",
  "user": null,
  "name": "gs://acme-reports/2024/",
  "status": "active",
  "health": "ok",
  "display": { "state": "ok" },
  "sync": {
    "state": "queued",
    "run_id": null,
    "phase": null,
    "last_success_at": null,
    "last_failure_at": null,
    "next_run_at": "2026-10-21T22:41:44.768045Z",
    "last_error": null
  },
  "documents": 0,
  "documents_verified_at": null,
  "config": {
    "bucket": "acme-reports",
    "prefixes": ["2024/"],
    "service_account": "conare-ingest@strong-ward-488314-c4.iam.gserviceaccount.com",
    "sync_interval_s": 3600
  },
  "failures": 0,
  "created_at": "2026-10-21T21:41:44.768045Z",
  "updated_at": "2026-10-21T21:41:44.768045Z"
}

Option	Type	What it does
`type`	string, required	`gcs`
`namespace`	string, required	The namespace to keep the files in
`bucket`	string, required	The bucket’s name
`prefixes`	array	Up to 20 prefixes, each up to 1,024 characters
`name`	string	A display name, up to 200 characters; defaults to `gs://bucket/prefix`
`sync_interval_s`	integer, default 3600	Seconds between syncs, 900 to 604,800 (7 days)

Field	Meaning
`health`	`ok`; `degraded` when the last sync held back its deletions; `failing` when the account lost its grant or the last 3 syncs failed
`sync`	`state` (`idle`, `queued` or `running`), `last_success_at`, `last_failure_at`, `next_run_at`, `last_error`
`documents`, `failures`	Documents it keeps, and files it couldn’t keep (below)
`config`	The bucket, prefixes, account and interval

Sync and disconnect

Action	Route	Answer
Sync now	`POST /sources/:id/sync`	`202` with the sync’s `run`, or `409 run_active` while one is queued or running
Disconnect	`DELETE /sources/:id`	`202` with the `run` that deletes its documents; the source is `deleting`, then `deleted`

Each sync lists the bucket and reads new and changed files.
A sync that would delete over 1,000 documents and a tenth of the source waits 24 hours for a second sync to confirm it.

Documents

Each file is one document, with an upload’s id and locations, plus source_url (gs://bucket/name), bucket, object and generation in their metadata.

{ "source_url": "gs://acme-reports/2024/prices.csv" }

GET /sources/:id/documents lists its files.

Failures

GET /sources/:id/failures

curl

curl https://api.conare.ai/sources/src_b805954930e968fe8d4a89f6/failures \
  -H "Authorization: Bearer $CONARE_API_KEY"

Response

{
  "data": [
    {
      "external_id": "gs://acme-reports/2024/minutes.pdf",
      "key": "minutes.pdf-69578779b32368c6",
      "version": "md5:0HKRGpcwGcvjCA5k1+5AFA==;type=pdf",
      "code": "type_mismatch",
      "message": "type_mismatch: the file is not the type declared: declared pdf, but the bytes are text",
      "run_id": "run_1a20d9cdd1b2092406b048ae",
      "attempts": 1,
      "failed_at": "2026-10-21T21:55:33.56975Z"
    }
  ],
  "next_cursor": null
}

Option	Type	What it does
`limit`	integer, default 1000	Failures per page, up to 1,000
`cursor`	string	The previous page’s `next_cursor`, `null` on the last page

`code`	Meaning
`unsupported_file`	Conare doesn’t read the file’s type
`too_large`	Over 512 MiB
`pages_unread`	OCR couldn’t read the pages the message names; the rest is searchable
Others	The parse’s error, such as `type_mismatch` or `unreadable_file`

A failure never stops a sync.
A file that failed for good is read again only when it changes.
One that failed for a passing reason is retried at the next 3 syncs, then daily.

Full schema: List the objects a source could not keep.

Errors

Error	When
`422 gcs_access_denied`	The account can’t list the bucket; the message names the account and role
`422 gcs_not_verified`	The bucket has no `.conare/<organization id>`; the message names the object
`422 gcs_bucket_not_found`	No bucket has that name
`503 gcs_auth_failed`, `503 gcs_unavailable`	Google Cloud Storage couldn’t be reached; retry after `Retry-After`

Get started

Core

Files

Sync

Models

Tools

Google Cloud Storage

The account

Preview a bucket

Connect a bucket

Sync and disconnect

Documents

Failures

Errors

​The account

​Preview a bucket

​Connect a bucket

​Sync and disconnect

​Documents

​Failures

​Errors

The account

Preview a bucket

Connect a bucket

Sync and disconnect

Documents

Failures

Errors