GCS Connectors

Bulk-import documents from Google Cloud Storage buckets into the ingestion pipeline via POST /v1/connectors/gcs/import.

GCS Bulk Import

POST /v1/connectors/gcs/import

Bucket name without gs:// prefix
Only import objects with this prefix

How GCS Import Works

1
Object Enumeration
The ingestion service lists all objects in the bucket matching the optional prefix filter.
2
Pub/Sub Queue
Each object is queued as an individual ingest job in Pub/Sub for parallel processing.
3
Document Processing
The processor chunks each document, generates embeddings via Vertex AI or OpenAI, and stores results in PostgreSQL + Vertex Vector Search.
✓
Ready for Query
Documents with job status SUCCEEDED are indexed and available for RAG queries.

Import History

This session

BucketPrefixTenantJobs QueuedTime
No imports yet