b/cited
← Glossary
[ Term ]

Ingestion

The seven-step background workflow that pulls a project's last 90 days of Google Search Console data, embeds and clusters the queries, scores authority, and persists everything for the dashboard.

Also known as:SyncIngest Pipeline

Ingestion is the durable Cloudflare Workflow that runs every time a project syncs — on initial onboarding, when you hit Re-sync, and on the nightly cron.

It's the only path that writes ranking data into b/cited. Every dashboard surface (clusters, authority, briefs, quick wins) reads from what ingestion produced.

The seven steps

  1. Claim — atomic project status transition from ready to ingesting, prevents double-runs.
  2. Fetch GSC — pulls the last 90 days of query-level data from Google Search Console via the read-only webmasters.readonly OAuth scope.
  3. Persist queries — batched D1 inserts; old vectors are purged before the new batch lands.
  4. Embed — OpenAI batch calls (parallelized 8-wide via AI Gateway) turn each query into a 1536-dim vector.
  5. Cluster + label — centroid-greedy clustering over Vectorize, then GPT-4.1-mini names each cluster.
  6. URL pass — second GSC pull at query × page granularity, used for ownership-status scoring.
  7. Authority — per-cluster weighted blend (ranking share, position strength, impression coverage, URL consolidation) produces the 0–100 authority score.

Total wall-clock is usually 3-5 minutes per project depending on query volume. Workflow steps are durable: a failure at step 5 retries from step 5, not from the top.

[ Related ]