Connecting RAG to Drive, Notion and SharePoint

A RAG source connector authenticates to an external system, pulls documents with stable IDs and permission metadata, and feeds them into ingestion so the index tracks the source without manual file copies. The three jobs are scoped auth, ACL fidelity, and sync under rate limits — parse and chunk come after. This page covers SharePoint, Notion, and Google Drive; the stage map after connect lives on document ingestion. Missing connectors are a common reason enterprise pilots never leave the demo corpus (why RAG pilots fail in enterprises).

How do you connect SharePoint to a RAG pipeline?

You connect SharePoint to a RAG pipeline by authenticating an Entra (Azure AD) app to Microsoft Graph, scoping it with Sites.Selected (or narrower), enumerating site / drive / items, downloading files, and attaching source metadata — then handing the bytes to a parser.

Azure GPT-RAG’s SharePoint howto (azure.github.io/GPT-RAG, live 2026-07-27) is explicit: Sites.Selected grants access to zero sites until each site is explicitly permitted via Graph or PowerShell, and CRON jobs schedule index and purge. Pathway’s SharePoint template uses certificate-based auth and a refresh_interval (their sample app.yaml shows 30 seconds for streaming mode). Ailog’s SharePoint guide walks the same Graph path: site → drive → items → download. After the pull, Office and PDF structure belong on PDF parsing — the connector’s job ends when the file and its IDs are in the ingest hand-off. SharePoint sprawl without a connector is the organisational failure pattern at enterprise RAG failure.

How do you connect Notion to a RAG pipeline?

You connect Notion to a RAG pipeline by creating a Notion integration token, sharing only the pages and databases the bot should see, then crawling pages via search and blocks and formatting them as RAG documents with page id, last_edited, and a content hash.

Ailog’s Notion + RAG guide (March 2026; live 2026-07-27) paginates search with page_size=100, filters incremental runs on last_edited_time, and stores content_hash in document metadata. The integration inherits only pages and databases explicitly shared with it — that visibility boundary is the ACL for most Notion connectors. Granular per-user ACL beyond workspace sharing is a product design choice, not something the public API invents for you. Field catalogue depth for those metadata keys lives on metadata extraction.

How do you connect Google Drive to a RAG pipeline?

You connect Google Drive to a RAG pipeline by creating a GCP service account, enabling the Drive API, sharing target folders with the service-account email as Viewer, then loading by folder id through a reader.

LlamaIndex’s live Google Drive ingestion example (developers.llamaindex.ai, live 2026-07-27) uses a service-account credentials.json and DocstoreStrategy.UPSERTS so reruns re-embed only changed docs. CocoIndex’s Drive source (dev.to / CocoIndex, live 2026-07-27) takes service_account_credential_path plus root_folder_ids after you share the folder Viewer to the service-account email. Google Cloud’s RAG Engine docs document the same pattern for managed Drive ingestion: share the folder with the RAG data service account as Viewer. Upsert and change-detection write paths deepen at incremental indexing.

How do connectors keep the index in sync when files change?

Connectors keep the index in sync by detecting adds, updates, and deletes and handing only the changed set to the indexer — they do not own chunk or embed write-path depth.

SharePoint connectors use Graph drive delta with @odata.deltaLink (Ailog SyncManager; Velaris delta sync; Microsoft Learn counts delta-with-token at 1 resource unit). Notion connectors filter on last_edited or combine webhooks with a periodic full reconcile — Ailog’s worker example runs incremental sync every 5 minutes and a daily full pass. Drive connectors typically reload the shared folder and rely on docstore upserts (LlamaIndex UPSERTS). The three write paths — hash registry, delete prune, and re-embed only what moved — live on keeping a RAG index current. Skipping deletes is how you get answers from documents you already deleted.

How do you preserve permissions when connecting enterprise sources?

You preserve permissions by extracting who can read each document at pull time and storing that as filterable chunk metadata — otherwise retrieval returns documents the user cannot open in the source.

SharePoint: Graph item permissions (Azure GPT-RAG explores the permissions endpoint) plus Sites.Selected so the app itself cannot see ungranted sites. Notion: the integration’s shared-page set is the ACL boundary. Drive: folder share to the service account is the ingest ACL; per-user Drive ACLs still need metadata at query time if end users differ from the service account. Velaris’s on-prem SharePoint architecture (February 2026) resolves user groups and filters vector search by intersecting permission groups — the connector’s job is to put those group IDs on every chunk. Permission lag after a revoke is multi-tenant leakage; retrieve-time policy design is access control in RAG retrieval. The metadata fields themselves are catalogued under metadata extraction.

How do rate limits shape RAG connector design?

Rate limits shape RAG connector design because connectors must pace pulls and honor HTTP 429 Retry-After — a full-corpus scrape that ignores throttles stalls ingestion and still burns quota.

As of July 2026, Microsoft Learn’s SharePoint Online throttling guidance prices Graph operations in resource units: 1 for single-item query, delta-with-token, or file download; 2 for multi-item list/create/update/delete/upload; 5 for permission resource operations including $expand=permissions. Per-app per-tenant resource-unit caps scale with license count — for example 1,250 resource units per minute at 0–1,000 licenses (defaults may change; verify before you size a crawler). Notion’s developer request-limits page states an average of 3 requests per second per connection, plus a workspace-level cap; exceeding either returns HTTP 429 with Retry-After in decimal seconds. Google Drive uses project quotas in Google Cloud Console and the same 429 pattern. Pathway’s refresh_interval is a pacing knob, not a published quota. Azure GPT-RAG’s SharePoint ingestion source retries embeddings and search uploads on 429 and respects Retry-After.

The handler below is the minimum contract — wait the header, then retry. Pin your HTTP client; the logic is library-agnostic.

import time
import requests  # pin: requests==2.32.3 (or your stack's HTTP client)

def get_with_backoff(url: str, headers: dict, max_attempts: int = 8) -> requests.Response:
    """Pull once; on HTTP 429 sleep Retry-After seconds and retry."""
    for _ in range(max_attempts):
        r = requests.get(url, headers=headers, timeout=60)
        if r.status_code != 429:
            r.raise_for_status()
            return r
        wait = int(r.headers.get("Retry-After", "2"))
        time.sleep(wait)
    raise RuntimeError(f"still rate-limited after {max_attempts} attempts: {url}")

Should you build a custom connector or use a managed one?

Build a custom connector when you need control over ACL metadata shape and sync semantics; use a managed connector when the vendor already preserves permissions and delta sync for your sources.

The trade is control versus speed — same three jobs either way:

After documents land, they still go through the six pipeline stages — see how to build a RAG pipeline from scratch. Connected docs land in a vector store such as Weaviate, Pinecone, Qdrant, or Milvus; store choice is not the connector decision.