Cloudflare Web Search API for AI Agents: Grounding in Workers

Cloudflare Web Search API for AI Agents: Build a Grounded Worker

SitePoint TeamPublished inAI·APIs·Web·
October 8, 2026

The AI briefing for Developers

Stay up to date with AI tools, model releases, and developer workflows that matter.

Weekly. Free. One click to leave.

SitePoint Premium
Stay Relevant and Grow Your Career in Tech

• Premium Results
• Publish articles on SitePoint
• Daily curated jobs
• Learning Paths
• Discounts to dev tools

7 Day Free Trial. Cancel Anytime.

Cloudflare Web Search API: Build a Grounded AI Agent on Workers

How to Build a Grounded AI Agent with the Cloudflare Web Search API

1. Scaffold a new Cloudflare Worker project using the TypeScript template and install @cloudflare/workers-types.
2. Configure the AI binding, AI Gateway ID, search provider, and result limit in wrangler.jsonc.
3. Accept a user question via a POST endpoint with input validation and bearer-token authentication.
4. Fetch live web results by calling env.AI.websearch() with the query, provider, and gateway ID.
5. Construct a grounded prompt that injects numbered search results and instructs the LLM to cite sources.
6. Generate a cited answer by passing the prompt to Workers AI via env.AI.run().
7. Deploy the Worker with npx wrangler deploy and test with cURL.
8. Monitor request logs, cache hit rates, and latency in the AI Gateway dashboard.

This tutorial builds a Cloudflare Worker that accepts a user question, fetches live web results LLM prompt, and returns a cited answer. The result is a grounded AI agent running entirely on Cloudflare’s edge infrastructure

Note: The Web Search API is in open beta. The API surface, provider list and pricing can change before general availability, so check the docs before you deploy.

Why AI Agents Need Live Web Grounding

Large language models generate fluent, confident text, but that confidence regularly outpaces accuracy. When a model’s training data has a cutoff, every fact beyond that boundary is a guess. When a user asks for current pricing, breaking news, or recent API changes, the model either refuses to answer or, more dangerously, fabricates a plausible response complete with invented citations. This failure mode, commonly called hallucination, erodes user trust rapidly in production systems.

Grounding solves this. In the context of AI agents, grounding means augmenting an LLM prompt with real-time, authoritative data retrieved from external stale parametric knowledge. A real-time web search API for AI agents is the most direct path to grounding: fetch live results, inject them into the prompt, and instruct the model to cite its

A real-time web search API for AI agents is the most direct path to grounding: fetch live results, inject them into the prompt, and instruct the model to cite its

Prerequisites

• A Cloudflare account. Every account has an AI Gateway named default. Searches are billed from AI Gateway credits, or you can store your own provider API key on the gateway
• A current Node.js LTS release
• The Wrangler CLI installed (npm install -g wrangler or use via npx)
• @cloudflare/workers-types as a dev dependency (installed below)
• Basic familiarity with Workers and REST APIs
• Confirmed availability of the env.AI.websearch() binding in your account (check the Workers AI docs for current status)

What Is the Cloudflare Web Search API?

Open Beta Launch and Core Capabilities

Cloudflare’s Web Search API is available in open beta. It gives Workers developers a native binding for live web search without leaving the platform. Inside any Worker, developers call env.AI.websearch(), which routes the request through an AI Gateway and resolves to a standard Response. Call response.json() to read the results. The same search is also available over a REST endpoint.

• provider: one of the supported search providers (ceramic, exa or linkup; ceramic is the default).
• query: the search string, typically derived from the user’s question.
• limit: the maximum number of results to return (1 to 10, default 10).
• gatewayId: the AI Gateway identifier that enables logging, caching, and rate limiting for the request.

The response body is an object with an items array, where each item has url, title and description fields, plus a metadata object with the query, a request ID and latency. The shape is the same across providers, so the Worker code does not need provider-specific parsing. The query can be up to 1,024 characters.

Zero Data Retention and Verified Bot Crawling

Zero Data Retention depends on the provider you pick. According to Cloudflare’s provider documentation, Ceramic.ai and Linkup offer ZDR and Exa does not. If your queries can contain personal or proprietary data, choose a ZDR provider and confirm the current status there before you rely on it for compliance.

Cloudflare also says every provider commits to its verified bot requirements and returns a link to the

For enterprise and privacy-conscious deployments, these guarantees reduce the work of auditing third-party data retention policies and provide a defensible posture when handling user queries that contain personally identifiable information or proprietary data.

Comparing Your Options: AI Gateway vs. Direct Providers vs. Aggregators

The Three Approaches to Web Search for AI Agents

There are three broad strategies for adding web search to an AI agent built on Cloudflare Workers.

The Cloudflare AI Gateway path uses the Web Search API binding (env.AI.websearch()). It abstracts provider details behind a single interface, routes requests through AI Gateway for built-in logging, analytics and billing, and lets you pick a provider with one parameter. ZDR depends on the provider.

Direct provider APIs involve calling provider REST endpoints directly. This gives full access to provider-specific features, advanced query syntax, and custom retry logic, but the developer manages API keys, error handling, and compliance independently

Third-party aggregators such as SerpAPI, Tavily, or Brave Search API offer broad coverage across search engines. However, they introduce an additional vendor dependency, have variable data retention policies, and lack native Workers integration.

Comparison Table

For most Workers-based AI agents, the AI Gateway path is the simplest starting point. It handles the undifferentiated work of key management, retry logic, and compliance. Direct APIs make sense when a project demands provider-specific advanced features that are not exposed through the abstraction layer.

Project Setup and Configuration

Scaffold the Worker

Start by creating a new Worker project using the TypeScript template:

npm create cloudflare@latest grounded-agent -- --template worker-typescript

Template output may differ across create-cloudflare versions. Run npm create cloudflare@latest --version to confirm yours. This tutorial assumes the generated project includes src/index.ts, wrangler.jsonc, and package.json.

Install the required type definitions:

npminstall --save-dev @cloudflare/workers-types

Add the Workers types to your tsconfig.json under compilerOptions:

// tsconfig.json (relevant excerpt){"compilerOptions": {"target": "ESNext","module": "ESNext","moduleResolution": "Bundler","types": ["@cloudflare/workers-types"],"strict": true}}

Next, configure the Worker to use the AI binding and set the environment variables for the search provider:

// wrangler.jsonc{"name": "grounded-agent","main": "src/index.ts","compatibility_date": "2026-10-08","ai": {"binding": "AI"},"vars": {"AI_GATEWAY_ID": "default","SEARCH_PROVIDER": "ceramic","SEARCH_RESULT_LIMIT": "5"}}

The ai block creates the env.AI binding, which exposes both env.AI.run() for LLM inference and env.AI.websearch() for search. The three environment variables keep provider configuration outside the code, making it triing logic changes

Important: The vars block in wrangler.jsonc is plaintext and typically committed to version control. If your search provider requires an API key, store it as a secret instead: npx wrangler secret put PROVIDER_API_KEY. Never put secret values in the vars block. Additionally, store your Worker’s authentication token as a secret: npx wrangler secret put API_SECRET_TOKEN.

Enable AI Gateway in the Dashboard

To get the gateway ID, navigate to the Cloudflare dashboard, open the AI Gateway section, and create a new gateway. Name it something descriptive like grounded-agent-gateway and copy the generated gateway ID into the AI_GATEWAY_ID variable in wrangler.jsonc.

The gatewayId parameter is required. It tells Cloudflare which AI Gateway to route the search through, which is what gives you logs and analytics for search requests. Every account has a gateway named default, so "AI_GATEWAY_ID": "default" works out of the box. To get separate logs, create a gateway in the dashboard (for example grounded-agent-gateway) and put its ID in wrangler.jsonc.

Building the Grounded Worker Step by Step

Step 1: Accept the User Query

The Worker listens for POST requests containing a JSON body with a question field. Input validation catches missing or empty queries early and returns a clear 400 error. An authentication check ensures only authorized callers can invoke the Worker.

exportinterfaceEnv{AI: Ai;AI_GATEWAY_ID:string;SEARCH_PROVIDER:string;SEARCH_RESULT_LIMIT:string;API_SECRET_TOKEN:string;}exportdefault{asyncfetch(request: Request, env: Env):Promise<Response>{if(request.method !=="POST"){returnnewResponse(JSON.stringify({ error:"Method not allowed"}),{status:405,headers:{"Content-Type":"application/json"},});}const authHeader = request.headers.get("Authorization")??"";if(!env.API_SECRET_TOKEN|| authHeader !==`Bearer${env.API_SECRET_TOKEN}`){returnnewResponse(JSON.stringify({ error:"Unauthorized"}),{status:401,headers:{"Content-Type":"application/json"},});}const contentType = request.headers.get("Content-Type")??"";if(!contentType.includes("application/json")){returnnewResponse(JSON.stringify({ error:"Content-Type must be application/json"}),{ status:415, headers:{"Content-Type":"application/json"}});}let body:{ question?:string};try{body =await request.json();}catch{returnnewResponse(JSON.stringify({ error:"Invalid JSON body"}),{status:400,headers:{"Content-Type":"application/json"},});}const question = body.question?.trim();if(!question || question.length >1024){returnnewResponse(JSON.stringify({ error:"Missing or oversized 'question' field (max 1024 chars)"}),{ status:400, headers:{"Content-Type":"application/json"}});}},};

Step 2: Fetch Live Web Results with env.AI.websearch()

The websearch call takes the user’s question as the query, routes it through the configured provider, and resolves to a Response. A helper function checks the status, parses the JSON, validates that items is an array, maps the results into a numbered context string for the prompt (with description and total length caps to stay within the model’s context window), and keeps the items for citation metadata. A small timeout wrapper prevents a slow provider from hanging the request.

interfaceSearchItem{url:string;title:string;description?:string;}interfaceSearchResponse{items: SearchItem[];metadata?:{ query?:string; requestId?:string; latencyMs?:number};}constMAX_DESCRIPTION_CHARS=300;constMAX_CONTEXT_CHARS=4000;functionwithTimeout<T>(promise:Promise<T>, ms:number):Promise<T>{let timer: ReturnType<typeof setTimeout>;const timeout =newPromise<never>((_, reject)=>{timer =setTimeout(()=>reject(newError(`Timed out after${ms}ms`)), ms);});returnPromise.race([promise, timeout]).finally(()=>clearTimeout(timer));}asyncfunctionfetchSearchResults(env: Env,question:string,limit:number):Promise<{ context:string; sources: SearchItem[]}>{const response =awaitwithTimeout(env.AI.websearch({provider: env.SEARCH_PROVIDER,query: question,limit,gatewayId: env.AI_GATEWAY_ID,}),8000);if(!response.ok){thrownewError(`websearch failed with status${response.status}`);}const data =(await response.json())as SearchResponse;if(!Array.isArray(data.items)){thrownewError("websearch response is missing the items array");}const sources = data.items;const context = sources.map((r, i)=>`[${i +1}]${r.title}-${r.url}:${(r.description ??"").slice(0,MAX_DESCRIPTION_CHARS)}`).join("n").slice(0,MAX_CONTEXT_CHARS);return{ context, sources };}

Each result is numbered sequentially. The [n] Title - URL: description format gives the LLM clear, attributable blocks to reference when composing its answer.

Step 3: Construct the Grounded Prompt

The system prompt is critical. It constrains the model to answer strictly from the provided search results and use bracketed citation notation. Without explicit instructions, models tend to blend search context with parametric knowledge, undermining the grounding guarantee.

functionbuildPrompt(question:string,context:string):{ systemPrompt:string; userMessage:string}{const systemPrompt ="You are a research assistant. Answer the user's question using ONLY the search results provided below. Cite sources using [n] notation. If the results are insufficient, say so.";const userMessage =`Question:${question}Search Results:${context}`;return{ systemPrompt, userMessage };}

The system message is terse by design. Lengthy system prompts consume tokens without proportionally improving citation behavior. The search results are placed in the user message rather than the system message so the system prompt stays focused on behavioral constraints and the

Step 4: Call Workers AI for the LLM Response

With the prompt assembled, the Worker calls env.AI.run() with a supported Workers AI model. The model processes the grounded prompt and returns a response that should reference the numbered

For this model the chat completion response has the shape { response: string }. Other model families can return a different shape, so check the model page under developers.cloudflare.com/workers-ai/models/ and confirm the model ID is not deprecated. The Llama 3.1 70B model that older tutorials use was deprecated on May 30, 2026.

asyncfunctiongenerateAnswer(env: Env,systemPrompt:string,userMessage:string):Promise<string>{const llmResponse =await env.AI.run("@cf/meta/llama-3.3-70b-instruct-fp8-fast",{messages:[{ role:"system", content: systemPrompt },{ role:"user", content: userMessage },],});const answer =(llmResponse as{ response?:string})?.response;if(!answer){thrownewError("Unexpected LLM response shape");}return answer;}

Step 5: Return the Cited Response

The final response bundles the LLM’s answer with thethe complete Worker file combining all steps:

exportinterfaceEnv{AI: Ai;AI_GATEWAY_ID:string;SEARCH_PROVIDER:string;SEARCH_RESULT_LIMIT:string;API_SECRET_TOKEN:string;}interfaceSearchItem{url:string;title:string;description?:string;}interfaceSearchResponse{items: SearchItem[];metadata?:{ query?:string; requestId?:string; latencyMs?:number};}constMAX_DESCRIPTION_CHARS=300;constMAX_CONTEXT_CHARS=4000;functionwithTimeout<T>(promise:Promise<T>, ms:number):Promise<T>{let timer: ReturnType<typeof setTimeout>;const timeout =newPromise<never>((_, reject)=>{timer =setTimeout(()=>reject(newError(`Timed out after${ms}ms`)), ms);});returnPromise.race([promise, timeout]).finally(()=>clearTimeout(timer));}asyncfunctionfetchSearchResults(env: Env,question:string,limit:number):Promise<{ context:string; sources: SearchItem[]}>{const response =awaitwithTimeout(env.AI.websearch({provider: env.SEARCH_PROVIDER,query: question,limit,gatewayId: env.AI_GATEWAY_ID,}),8000);if(!response.ok){thrownewError(`websearch failed with status${response.status}`);}const data =(await response.json())as SearchResponse;if(!Array.isArray(data.items)){thrownewError("websearch response is missing the items array");}const sources = data.items;const context = sources.map((r, i)=>`[${i +1}]${r.title}-${r.url}:${(r.description ??"").slice(0,MAX_DESCRIPTION_CHARS)}`).join("n").slice(0,MAX_CONTEXT_CHARS);return{ context, sources };}functionbuildPrompt(question:string,context:string):{ systemPrompt:string; userMessage:string}{const systemPrompt ="You are a research assistant. Answer the user's question using ONLY the search results provided below. Cite sources using [n] notation. If the results are insufficient, say so.";const userMessage =`Question:${question}Search Results:${context}`;return{ systemPrompt, userMessage };}asyncfunctiongenerateAnswer(env: Env,systemPrompt:string,userMessage:string):Promise<string>{const llmResponse =await env.AI.run("@cf/meta/llama-3.3-70b-instruct-fp8-fast",{messages:[{ role:"system", content: systemPrompt },{ role:"user", content: userMessage },],});const answer =(llmResponse as{ response?:string})?.response;if(!answer){thrownewError("Unexpected LLM response shape");}return answer;}exportdefault{asyncfetch(request: Request, env: Env):Promise<Response>{if(request.method !=="POST"){returnnewResponse(JSON.stringify({ error:"Method not allowed"}),{status:405,headers:{"Content-Type":"application/json"},});}const authHeader = request.headers.get("Authorization")??"";if(!env.API_SECRET_TOKEN|| authHeader !==`Bearer${env.API_SECRET_TOKEN}`){returnnewResponse(JSON.stringify({ error:"Unauthorized"}),{status:401,headers:{"Content-Type":"application/json"},});}const contentType = request.headers.get("Content-Type")??"";if(!contentType.includes("application/json")){returnnewResponse(JSON.stringify({ error:"Content-Type must be application/json"}),{ status:415, headers:{"Content-Type":"application/json"}});}let body:{ question?:string};try{body =await request.json();}catch{returnnewResponse(JSON.stringify({ error:"Invalid JSON body"}),{status:400,headers:{"Content-Type":"application/json"},});}const question = body.question?.trim();if(!question || question.length >1024){returnnewResponse(JSON.stringify({ error:"Missing or oversized 'question' field (max 1024 chars)"}),{ status:400, headers:{"Content-Type":"application/json"}});}const rawLimit =parseInt(env.SEARCH_RESULT_LIMIT,10);if(isNaN(rawLimit)|| rawLimit <1|| rawLimit >10){returnnewResponse(JSON.stringify({ error:"Worker misconfiguration: invalid SEARCH_RESULT_LIMIT"}),{ status:500, headers:{"Content-Type":"application/json"}});}try{const{ context, sources }=awaitfetchSearchResults(env, question, rawLimit);const{ systemPrompt, userMessage }=buildPrompt(question, context);const answer =awaitgenerateAnswer(env, systemPrompt, userMessage);const response ={answer,sources: sources.map((s, i)=>({index: i +1,title: s.title,url: s.url,})),};returnnewResponse(JSON.stringify(response),{headers:{"Content-Type":"application/json"},});}catch(err){console.error("[grounded-agent] handler error:", err);returnnewResponse(JSON.stringify({ error:"Search or inference failed. Check server logs."}),{ status:502, headers:{"Content-Type":"application/json"}});}},};

The complete Worker file (Step 5) totals approximately 115 lines of TypeScript. Before deploying, ensure @cloudflare/workers-types is installed, confirm the tsconfig.json includes the Workers types, store your API_SECRET_TOKEN as a wrangler secret (npx wrangler secret put API_SECRET_TOKEN), and confirm the Web Search API binding is available in your account.

Deploy, Test, and Observe

Deploy with Wrangler

Deploy the Worker to Cloudflare’s edge network:

npx wrangler deploy

Wrangler outputs the Worker URL upon successful deployment. Note this URL for testing.

Test with cURL

Send a test question to the deployed Worker:

curl-X POST https://grounded-agent.<your-subdomain>.workers.dev -H"Content-Type: application/json"-H"Authorization: Bearer YOUR_API_SECRET_TOKEN"-d'{"question": "What are the latest changes to Cloudflare Workers?"}'

An example response (the answer text will differ with your provider and the live results):

{"answer":"Cloudflare's Web Search API is in open beta. You call env.AI.websearch() from a Worker, it routes the request through AI Gateway, and it returns titles, URLs and descriptions you can pass to a model [1][2].","sources":[{"index":1,"title":"Web Search API overview","url":"https://developers.cloudflare.com/web-search/"},{"index":2,"title":"How to use Web Search API","url":"https://developers.cloudflare.com/web-search/how-to-use/"}]}

Monitor in AI Gateway

In the Cloudflare dashboard, navigate to AI Gateway and select the gateway created earlier. The dashboard displays request logs showing each websearch call and LLM inference request, cache hit rates for repeated queries, and per-provider latency breakdowns. Look for requests with high latency or error status codes; these indicate provider issues or query timeouts that need attention.

Production Considerations

Choosing the Right Provider

Each provider has different strengths. Check the Web Search API documentation for the current provider list, which documents each provider’s focus area (broad web coverage, semantic/neural search, structured data extraction, and so on).

Because the provider parameter is a simple string, A/B testing across providers requires no code changes. Set it as an environment variable, deploy variants, and compare result quality and latency in the AI Gateway dashboard.

Caching, Rate Limiting, and Cost Control

How do you balance freshness against cost? AI Gateway can cache responses and apply rate limits, both configured per gateway in the dashboard. Caching reduces cost and latency for repeated queries, but trades away freshness. Pick a TTL that matches how fast your topic changes, and disable caching for breaking-news use cases.

Rate limiting in AI Gateway prevents runaway costs during traffic spikes. Set per-minute or per-hour limits based on expected traffic and provider pricing tiers. Monitor per-provider costs in the dashboard to catch unexpected billing increases early.

Handling Failures Gracefully

If a provider goes down, the websearch call will fail. The Step 5 code above wraps the core logic in a try/catch block that returns a structured error response without leaking internal details. For production Workers, consider extending this with fallback logic: catch the error, attempt a second provider, and if all providers fail, return a degraded response that acknowledges the inability to fetch live data rather than silently falling back to ungrounded LLM output.

The fetchSearchResults function wraps the call in an 8-second timeout. The websearch() request type has no signal option, so the timeout stops the Worker from waiting; it does not cancel the upstream request.

What Grounded AI Agents Unlock

This tutorial produced a Cloudflare Worker that grounds an LLM in live web data and returns answers with verifiablefferentiated work of managing search provider integrations, rotating API keys, and auditing data retention compliance

Deploy the Worker, swap the SEARCH_PROVIDER value between available providers, and iterate on the system prompt to improve citation precision. Grounding is not something to bolt on after launch. Without it, users get answers they cannot verify, and unverifiable answers train users to distrust the agent entirely.

Sharing our passion for building incredible internet things.