[AI Search] Namespace binding docs, API restructure, and managed storage (#29601)

* ai search new bindings

* Update src/content/docs/ai-search/usage/rest-api.mdx

Co-authored-by: Pedro Sousa <680496+pedrosousa@users.noreply.github.com>

* Update src/content/docs/ai-search/usage/workers-binding.mdx

Co-authored-by: Pedro Sousa <680496+pedrosousa@users.noreply.github.com>

* update binding

* [AI Search] Add namespace binding docs, restructure API section, and update overview

- Add ai_search_namespaces binding documentation with full API reference
- Restructure docs under api/ section: Search, Instances, Items, Migration
- Add namespace concept page under concepts/
- Add Items API documentation (upload, uploadAndPoll, list, delete, get)
- Add how-to examples: multi-language search, knowledge base, per-tenant search
- Update overview page with new positioning and feature cards
- Update get-started pages for managed storage (no R2 prerequisite)
- Update service API token page (only required for R2 connections)
- Update wrangler configuration reference with ai_search_namespaces
- Update all cross-references and add redirects for moved pages
- Update code examples across config and how-to pages to use namespace binding

* namespace updates

* updates to API section

* search api docs

* fix migration pages

* update data source

* all other docs pages

* new sidebar sections

* hybrid search concept

* fix links

* update changelog, release note and overview

* update note

* PCX review

* Fixing Wrangler commands for AI Search docs

* Deleting unnecessary file

* Fixing broken links

* Adding horizontal lines to make Workers Bindings API clearer

* final changes

* Fix links

* Fix Browser Rendering occurrences

* final small fixes

* changelog updates

---------

Co-authored-by: Pedro Sousa <680496+pedrosousa@users.noreply.github.com>
Co-authored-by: Jun Lee <junlee@cloudflare.com>

Author: 3 people

package-lock.json

@@ -273,10 +273,40 @@
 /api-shield/security/sequence-mitigation/configure/ /api-shield/security/sequence-mitigation/api/ 301
 /api-shield/frequently-asked-questions/ /api-shield/ 301
 /autorag/usage/recipes/ /ai-search/how-to/ 301
-/autorag/configuration/metadata-filtering/ /ai-search/configuration/metadata/ 301
+/autorag/configuration/metadata-filtering/ /ai-search/configuration/retrieval/metadata/ 301
 /ai-search/tutorial/ /ai-search/how-to/ 301
-/ai-search/tutorial/brower-rendering-autorag-tutorial/ /ai-search/how-to/brower-rendering-autorag-tutorial/ 301
+/ai-search/tutorial/brower-rendering-autorag-tutorial/ /ai-search/how-to/ 301
 /ai-search/autorag-api/ /api/resources/ai_search/ 301
+/ai-search/usage/workers-binding/ /ai-search/api/search/workers-binding/ 301
+/ai-search/usage/workers-binding-legacy/ /ai-search/api/migration/workers-binding-legacy/ 301
+/ai-search/api/workers-binding-legacy/ /ai-search/api/migration/workers-binding-legacy/ 301
+/ai-search/usage/rest-api/ /ai-search/api/search/rest-api/ 301
+/ai-search/usage/public-endpoint/ /ai-search/api/search/public-endpoint/ 301
+/ai-search/usage/mcp/ /ai-search/api/search/mcp/ 301
+/ai-search/usage/ /ai-search/api/search/ 301
+/ai-search/instances-api/workers-binding/ /ai-search/api/instances/workers-binding/ 301
+/ai-search/instances-api/rest-api/ /ai-search/api/instances/rest-api/ 301
+/ai-search/instances-api/ /ai-search/api/instances/ 301
+/ai-search/how-to/migrate-from-autorag-api/ /ai-search/api/migration/rest-api/ 301
+/ai-search/autorag-filter-format/ /ai-search/api/migration/autorag-filter-format/ 301
+/ai-search/how-to/multitenancy/ /ai-search/how-to/per-tenant-search/ 301
+/ai-search/configuration/path-filtering/ /ai-search/configuration/indexing/path-filtering/ 301
+/ai-search/configuration/chunking/ /ai-search/configuration/indexing/chunking/ 301
+/ai-search/configuration/indexing/ /ai-search/configuration/indexing/syncing/ 301
+/ai-search/configuration/keyword-search/ /ai-search/configuration/indexing/keyword-search/ 301
+/ai-search/configuration/service-api-token/ /ai-search/configuration/indexing/service-api-token/ 301
+/ai-search/configuration/retrieval-configuration/ /ai-search/configuration/retrieval/result-filtering/ 301
+/ai-search/configuration/retrieval/retrieval-configuration/ /ai-search/configuration/retrieval/result-filtering/ 301
+/ai-search/configuration/hybrid-search/ /ai-search/configuration/indexing/hybrid-search/ 301
+/ai-search/configuration/metadata/ /ai-search/configuration/indexing/metadata/ 301
+/ai-search/configuration/retrieval/metadata/ /ai-search/configuration/indexing/metadata/ 301
+/ai-search/configuration/boosting/ /ai-search/configuration/retrieval/boosting/ 301
+/ai-search/configuration/reranking/ /ai-search/configuration/retrieval/reranking/ 301
+/ai-search/configuration/query-rewriting/ /ai-search/configuration/retrieval/query-rewriting/ 301
+/ai-search/configuration/system-prompt/ /ai-search/configuration/retrieval/system-prompt/ 301
+/ai-search/configuration/cache/ /ai-search/configuration/retrieval/cache/ 301
+/ai-search/configuration/public-endpoint/ /ai-search/configuration/retrieval/public-endpoint/ 301
+/ai-search/configuration/embed-search-snippets/ /ai-search/configuration/retrieval/embed-search-snippets/ 301
 
 # bots
 /bots/about/plans/ /bots/plans/ 301

public/__redirects

@@ -14,7 +14,7 @@ With AutoRAG, you can:
 - **Customize your pipeline:** Choose from [Workers AI](/workers-ai) models, configure chunking strategies, edit system prompts, and more.
 - **Instant setup:** AutoRAG provisions everything you need from [Vectorize](/vectorize), [AI gateway](/ai-gateway), to pipeline logic for you, so you can go from zero to a working RAG pipeline in seconds.
 - **Keep your index fresh:** AutoRAG continuously syncs your index with your data source to ensure responses stay accurate and up to date.
-- **Ask questions:** Query your data and receive grounded responses via a [Workers binding](/ai-search/usage/workers-binding/) or [API](/ai-search/usage/rest-api/).
+- **Ask questions:** Query your data and receive grounded responses via a [Workers binding](/ai-search/api/search/workers-binding/) or [API](/ai-search/api/search/rest-api/).
 
 Whether you're building internal tools, AI-powered search, or a support assistant, AutoRAG gets you from idea to deployment in minutes.
 

src/assets/images/ai-search/hybrid-search.png

@@ -6,9 +6,9 @@ products:
 date: 2025-04-23
 ---
 
-You can now filter [AutoRAG](/ai-search/) search results by `folder` and `timestamp` using [metadata filtering](/ai-search/configuration/metadata) to narrow down the scope of your query.
+You can now filter [AutoRAG](/ai-search/) search results by `folder` and `timestamp` using [metadata filtering](/ai-search/configuration/indexing/metadata/) to narrow down the scope of your query.
 
-This makes it easy to build [multitenant experiences](/ai-search/how-to/multitenancy/) where each user can only access their own data. By organizing your content into per-tenant folders and applying a `folder` filter at query time, you ensure that each tenant retrieves only their own documents.
+This makes it easy to build [multitenant experiences](/ai-search/how-to/per-tenant-search/) where each user can only access their own data. By organizing your content into per-tenant folders and applying a `folder` filter at query time, you ensure that each tenant retrieves only their own documents.
 
 **Example folder structure:**
 

src/content/changelog/ai-search/2025-04-07-autorag-open-beta.mdx

@@ -6,7 +6,7 @@ products:
 date: 2025-06-19
 ---
 
-In [AutoRAG](/ai-search/), you can now view your object's custom metadata in the response from [`/search`](/ai-search/usage/workers-binding/) and [`/ai-search`](/ai-search/usage/workers-binding/), and optionally add a `context` field in the custom metadata of an object to provide additional guidance for AI-generated answers.
+In [AutoRAG](/ai-search/), you can now view your object's custom metadata in the response from [`/search`](/ai-search/api/search/workers-binding/) and [`/ai-search`](/ai-search/api/search/workers-binding/), and optionally add a `context` field in the custom metadata of an object to provide additional guidance for AI-generated answers.
 
 You can add [custom metadata](/r2/api/workers/workers-api-reference/#r2putoptions) to an object when uploading it to your R2 bucket.
 
@@ -46,4 +46,4 @@ For example:
 
 This gives you more control over how your content is interpreted, without requiring you to modify the original contents of the file.
 
-Learn more in AutoRAG's [metadata filtering documentation](/ai-search/configuration/metadata).
+Learn more in AutoRAG's [metadata filtering documentation](/ai-search/configuration/indexing/metadata/).

src/content/changelog/ai-search/2025-04-23-autorag-metadata-filtering.mdx

@@ -6,7 +6,7 @@ products:
 date: 2025-06-19
 ---
 
-In [AutoRAG](/ai-search/), you can now [filter](/ai-search/configuration/metadata/) by an object's file name using the `filename` attribute, giving you more control over which files are searched for a given query.
+In [AutoRAG](/ai-search/), you can now [filter](/ai-search/configuration/indexing/metadata/) by an object's file name using the `filename` attribute, giving you more control over which files are searched for a given query.
 
 This is useful when your application has already determined which files should be searched. For example, you might query a PostgreSQL database to get a list of files a user has access to based on their permissions, and then use that list to limit what AutoRAG retrieves.
 
@@ -25,4 +25,4 @@ const response = await env.AI.autorag("my-autorag").search({
 
 This allows you to connect your application logic with AutoRAG's retrieval process, making it easy to control what gets searched without needing to reindex or modify your data.
 
-Learn more in AutoRAG's [metadata filtering documentation](/ai-search/configuration/metadata/).
+Learn more in AutoRAG's [metadata filtering documentation](/ai-search/configuration/indexing/metadata/).

src/content/changelog/ai-search/2025-06-19-autorag-custom-metadata-and-context.mdx

@@ -10,36 +10,36 @@ date: 2025-10-28
 
 ## Rerank for more relevant results
 
-You can now enable [reranking](/ai-search/configuration/reranking/) to reorder retrieved documents based on their semantic relevance to the user’s query. Reranking helps improve accuracy, especially for large or noisy datasets where vector similarity alone may not produce the optimal ordering.
+You can now enable [reranking](/ai-search/configuration/retrieval/reranking/) to reorder retrieved documents based on their semantic relevance to the user’s query. Reranking helps improve accuracy, especially for large or noisy datasets where vector similarity alone may not produce the optimal ordering.
 
 You can enable and configure reranking in the dashboard or directly in your API requests:
 
 ```javascript
 const answer = await env.AI.autorag("my-autorag").aiSearch({
-  query: "How do I train a llama to deliver coffee?",
-  model: "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
-  reranking: {
-    enabled: true,
-    model: "@cf/baai/bge-reranker-base"
-  }
+	query: "How do I train a llama to deliver coffee?",
+	model: "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
+	reranking: {
+		enabled: true,
+		model: "@cf/baai/bge-reranker-base",
+	},
 });
 ```
 
 ## Set system prompts in API
 
-Previously, [system prompts](/ai-search/configuration/system-prompt/) could only be configured in the dashboard. You can now define them directly in your API requests, giving you per-query control over behavior. For example:
+Previously, [system prompts](/ai-search/configuration/retrieval/system-prompt/) could only be configured in the dashboard. You can now define them directly in your API requests, giving you per-query control over behavior. For example:
 
 ```javascript
 // Dynamically set query and system prompt in AI Search
 async function getAnswer(query, tone) {
-  const systemPrompt = `You are a ${tone} assistant.`;
+	const systemPrompt = `You are a ${tone} assistant.`;
 
-  const response = await env.AI.autorag("my-autorag").aiSearch({
-    query: query,
-    system_prompt: systemPrompt
-  });
+	const response = await env.AI.autorag("my-autorag").aiSearch({
+		query: query,
+		system_prompt: systemPrompt,
+	});
 
-  return response;
+	return response;
 }
 
 // Example usage
@@ -50,5 +50,4 @@ const answer = await getAnswer(query, tone);
 console.log(answer);
 ```
 
-Learn more about [Reranking](/ai-search/configuration/reranking/) and [System Prompt](/ai-search/configuration/system-prompt/) in AI Search.
-
+Learn more about [Reranking](/ai-search/configuration/retrieval/reranking/) and [System Prompt](/ai-search/configuration/retrieval/system-prompt/) in AI Search.

src/content/changelog/ai-search/2025-06-19-autorag-filename-filter.mdx

@@ -6,7 +6,7 @@ products:
 date: 2026-01-20
 ---
 
-[AI Search](/ai-search/) now includes [path filtering](/ai-search/configuration/path-filtering/) for both [website](/ai-search/configuration/data-source/website/#path-filtering) and [R2](/ai-search/configuration/data-source/r2/#path-filtering) data sources. You can now control which content gets indexed by defining include and exclude rules for paths.
+[AI Search](/ai-search/) now includes [path filtering](/ai-search/configuration/indexing/path-filtering/) for both [website](/ai-search/configuration/data-source/website/#path-filtering) and [R2](/ai-search/configuration/data-source/r2/#path-filtering) data sources. You can now control which content gets indexed by defining include and exclude rules for paths.
 
 By controlling what gets indexed, you can improve the relevance and quality of your search results. You can also use path filtering to split a single data source across multiple AI Search instances for specialized search experiences.
 
@@ -20,4 +20,4 @@ Path filtering uses [micromatch](https://github.com/micromatch/micromatch) patte
 | Keep admin pages out of results | —            | `**/admin/**`       |
 | Index only English content      | `**/en/**`   | —                   |
 
-Configure path filters when creating a new instance or update them anytime from **Settings**. Check out [path filtering](/ai-search/configuration/path-filtering/) to learn more.
+Configure path filters when creating a new instance or update them anytime from **Settings**. Check out [path filtering](/ai-search/configuration/indexing/path-filtering/) to learn more.

src/content/changelog/ai-search/2025-10-27-ai-search-reranking-system-prompt.mdx

@@ -8,7 +8,7 @@ date: 2026-01-20
 
 You can now create [AI Search](/ai-search/) instances programmatically using the [API](/ai-search/get-started/api/). For example, use the API to create instances for each customer in a multi-tenant application or manage AI Search alongside your other infrastructure.
 
-If you have created an AI Search instance via the [dashboard](/ai-search/get-started/dashboard/) before, you already have a [service API token](/ai-search/configuration/service-api-token/) registered and can start creating instances programmatically right away. If not, follow the [API guide](/ai-search/get-started/api/) to set up your first instance.
+If you have created an AI Search instance via the [dashboard](/ai-search/get-started/dashboard/) before, you already have a [service API token](/ai-search/configuration/indexing/service-api-token/) registered and can start creating instances programmatically right away. If not, follow the [API guide](/ai-search/get-started/api/) to set up your first instance.
 
 For example, you can now create separate search instances for each language on your website: