Last Reviewed: 2026-06-25

Apache Solr for Drupal

Dense Vector (AI Semantic) Search

Set up dense vector (AI-powered semantic) search for Drupal on Pantheon using Solr 9's DenseVectorField and the Search API Solr Dense Vector module.

Discuss in Slack

Edit this page on GitHub Report an issue with this doc

This guide covers the dense-vector-specific setup for semantic (AI-powered) search on a Drupal site on Pantheon, using Solr 9's DenseVectorField and the Search API Solr Dense Vector module. It assumes Pantheon Search is already set up; base Solr steps link out to Apache Solr for Drupal.

Traditional keyword search matches exact words. Dense vector search converts text into numerical vectors (embeddings) with an AI model, then finds content that is semantically similar. For example, a search for "fixing automobiles" matches an article about "car engine maintenance" even though the two share no keywords.

Alpha Release

Dense vector search requires Solr 9.6 or higher (Pantheon runs 9.10.1), and its setup ordering is sensitive. Follow the steps in the exact order described in this guide. Tested on Solr 9.10.1, Drupal 10.6.7 / 11.3.8.

This setup relies on alpha contributed modules: search_api_solr_dense_vector (1.0.0-alpha9) for dense vector search, and ai_search (2.0.0-alpha2), which is required for RAG search (Step 11). Alpha modules are unstable and can change or break between releases. Test thoroughly on non-production environments and do not rely on them for production search.

This setup uses contributed modules that Pantheon does not support, and an AI embedding provider that you bring yourself. It is not a Pantheon-supported feature.

Before You Begin

Complete the base Pantheon Search setup first.
Install and enable the Search API Pantheon module. Refer to Install the Search API Pantheon Module.
Set Solr 9 in pantheon.yml (requires Solr 9.6+; Pantheon runs 9.10.1). Refer to Configure the Solr Version. If you are using a Multidev, create the Multidev first, then set the Solr version — refer to the Multidev Solr inheritance note.
Create a Search API server and index with Title and Body fields. Refer to Add Search Index.
Install and authenticate Terminus.
Make Composer changes with Integrated Composer (commit only composer.json and composer.lock).
Obtain an AI embedding provider account: Google Vertex AI, OpenAI, or Google Gemini (see below).

This guide uses Terminus. Examples use <site> and <env> placeholders. Substitute your own site name and environment (for example, my-site.dev).

Choose an Embedding Provider

Step 1: Install the Dense Vector Modules

Pantheon Search and the search_api_pantheon module must already be installed (see Before You Begin). Add the dense-vector modules with Composer on your local clone:

Install the AI provider that matches your choice from Before You Begin:

Commit and push via Integrated Composer (only composer.json and composer.lock), then wait for the build to finish.

Step 2: Enable the Modules

Enable the dense-vector and AI modules with Terminus. The Key module stores API credentials securely; it is installed automatically as a dependency but must be enabled explicitly.

(search_api_pantheon is enabled as part of the base Solr setup.)

Step 3: Configure API Credentials

The AI provider needs credentials to call the embedding API. The Key module stores them securely. Go to Configuration > System > Keys (/admin/config/system/keys) and select Add key.

Note

Store API keys securely. Use a Key provider that keeps the value out of the Drupal database:

Pantheon Secret (recommended). The Pantheon Secrets module is a Key-module provider: set the value with terminus secret:site:set (runtime type), then select Pantheon Secret as the key provider. The key stays in Pantheon's secure vault, never in the site database or codebase. Refer to Use Pantheon Secrets with Drupal.
File. Upload the key to private:// and use the Key module File provider. Pantheon's read-only filesystem keeps it off the web-writable path.

Do not use the Configuration key provider for API keys: it stores the value in Drupal config (the database), where a module vulnerability could exfiltrate it. Secrets set with terminus secret:set --type=env --scope=web are not exposed to PHP ($_ENV, $_SERVER, getenv()) and cannot be read by the Key module; use the runtime type with the Pantheon Secrets module instead.

Step 4: Configure the AI Provider

Warning

Order matters. Complete this step before you configure the index processor. If the provider is not configured when you save the processor, the provider ID is stored as an empty string and embeddings silently fail during indexing.

Set the Default Embedding Provider

Go to Configuration > AI > Settings (/admin/config/ai/settings) and scroll to Vector Data Capabilities > Embedding Providers:

Set Embeddings Provider to your chosen provider.
Set Default Model to your embedding model (for example, Gemini Embedding 001, or text-embedding-3-small).
Select Save configuration.

Configure Provider Credentials

Step 5: Add a Dense Vector Field to the Index

The processor generates embeddings only for fields with the Dense Vector data type. On your existing index (see Add Search Index), go to /admin/config/search/search-api/index/<index-name>/fields:

Select Add fields and choose Content > Body (or your main content field).
Add it as a new field. Do not modify an existing body field. It is auto-named (for example, body_1).
Change the Type dropdown to Dense Vector.
Select Save.

The same source field can be indexed with multiple types: a Fulltext version for keyword search, a Dense Vector version for semantic search.

Step 6: Enable Vector Search and Reranking

Warning

Order matters. Do this before Step 7. Saving the index edit page can reset processor settings.

Go to /admin/config/search/search-api/index/<index-name>/edit, scroll to Dense Vector:

Check Enable vector comparison search and reranking.
Query/Re-ranking Plugin: Pure Vector (ranks on vector similarity only).
topK value: 10 (nearest neighbors to retrieve).
Minimum return: 0.5 (filter results below 50% similarity).
Minimum traverse: 0.
Select Save.

Step 7: Configure the Dense Vector Processor

Go to /admin/config/search/search-api/index/<index-name>/processors, select Dense Vector:

AI provider: the provider from Step 4.
AI model: the embedding model from Step 4.
Similarity function: Cosine.
Content field: the Dense Vector field from Step 5.
Select Save.

Verify the config is correct:

ai_provider must not be empty. If it is, the provider was not configured when the processor was saved. Return to Step 4 and re-save the processor.

Step 8: Post the Schema

Post the schema so Solr knows the DenseVectorField definition. See Post the Schema for the base mechanics.

Note

Re-post the schema whenever you change the dense vector configuration (model, dimensions, or similarity function). If postSchema returns a 502, wait a few seconds and retry. Intermittent gateway timeouts can occur.

Step 9: Index Content

Index your content (see Index Content). During indexing, the processor sends each item's text to the embedding API and stores the vector in Solr.

Verify vectors are actually stored:

Look for a field starting with knns_ (for example, knns_body_1). If absent, embeddings are not being generated. See Indexing succeeds but no knns_ field in Solr documents in Troubleshooting.

Step 10: Test Semantic Search

Create a Drupal View backed by your Search API index (Structure > Views > Add view; under Show, select the index, not Content), add a page path (for example, /vector-search), an exposed Fulltext search filter, and Title/Body fields.

Test with queries that share meaning but not keywords with your content:

Query	What it tests
fixing automobiles	Matches content about car maintenance (no keyword overlap)
preparing food	Matches cooking and food content
protecting nature	Matches environment and gardening content

If results rank by semantic relevance rather than keyword matching, dense vector search is working.

Step 11: RAG Search (Optional, Drupal 11.1+)

RAG (Retrieval-Augmented Generation) adds an LLM answer-generation step on top of vector search: a user asks a question, the system finds relevant content with KNN, then an LLM generates a natural language answer.

Alpha software

RAG search requires the ai_search module, which is alpha (2.0.0-alpha2). Treat RAG search as experimental, expect breaking changes between releases, and test only on non-production environments.

Prerequisites: Drupal 11.1+ (the rag_search module uses D11.1+ APIs), a working dense vector setup (Steps 1-10), and a chat-capable AI provider (OpenAI GPT-4o-mini recommended, or Gemini Pro).

Install and enable:

Set the default chat provider at /admin/config/ai:

Default chat provider: OpenAI (recommended) or Gemini
Default chat model: gpt-4o-mini or gemini-2.5-flash

Configure RAG at /admin/config/rag_search/settings: enable the route and set a route title/path (for example, /rag-search); set VDB Index to your Search API index; fill in the question/format/CTA/no-response text. Leave cache disabled for testing.

RAG Search Known Issues

Rate limiting form bug. Hidden rate limit fields have min=1 and value=0, failing HTML5 validation. Workaround: check Enable rate limiting, fill in the values, save. Uncheck afterward if not needed.
Source article links broken. Article titles appear as links in the answer but do not resolve to the correct node URLs.
PHP 8.2+ serialization error. On form rebuild: Cannot initialize readonly property Drupal\rag_search\Form\RagSearchForm::$ragSearchProcessService. Intermittent on a second submission; the initial answer still works.
Gemini free tier quota. RAG makes both embedding and chat calls; the free tier exhausts quickly. Use OpenAI for sustained testing.

Troubleshooting

Error: incorrect vector dimensions

Solr returns HTTP 500 during indexing with incorrect vector dim. The schema's vectorDimension does not match the model's output, usually after switching models without re-posting the schema, or when the provider returns 0 for embeddingsVectorSize().

Clear cache: terminus drush <site>.<env> -- cr
Re-post schema: terminus drush <site>.<env> -- search-api-pantheon:postSchema
Clear index: terminus drush <site>.<env> -- search-api:clear <index>
Reindex: terminus drush <site>.<env> -- search-api:index

If the provider returns 0 for dimensions (known issue with gemini-embedding-2-preview and gemini-embedding-2), set it manually at /admin/config/search/search-api/server/<server>/solr_field_type/knn_vector_und_9_0_0/edit (set vectorDimension = 3072), then re-post schema and reindex.

Indexing succeeds but no `knns_` field in Solr documents

search-api:index reports all items indexed, but documents have no knns_body_1 field and search returns keyword matches. The processor is not generating embeddings. Most common causes:

Empty ai_provider in the processor config (provider not configured when the processor was first saved).
No Dense Vector field in the index (only a Fulltext body field).
Enable vector comparison search and reranking not checked on the index edit page.

Fix each, then post schema, clear, and reindex.

Search returns keyword matches instead of semantic matches

A search for "protecting nature" returns "Natural Language Processing" (keyword match on "natural") instead of a semantic match. The query is using fulltext search, not KNN.

Confirm Enable vector comparison search and reranking is checked.
Confirm Query/Re-ranking Plugin is Pure Vector, not hybrid.
Confirm vectors are stored (knns_ field present).
Clear cache and retest.

Provider credentials not working

The verification command returns empty or errors, and no calls appear in the provider dashboard.

Verify the key loaded (Key details page shows key length > 0).
Vertex AI: project_id in the JSON matches the model config, aiplatform.googleapis.com is enabled, the service account has roles/aiplatform.user.
OpenAI: key starts with sk-, billing enabled.
Gemini: key starts with AIza; test it at Google AI Studio.