How search works
Every time a chatter sends a message, the agent searches its sources for the parts that answer it. Those parts go to the chat model with the question. The model is told to answer only from them. Good search is what keeps the answers right, so this page explains it in detail.
All the settings on this page are in the admin panel, under Settings → Search. They apply to every agent on the install. Each option has a ? next to it that explains what it does.
The steps
- Understand follow-ups. If the message continues the chat, it is rewritten into a full question.
- Check the Q&A pairs. A matching saved answer is sent as it is, and the search stops here.
- Search four ways at once: by meaning and by keywords, on the chunk text and on the questions written for each chunk.
- Merge the results into one list.
- Leave out weak matches with a reranker or a similarity check.
- Keep the best few, within a size limit, and give them to the model.
Search runs on every message, even "hello". A greeting simply finds nothing that passes step 5, so no knowledge is added.
Step 1: Follow-up questions
Chatters often ask short follow-ups:
Chatter: How do I reset my password on the web app?
Agent: Open Settings, then Security, and choose Reset password.
Chatter: and on Android?
Searching for "and on Android?" alone finds nothing useful. So, when a chat has earlier messages, a small AI model reads the last few messages and rewrites the follow-up into a full question: "How do I reset my password on Android?". The same call also detects the chatter's language.
The rewritten question is only used for the search. The agent still answers the message the chatter actually wrote. If the rewrite fails, Chatvia searches with the original message.
- Turn it on or off: Understand follow-up questions, in the Ambiguous questions card (on by default).
- Pick the model: Settings → AI models → Models for app tasks → Follow-up questions. It runs before every reply, so pick a small, fast model.
Step 2: Q&A pairs
Before searching, Chatvia checks the agent's Q&A pairs. If the question matches one, the saved answer is sent word for word, and no search runs. See Q&A pairs are special.
Step 3: Hybrid search
Chatvia searches in two ways at once. Using both together is called hybrid search. It is the usual way to search in retrieval-augmented generation (RAG), the method of answering from your own sources that Chatvia uses.
Search by meaning
The question is turned into an embedding, the same way the chunks were (see Embeddings). Then PostgreSQL finds the chunks whose embeddings are closest to it.
This finds the right chunk even when the words differ. "Can I get my money back?" finds a chunk about the "refund policy", although the two share no word.
Search by meaning is weak at exact things: product codes, order numbers, names, error messages, rare words. To an embedding model, "ERR-4021" and "ERR-4012" look almost the same.
Search by keywords
Search by keywords finds the chunks that share words with the question. It is good at exactly what search by meaning is weak at. A chunk that contains "ERR-4021" is found when the chatter types "ERR-4021".
It ranks a chunk higher when it shares more of the question's words, and when those words are rare. A word that appears in almost every chunk, like "the" or your company name, counts for little.
Keyword search works in every language. It does not reduce words to their root ("running" does not match "run"), because chatters and sources can be in any language. Search by meaning covers that gap.
Four searches, not two
When AI enhancement is on, every chunk also has a list of questions it answers. Chatvia searches those questions too, both ways. So one message runs up to four searches:
| By meaning | By keywords | |
|---|---|---|
| Chunk text (with its summary) | ✓ | ✓ |
| Questions written for the chunk | ✓ | ✓ |
Turning hybrid search off
The Hybrid search switch (in Search → Finding answers, on by default) turns the two keyword searches off. Then Chatvia searches by meaning only. Leave it on unless you have a reason. It makes search better for names, codes and rare words, and it adds almost no time to a reply. You can change it at any time. Nothing needs to be trained again.
The two keyword search engines
Keyword search can run on one of two engines inside PostgreSQL. Chatvia picks one on its own, when it starts.
| pg_textsearch | Built-in full-text search | |
|---|---|---|
| What it is | A PostgreSQL extension from Timescale. | Part of every PostgreSQL. |
| Ranking | BM25, the ranking used by most search engines. It weighs rare words higher and is fair to short and long chunks. | A simpler score that mostly counts matching words. |
| Needs | Installing the extension and loading it when PostgreSQL starts. | Nothing. |
| Where it works | Your own server, the Chatvia Docker image. | Everywhere, including managed databases (Amazon RDS, Cloud SQL…) that don't allow extra extensions. |
Both engines find the same kind of matches. pg_textsearch puts the best ones higher in the list, so it is the better choice when you can install it. It is not required: without it, keyword search still works with the built-in engine. That's why the Hybrid search switch stays available on every install.
To see which engine your install uses, open the admin panel's Status page. The Keyword search row shows pg_textsearch (BM25) or Built-in full-text search.
Choosing the engine yourself
By default (SEARCH_LEXICAL_DRIVER=auto in .env), Chatvia uses
pg_textsearch when the extension is installed in the database, and the
built-in engine when it is not. Set SEARCH_LEXICAL_DRIVER to
pg_textsearch or postgres to force one.
The two engines need different search indexes. After you add or remove pg_textsearch, or change the setting, rebuild the indexes:
php artisan chatvia:search:lexical-indexesSee Installation → The database for how to install pg_textsearch.
Step 4: Merge the results
Each of the four searches returns its own ranked list. The scores of a meaning search and a keyword search can't be compared, so Chatvia merges the lists by rank, with a method called Reciprocal Rank Fusion. A chunk gets points for each list it appears in, and more points the higher it is in that list. A chunk that several searches agree on rises to the top.
The result is one list of about 25 candidates.
Step 5: Leave out weak matches
Search always returns something, even when the sources don't cover the question. Sending weak matches to the model makes it more likely to answer from the wrong part. So Chatvia checks every candidate and drops those that don't really answer the question. This is the relevance check (Leave out weak matches in Advanced → Fine-tuning, on by default).
It works in one of two ways:
- With a reranker. A reranker is an AI model made for one job: reading a question and a text, and scoring how well the text answers it. It is much more precise than the search itself. Chatvia sends it the candidates, re-sorts them by its scores, and drops those below the minimum score. Supported: OpenRouter (with Cohere, Voyage or Qwen3 rerank models), Cohere Rerank directly, or your own reranker server. Pick it in Settings → AI models → Reranker.
- Without a reranker. Chatvia drops the candidates whose meaning is not close enough to the question (a similarity check). This is less precise, and it only works with an embedding model whose scores Chatvia knows (Qwen3), or when you set Minimum similarity (no reranker) yourself. Otherwise, with no reranker, the best candidates are kept without a check.
If the reranker can't be reached, Chatvia uses the similarity check for that message, so the chat keeps working.
The minimum scores (Minimum score (reranker) and Minimum similarity (no reranker)) are set to tested values for the default models. Change them only if you have tested other values.
Step 6: Keep the best few
Chatvia keeps the best candidates, up to two limits:
- Results per question (default 5, at most 20): how many chunks the agent gets.
- Most tokens per question (default 3,000, in Advanced → Fine-tuning): the most text they may add up to. The best chunk is always kept. A chunk that would go over the limit is skipped, and a smaller one after it may still fit.
More results give the model more to work with, but also more to get wrong, and every message gets longer. 5 suits most agents.
When nothing matches
If no chunk passes the check, the model gets the question with no knowledge. The agent's instructions tell it to say briefly that it doesn't have that information, and to offer a useful next step. It must not make up an answer.
Two options in the Ambiguous questions card change this. Both are off by default:
- Ask back when a message is unclear: when a message is too vague to search, the agent asks what the chatter means instead. It needs Understand follow-up questions.
- Say what the agent can help with: the agent tells the chatter which topics it can help with.
In Chatlog, every answer shows a confidence: how well the agent's knowledge matched the question (not how good the answer is). Low confidence on many chats about the same subject usually means a source is missing.
Showing sources to chatters
The chunks used for an answer are always saved with it. To show them to chatters, open the agent's Setup → Settings and pick how Sources are shown: Hidden (the default), List under the answer (a button under each answer lists the files, pages and web pages the agent used), or Numbered citations (a small number after each sentence opens its source).
Big agents
For an agent with up to 10,000 chunks, Chatvia compares the question with every chunk, which gives the most exact results. For bigger agents it uses the search indexes, which are much faster but can miss a match now and then. The Search mode setting (in Advanced → Fine-tuning) trades speed for accuracy on those bigger agents: Faster, Balanced (the default) or More accurate.