Index
Index
The Index function allows you to create and manage indexes that organize documents for efficient retrieval by AI agents. Indexes are essential for allowing agents to quickly and accurately access the information needed for specific tasks.
Creating a New Index
To create a new index:
Click the "Create Index" button.
Provide a name and description to define the index’s purpose.
Choose an Index Type from the following options:
BM25 Index
Vector Index
Summary Index
Chunk Size Recommendations
For index types that require chunking (BM25 and Vector), selecting the appropriate chunk size is essential for effective retrieval:
Smaller Chunks: Ideal for precise retrieval in technical or structured documents like user manuals, FAQs, or knowledge bases.
Larger Chunks: Suitable for content where broader context improves comprehension, such as reports, narratives, or articles.
Index Types
1. BM25 Index
Uses the Best Match 25 (BM25) algorithm for keyword-based matching.
Best suited for queries that rely on exact term relevance.
Requires chunk size selection.
2. Vector Index
A vector-based index enabling semantic search and natural language understanding.
Ideal for retrieving conceptually similar information, even if phrased differently.
Requires chunk size selection.
3. Summary Index
Generates summaries (up to 5,000 characters per document) using a selected Large Language Model (LLM).
Stores the summary in a vector index for fast and efficient retrieval.
Does not require chunk size selection.
Steps:
Enter a summary query to guide summarization.
Select a suitable LLM.
Click "Create Index" to generate and store summaries.
Last updated