Usage

• 1: Indexing
• 2: Search
• 3: Monitoring
• 4: Quantization
• 5: `pgvector` compatibility

1 - Indexing

Indexing is the process of building a data structure that allows for efficient search. pgvecto.rs supports three indexing algorithms: brute force (flat), IVF, and HNSW. The default algorithm is HNSW.

To construct an index for vectors, first create a table named items with a column named embedding of type vector(n). Then, populate the table with generated data.

CREATE TABLE items (embedding vector(3));
INSERT INTO items (embedding) SELECT ARRAY[random(), random(), random()]::real[] FROM generate_series(1, 1000);

You can create a vector index for squared Euclidean distance with the following SQL.

CREATE INDEX ON items USING vectors (embedding vector_l2_ops);

The vector_l2_ops operator class calculates squared Euclidean distance. Refer to the documentation for a list of operator classes.

To switch the indexing algorithm from the default HNSW to IVF, execute the SQL command below. For additional examples of usage, refer to the documentation.

CREATE INDEX ON items USING vectors (embedding vector_l2_ops)
WITH (options = $$
[indexing.ivf]
$$);

You can now perform a KNN search with the following SQL, leveraging the vector index for your query.

SELECT * FROM items ORDER BY embedding <-> '[3,2,1]' LIMIT 5;

Examples

Here are some examples of how to create indexes using various algorithms and settings. For a complete list of options, please consult the reference guide.

-- HNSW algorithm, default settings.

CREATE INDEX ON items USING vectors (embedding vector_l2_ops);

-- Use FP16 vector type for optimizing data storage.

CREATE TABLE items (embedding vecf16(3));
CREATE INDEX ON items USING vectors (embedding vecf16_l2_ops)

--- Or using brute force with product quantization (PQ).

CREATE INDEX ON items USING vectors (embedding vector_l2_ops)
WITH (options = $$
[indexing.flat]
quantization.product.ratio = "x16"
$$);

-- Or using brute force with scalar quantization (SQ).
CREATE INDEX ON items USING vectors (embedding vector_l2_ops)
WITH (options = "[indexing.hnsw.quantization.scalar]");

--- Or using IVFPQ algorithm.

CREATE INDEX ON items USING vectors (embedding vector_l2_ops)
WITH (options = $$
[indexing.ivf]
quantization.product.ratio = "x16"
$$);

-- Use more threads for background building the index.

CREATE INDEX ON items USING vectors (embedding vector_l2_ops)
WITH (options = $$
optimizing.optimizing_threads = 16
$$);

-- Prefer smaller HNSW graph.

CREATE INDEX ON items USING vectors (embedding vector_l2_ops)
WITH (options = $$
segment.max_growing_segment_size = 200000
$$);

Suggestions

Selecting the appropriate indexing algorithm and settings is crucial for vector search performance. Below are recommendations tailored to various use cases.

Brute force (flat)

If you’re conducting a limited number of searches (between 1,000 and 10,000) or require precise results, the brute force algorithm is a viable option.

This algorithm sets the standard for other indexing methods. It retains vectors in their original form without compression and avoids additional overhead. While straightforward and user-friendly, it’s not ideal for processing large datasets.

In addition, you can use quantization to reduce memory usage. For a complete list of all options, please refer to the documentation. Detailed information about quantization can also be found there.

Inverted file index (IVF)

IVF mainly uses the idea of inverted indexing to store the vectors (id, vector) under each cluster center. When querying a vector, it finds the nearest several centers and searches for the vectors under these centers respectively.

Before you begin your search, it’s necessary to train nlist cluster centers. You also have the option to set nsample for K-Means clustering; a higher value yields more precise clusters but increases training time. For additional settings, please consult the reference guide.

Before searching, train nlist cluster centers using K-Means clustering with nsample points. A higher value yields more precise clustering but requires longer training time. For additional settings, please consult the reference guide.

Hierarchical navigable small world graph (HNSW)

HNSW merges the concepts of skip list with navigable small world (NSW) graphs, employing a hierarchical structure that features longer edges on higher layers for rapid searching and shorter edges on lower layers to enhance search precision.

Navigable small world

Navigable Small World operate on the premise that each node is accessible from any other through a minimal number of “hops.” Once an entry point is chosen—be it random, heuristic-based, or algorithmically determined—the surrounding nodes are examined to identify if they’re nearer than the current one. The search progresses to the nearest neighbor and continues until no closer nodes can be found.

When constructing the graph, the m parameter determines how many connections a new node will establish with its nearest neighbors. A higher m value results in a denser and more interconnected graph but also increases memory usage and slows down insertion times. During node insertion, the algorithm identifies the closest m nodes to create two-way links with them.

The search process navigates through an NSW graph, beginning at a pre-defined entry point. The algorithm proceeds by greedily moving to adjacent vertices closer to the query vector. The process stops once it finds no closer neighbors in the current vertex’s adjacency list.

A limitation of the NSW search is that it opts for the seemingly shortest path to the nearest node, disregarding the graph’s overall structure. This “greedy search” approach can result in getting stuck at a local optimum, an issue often referred to as “early stopping.”

Skip list

Skip lists are constructed using a probabilistic approach, where nodes become less likely to appear as you move up the layers, resulting in fewer nodes at higher levels. To search for a node, the algorithm starts at the top layer’s “head” and moves to the next node that is greater than or equal to the target. If the node isn’t found, it descends to a lower (more fine-grained) layer and continues this pattern until it locates either the desired node or its nearest neighbors.

HNSW

Constructing this multi-layered graph begins with a foundational layer that encapsulates the entire dataset. Moving up the hierarchy, each successive layer provides a streamlined summary of its predecessor, featuring fewer nodes and functioning as fast tracks for making broader leaps within the graph.

Smaller m) values are better for lower-dimensional data or when you require lower recall. Larger m values are useful for higher-dimensional data or when high recall is important.

The ef_construction parameter determines the dynamic candidate list size when adding new nodes; increasing this value may enhance recall but could extend index construction time.

The HNSW index is resource-intensive, requiring additional RAM and an adjustment to the maintenance_work_mem setting for larger datasets. If you’re seeking performance that’s markedly faster than IVF, with a high recall rate and scalability that matches dataset size, HNSW is an excellent option.

2 - Search

Get the nearest 5 neighbors to a vector

SET vectors.hnsw_ef_search = 64;
SELECT * FROM items ORDER BY embedding <-> '[3,2,1]' LIMIT 5;

Operators

These operators are used for distance metrics:

For their definitions, see overview.

Filter

For a given category, get the nearest 10 neighbors to a vector

SELECT 1 FROM items WHERE category_id = 1 ORDER BY embedding <#> '[0.5,0.5,0.5]' limit 10

Query options

Set ivf scan lists to 1 in session:

SET vectors.ivf_nprobe=1;

Set hnsw search scope to 40 in transaction:

SET LOCAL vectors.hnsw_ef_search=40;

For all options, refer to search options.

Search modes

There are two search modes: vbase and basic.

vbase

As the default search mode, vbase is suitable for most scenarios.

For how it works, refer to the thesis VBASE: Unifying Online Vector Similarity Search and Relational Queries via Relaxed Monotonicity.

basic

basic is behaviorally consistent with vector algorithm libraries such as faiss. It will be useful if you want to align them.

Enabling basic, you must respect these restrictions:

• Execute VACUUM after updates before searches.
• Search without filter

3 - Monitoring

We provide a view pg_vector_index_stat to monitor the progress of indexing.

{
    "vector": {
        "dimensions": 256,
        "distance": "L2",
        "kind": "F32"
    },
    "segment": {
        "max_growing_segment_size": 20000,
        "max_sealed_segment_size": 1000000
    },
    "optimizing": {
        "sealing_secs": 60,
        "sealing_size": 1,
        "delete_threshold": 0.2,
        "optimizing_threads": 2
    },
    "indexing": {
        "hnsw": {
            "m": 12,
            "ef_construction": 300,
            "quantization": {
                "trivial": {}
            }
        }
    }
}

If there are no more insertions and you wait for all indexing to be finished, you can check the field idx_indexing every 60 seconds util it’s true.

4 - Quantization

Quantization is a technique to reduce the memory consumption and computation complexity with the affordable decrease in precision. In the context of vector search, there are mainly two types of quantization: production quantization and scalar quantization.

Production Quantization

Production quantization is a method that trains several clustering models on the dataset and maps each vector to a list of cluster center indexes. It is commonly used address the problem of excessive memory usage with high-dimensional data.

In pgvecto.rs, you can enable the product quantization with:

CREATE INDEX ON items USING vectors (embedding vector_l2_ops)
WITH (options = $$
[indexing.ivf]
quantization.product.ratio = "x16"
$$);

More configurations can be found in indexing.

Scalar Quantization

Scalar quantization maps each value to low-bit format like from 32-bit float to 8-bit integer. Usually, this won’t affect the precision a lot but can reduce the memory usage and improve the query performance. pgvecto.rs supports uint8 scalar quantization with:

CREATE INDEX ON items USING vectors (embedding vector_l2_ops)
WITH (options = "[indexing.hnsw.quantization.scalar]");

Suggestions

When quantization is required, it’s always recommended to try the scalar quantization first. It’s a good balance between precision and memory usage.

If the memory usage is still too high, you can try the product quantization. This method can reduce the memory usage significantly but may affect the precision. Since it’s highly related to the training dataset. You need to try different ratio and sample values to find the best one for your dataset.

Otherwise, you may also use some dimension reduction methods like PCA to reduce the vector dimension.

5 - `pgvector` compatibility

pgvecto.rs is natively compatible with pgvector at:

• CREATE TABLE commands, e.g. CREATE TABLE t (val vector(3))
• INSERT INTO commands, e.g. INSERT INTO t (val) VALUES ('[0.6,0.6,0.6]')

pgvecto.rs can be configured to be compatible with pgvector at:

• Index options, which allows you to create index by USING hnsw (val vector_ip_ops)
• Query options, which can be set by SET ivfflat.probes = 10

This feature is named pgvector compatibility. It can be enabled by SET vectors.pgvector_compatibility = on.

Index Options & Query Options

For index ivfflat and hnsw only the following options are available.

• Default value for index options is different from pgvecto.rs original, which keeps the same as in pgvector.
• Default value for query options is the same as in pgvecto.rs.

Index options for ivfflat:

Query options for ivfflat:

Index options for hnsw:

Query options for hnsw:

Examples

It’s easy to enable pgvector compatibility and start a vector query.

DROP TABLE IF EXISTS t;
SET vectors.pgvector_compatibility=on;
SET hnsw.ef_search=40;
CREATE TABLE t (val vector(3));
INSERT INTO t (val) SELECT ARRAY[random(), random(), random()]::real[] FROM generate_series(1, 1000);
CREATE INDEX hnsw_cos_index ON t USING hnsw (val vector_cosine_ops);
SELECT COUNT(1) FROM (SELECT 1 FROM t ORDER BY val <-> '[0.5,0.5,0.5]' limit 100) t2;
DROP INDEX hnsw_cos_index;

Multiply types of indexes are accepted:

SET vectors.pgvector_compatibility=on;
SET hnsw.ef_search=40;
-- [hnsw + vector_l2_ops] index with default options
CREATE INDEX hnsw_l2_index ON t USING hnsw (val vector_l2_ops);
-- [hnsw + vector_cosine_ops] index with single ef_construction option
CREATE INDEX hnsw_cosine_index ON t USING hnsw (val vector_cosine_ops) WITH (ef_construction = 80);
-- anonymous [hnsw + vector_ip_ops] with all options
CREATE INDEX ON t USING hnsw (val vector_ip_ops) WITH (ef_construction = 80, m = 12);
SET ivfflat.probes=1;
-- [ivfflat + vector_l2_ops] index with default options
CREATE INDEX ivfflat_l2_index ON t USING ivfflat (val vector_l2_ops);
-- [ivfflat + vector_ip_ops] index with all options
CREATE INDEX ivfflat_ip_index ON t USING ivfflat (val vector_cosine_ops) WITH (nlist = 80);
-- anonymous [ivf + vector_ip_ops] with all options
CREATE INDEX ON t USING ivfflat (val vector_ip_ops) WITH (lists = 80)