Using the Postgres Vector Store connector (Preview)

Warning

The Postgres Vector Store functionality is in preview, and improvements that require breaking changes may still occur in limited circumstances before release.

Warning

The Semantic Kernel Vector Store functionality is in preview, and improvements that require breaking changes may still occur in limited circumstances before release.

Warning

The Semantic Kernel Vector Store functionality is in preview, and improvements that require breaking changes may still occur in limited circumstances before release.

Overview

The Postgres Vector Store connector can be used to access and manage data in Postgres and also supports Neon Serverless Postgres.

The connector has the following characteristics.

Feature Area	Support
Collection maps to	Postgres table
Supported key property types	short int long string Guid
Supported data property types	bool short int long float double decimal string DateTime DateTimeOffset Guid byte[] bool Enumerables short Enumerables int Enumerables long Enumerables float Enumerables double Enumerables decimal Enumerables string Enumerables DateTime Enumerables DateTimeOffset Enumerables Guid Enumerables
Supported vector property types	ReadOnlyMemory<float> Embedding<float> float[] ReadOnlyMemory<Half> Embedding<Half> Half[] BitArray Pgvector.SparseVector
Supported index types	Hnsw
Supported distance functions	CosineDistance CosineSimilarity DotProductSimilarity EuclideanDistance ManhattanDistance
Supported filter clauses	AnyTagEqualTo EqualTo
Supports multiple vectors in a record	Yes
IsIndexed supported?	No
IsFullTextIndexed supported?	No
StorageName supported?	Yes
HybridSearch supported?	No

Limitations

Important

When initializing NpgsqlDataSource manually, it is necessary to call UseVector on the NpgsqlDataSourceBuilder. This enables vector support. Without this, usage of the VectorStore implementation will fail.

Here is an example of how to call UseVector.

NpgsqlDataSourceBuilder dataSourceBuilder = new("Host=localhost;Port=5432;Username=postgres;Password=example;Database=postgres;");
dataSourceBuilder.UseVector();
NpgsqlDataSource dataSource = dataSourceBuilder.Build();

When using the AddPostgresVectorStore dependency injection registration method with a connection string, the datasource will be constructed by this method and will automatically have UseVector applied.

Getting started

Add the Postgres Vector Store connector NuGet package to your project.

dotnet add package Microsoft.SemanticKernel.Connectors.PgVector --prerelease

You can add the vector store to the IServiceCollection dependency injection container using extension methods provided by Semantic Kernel.

In this case, an instance of the Npgsql.NpgsqlDataSource class, which has vector capabilities enabled, will also be registered with the container.

using Microsoft.Extensions.DependencyInjection;

// Using IServiceCollection with ASP.NET Core.
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddPostgresVectorStore("Host=localhost;Port=5432;Username=postgres;Password=example;Database=postgres;");

Extension methods that take no parameters are also provided. These require an instance of the Npgsql.NpgsqlDataSource class to be separately registered with the dependency injection container.

using Microsoft.Extensions.DependencyInjection;
using Npgsql;

// Using IServiceCollection with ASP.NET Core.
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddSingleton<NpgsqlDataSource>(sp => 
{
    NpgsqlDataSourceBuilder dataSourceBuilder = new("Host=localhost;Port=5432;Username=postgres;Password=example;Database=postgres;");
    dataSourceBuilder.UseVector();
    return dataSourceBuilder.Build();
});

builder.Services.AddPostgresVectorStore();

You can construct a Postgres Vector Store instance directly with a custom data source or with a connection string.

using Microsoft.SemanticKernel.Connectors.PgVector;
using Npgsql;

NpgsqlDataSourceBuilder dataSourceBuilder = new("Host=localhost;Port=5432;Username=postgres;Password=example;Database=postgres;");
dataSourceBuilder.UseVector();
var dataSource = dataSourceBuilder.Build();

var connection = new PostgresVectorStore(dataSource, ownsDataSource: true);

using Microsoft.SemanticKernel.Connectors.PgVector;

var connection = new PostgresVectorStore("Host=localhost;Port=5432;Username=postgres;Password=example;Database=postgres;");

It is possible to construct a direct reference to a named collection with a custom data source or with a connection string.

using Microsoft.SemanticKernel.Connectors.PgVector;
using Npgsql;

NpgsqlDataSourceBuilder dataSourceBuilder = new("Host=localhost;Port=5432;Username=postgres;Password=example;Database=postgres;");
dataSourceBuilder.UseVector();
var dataSource = dataSourceBuilder.Build();

var collection = new PostgresCollection<string, Hotel>(dataSource, "skhotels", ownsDataSource: true);

using Microsoft.SemanticKernel.Connectors.PgVector;

var collection = new PostgresCollection<string, Hotel>("Host=localhost;Port=5432;Username=postgres;Password=example;Database=postgres;", "skhotels");

Data mapping

The Postgres Vector Store connector provides a default mapper when mapping from the data model to storage. This mapper does a direct conversion of the list of properties on the data model to the columns in Postgres.

Property name override

You can provide override property names to use in storage that is different to the property names on the data model. The property name override is done by setting the StorageName option via the data model property attributes or record definition.

Here is an example of a data model with StorageName set on its attributes and how that will be represented in a Postgres command.

using Microsoft.Extensions.VectorData;

public class Hotel
{
    [VectorStoreKey]
    public int HotelId { get; set; }

    [VectorStoreData(StorageName = "hotel_name")]
    public string? HotelName { get; set; }

    [VectorStoreData(StorageName = "hotel_description")]
    public string? Description { get; set; }

    [VectorStoreVector(Dimensions: 4, DistanceFunction = DistanceFunction.CosineDistance)]
    public ReadOnlyMemory<float>? DescriptionEmbedding { get; set; }
}

CREATE TABLE public."Hotels" (
    "HotelId" INTEGER NOT NULL,
    "hotel_name" TEXT ,
    "hotel_description" TEXT ,
    "DescriptionEmbedding" VECTOR(4) ,
    PRIMARY KEY ("HotelId")
);

Coming soon

More info coming soon.

JDBC

The JDBC connector can be used to connect to Postgres.