hybrid_search

Langchain supports hybrid search with a Supabase Postgres database. The hybrid search combines the postgres pgvector extension (similarity search) and Full-Text Search (keyword search) to retrieve documents. You can add documents via SupabaseVectorStore addDocuments function. SupabaseHybridKeyWordSearch accepts embedding, supabase client, number of results for similarity search, and number of results for keyword search as parameters. The getRelevantDocuments function produces a list of documents that has duplicates removed and is sorted by relevance score.

Installation

Copyselect dbdev.install('langchain-hybrid_search');
create extension if not exists vector;
create extension "langchain-hybrid_search"
    schema public
    version '1.1.0';

Note:

vector is a dependency of langchain-hybrid_search. Dependency resolution is currently under development. In the near future it will not be necessary to manually create dependencies.

Once created, you can access the vector store for search using langchain as shown below:

Copyimport { OpenAIEmbeddings } from "langchain/embeddings/openai";
import { createClient } from "@supabase/supabase-js";
import { SupabaseHybridSearch } from "langchain/retrievers/supabase";

const privateKey = process.env.SUPABASE_PRIVATE_KEY;
if (!privateKey) throw new Error(`Expected env var SUPABASE_PRIVATE_KEY`);

const url = process.env.SUPABASE_URL;
if (!url) throw new Error(`Expected env var SUPABASE_URL`);

export const run = async () => {
  const client = createClient(url, privateKey);

  const embeddings = new OpenAIEmbeddings();

  const retriever = new SupabaseHybridSearch(embeddings, {
    client,
    //  Below are the defaults, expecting that you set up your supabase table and functions according to the guide above. Please change if necessary.
    similarityK: 2,
    keywordK: 2,
    tableName: "documents",
    similarityQueryName: "match_documents",
    keywordQueryName: "kw_match_documents",
  });

  const results = await retriever.getRelevantDocuments("hello bye");

  console.log(results);
};

For more details, checkout the LangChain Supabase Hybrid Search docs: https://js.langchain.com/docs/modules/indexes/retrievers/supabase-hybrid