Servinge Embeddings and Rankers with BentoML

BentoML API for Jina v3 multilingual embeddings and cross encoder reranker using the reranker wrapper.

Preprequisites

Install astral uv.

Run the BentoML Service locally

uv run bento serve service:Embeddings

This will run the BentoML API with a Swagger Documentation up and running.

Run as docker container

docker compose up --build

Push docker image to quay.io

docker push quay.io/ktbs/fd-itbs-dms/embeddings

Change the embedding or reranker model

In the service.py file:

For the embedding model, change EMBEDDING_MODEL_ID to your hugging face sentence transformer compatible embedding model.

For the reranker change RANKER_TYPE and LANGUAGE according to the docs of the rerankers library.

Run tests

uv run pytest