Using Vector Databases: Practical Advice for Production // Sam Partee // LLMs in Prod Conference

This portion is sponsored by Redis
Website: https://redis.io/
Redis is an in-memory data structure store, used as a distributed, in-memory key–value database, cache and message broker, with optional durability. Redis supports different kinds of abstract data structures, such as strings, lists, maps, sets, sorted sets, HyperLogLogs, bitmaps, streams, and spatial indices.

//Abstract
In the last LLM in Production event, Sam spoke on some of the ways they’ve seen people use a vector database for large language models. This included use cases like information/context retrieval, conversational memory for chatbots, and semantic caching.

These are great and make for flashy demos, however, using this in production isn’t trivial. Often times, the less flashy side of these use cases can present huge challenges such as: Advice on prompts? How do I chunk up text? What if I need HIPAA compliance? On-premise? What if I change my embeddings model? What index type? How do I do A/B tests? Which cloud platform or model API should I use? Deployment strategies? How can I inject features from my feature platform? Langchain or LlamaIndex or RelevanceAI???

This talk details a distillation of a year+ worth of deploying Redis for these use cases for customers and distill it down into 20 minutes.

//Bio
Sam is a Principal Engineer who helps guide the direction of AI efforts within Redis. Sam assists customers and partners deploying Redis in ML pipelines for feature storage, search, and inference workloads. His background is in high performance computing and machine learning systems.