Replit + Weights & Biases: Building a RAG Bot

Guest post by Weights & Biases

Weights & Biases (W&B) provides AI developers with powerful tools to build better models faster and is used by almost every top LLM research lab today.

To help support internal and external users, W&B built WandBot – a question-answering bot built on the robust llama-index library and the intelligence of OpenAI's GPT-4. By referencing documentation, wandb source code, and code examples, WandBot helps users onboard to W&B faster, and experienced users solve more advanced issues.

There is even a custom GPT available for Wandbot:

WandBot: Behind the scenes of Weights & Biases' AI-powered assistant

W&B’s WandBot utilizes retrieval augmented generation (RAG) with a FAISS vector store, Cohere embeddings, and OpenAI’s GPT-4, ensuring efficient and accurate responses to user queries. Integrated with Discord, Slack, Zendesk, and ChatGPT’s GPTs, WandBot can be served quickly across commonly used collaboration and support platforms. For a full technical report on how WandBot was built, check out this blog post on Weights & Biases. You can also try WandBot by visiting wandb.me/gpt or via the `#wandbot` channel in the W&B Discord.

"Our goal was to create an interface where users could converse with our documentation and examples as naturally as they would with a colleague," explains Bharat Ramanathan, a key developer behind WandBot. "This meant ensuring wandbot could not only just respond naturally to users but also retrieve knowledge like an expert. Ensuring that WandBot also acts sensibly was critical, such as telling the user that it doesn’t know the answer to a query instead of trying to hallucinate an answer."

The bot can be spun up and hosted on Replit in just a few clicks.

Deploying on Replit

Replit offers a dynamic environment for hosting applications like WandBot. Its advantages include ease of setup, robust community support, and seamless integrations for running the support bot on Slack, Discord, and Zendesk. Replit Deployments scale automatically based on resource needs to ensure they’re utilized most efficiently.

“We’ve been using WandBot with Replit for almost a year now,” says Morgan McGuire, another contributor to the project. “Replit Deployments is a really great feature. It is easy to use, has professionalized our WandBot deployment, and made it much more stable.”

Continuous improvement

WandBot's evolution is driven by meticulous logging and analysis with W&B Tables; you can see all public queries to and responses from WandBot logged in this public W&B project here. For example:

Continual improvement is in WandBot's DNA. "We also implemented a knowledge ingestion system where WandBot regularly adds new and updated documentation and code examples, making sure its responses are always up to date with the latest W&B features we release," says the Weights & Biases team. Here, for example, you can see a recent report from this data ingestion process:

Installation and usage

Setting up WandBot is straightforward with the help of poetry for dependency management. By setting the necessary environment variables and running a few commands, you can activate the Q&A bot on your chosen platform.

To get started, fork the Repl here: https://replit.com/@MorganMcGuire/wandbot-en?v=1

Real-world application

WandBot isn't just theoretical. It's a living example of a real-world RAG application in production, handling W&B queries for customers, and is fully open-sourced on GitHub under the Apache 2.0 License. The team at W&B is keen to keep improving the system and is taking community contributions and feedback to the codebase.

WandBot is a testament to W&B's mission to make the best tools for AI developers by ensuring they can onboard to the product as fast as possible. You can try WandBot in the `#wandbot` channel of the W&B Discord, and the team is now accepting contributions to the GitHub repo to improve this support bot system for everyone.