Building a Simple RAG System Application with Rust

In this tutorial, we will be building a simple Chatbot to query, your documents on your local machine, using GEMINI for embeddings. Tauri for app development and Rig as our SDK for Gemini.

Retrieval-augmented generation (RAG) systems enhance large language models (LLMs) by grounding them with an external, often domain-specific, knowledge base.

This means instead of solely relying on the LLM's pre-trained knowledge, a RAG system first retrieves relevant information from a source like documents, databases, or web content, and then generates a response based on both the prompt and the retrieved context.

This makes the LLM more accurate, reliable, and capable of answering questions about niche or rapidly changing information. This tutorial walks you through building a simple desktop chatbot that queries your local documents using GEMINI for embeddings and response generation.

We'll leverage the power of Rust for its performance and safety, ensuring efficient processing of documents and secure handling of data. Tauri will be used to package our application for desktop deployment, providing a cross-platform, lightweight alternative to heavier frameworks. We'll employ Rig.rs, a Rust SDK for LLMs, to streamline our interactions with the GEMINI API, allowing us to easily generate embeddings and completions.

Finally, we'll be using GEMINI (via Rig.rs) for both generating embeddings of our documents and for generating the final answer to the user's prompt, due to its balance of performance and accessibility.

This tutorial is targeted at developers with some familiarity with Rust and basic frontend concepts who want to explore building their own RAG applications. While no prior experience with LLMs or RAG systems is assumed, a basic understanding of these concepts will be beneficial.

So we’ll build a simple RAG system using the Rig.rs crate and integrate the Gemini model provider. We’ll create a system that extracts text (for example, from PDFs or other documents), builds embeddings, stores them in an in‑memory vector store, and then uses a Gemini-powered agent to answer user queries based on the retrieved context.

• Learn more about the fundamentals of RAG here.

• Learn more about Rig.rs here

• Here is the github repository

Setting up Tauri

We would need to set up Tauri as that would be what we would use to bootstrap the desktop application.

Install and Setup Tauri

To set up and install Tauri view its documentation for your computer specifications. View Tauri Documentation for installation here.

Setting up our Tauri Project

Now we have Tauri and its dependencies installed we have to set up the project. Run the command.

cargo create-tauri-app

Fill in all the necessary fields as requested. Use the image below as a guide.

Notice we named our project DocConvo. Remember to change the directory to your New project

• Install Tailwind CSS: For our styling, we would be using tailwinds. Do the following to install it in our project.

  • Run the command to install Tailwind CSS:

    bun install tailwindcss @tailwindcss/vite

  • Configure the Vite plugin: Add the @tailwindcss/vite plugin to your Vite configuration.

    import { defineConfig } from 'vite'
    import tailwindcss from '@tailwindcss/vite'
    export default defineConfig({
      plugins: [
        tailwindcss(),
      ],
    })

  • Import Tailwind CSS: Add an @import to your App.css file that imports Tailwind CSS.

    @import "tailwindcss";

• Install Lucide React for Icons and Markdown For rendering Markdown AI response

  Run the below command to install Lucide React.

  bun install lucide-react

  Run the command below to install Markdown.

  bun install Markdown

• Confirm Project Runs and Clean Setup Templates: Now we can confirm our setup works by running the below command to start the dev build for Tauri.

  bun run tauri dev

  This might take some time so sit tight and allow rust to do its build thing. You might want to grab a coffee. Once this is done, a new window should be created by your Tauri project.

  To clean the templates, clear all the imports and delete all the contents of your App.tsx file. Replace it with this:

  
  import "./App.css";
  
  function App() {
    return (
      <div></div>
    )
  }
  
  export default App;

  This would trigger a rebuild, and you should find out that the front end has changed on your dev window.

  You have now successfully set up your dev environment, at least for the front end.

• Setting Up the Backend Env: Now we have the front end set up we have to set up the rust backend for the App, where would write all the commands and do the core work. Now let's install all our required dependencies by running:

  cd src-tauri // to change into your rust dir with the Cargo.toml file
  cargo add rig-core
  cargo add tokio --features full
  cargo add anyhow
  cargo add pdf-extract
  cargo add dotenv
  cargo add walkdir
  bun tauri add dialog
  bun tauri add fs

Here’s a brief explanation of each dependency:

• rig-core: The main Rig library for building LLM applications

• tokio: An asynchronous runtime for Rust

• anyhow: Crate for flexible error handling

• pdf-extract: Carte used to extract text from PDF files

• dotenv : For loading env variables form .env files at compile time

• walkdir : For walking directory

• dialog plugin : Tauri Plugin for accessing native dialogs

• fs plugin : Tauri plugin for accessing the native file system

This would take some time to add to the project so be patient. Next, Create an .env file where you would save your API keys. In your env file set your gemini API key like this:

GEMINI_API_KEY="YOUR_KEY"

Visit here to get your GEMINI API KEY

After this, make sure to re-run the project to confirm everything works fine.

FRONT END STRUCTURE

Now let's build the front end. It is a simple text box with a button atop to allow you to select folders to chat with. In these folders, any pdf or txt files would be processed for querying.

This is what the wireframe should look like:

Now you get the idea, if you can just skip to code it out. This is what our front end now looks like after some designing.

To get to this update your App.tsx with this.

import { useEffect, useRef, useState } from "react";
import "./App.css";
import { open } from '@tauri-apps/plugin-dialog';
import { invoke } from "@tauri-apps/api/core";
import {Bot, CircleUser} from "lucide-react";
import Markdown from 'react-markdown'

interface Message {
  person: "user" | "system",
  message: string
}

function App() {
  const [dirPaths, setDirPaths] = useState<undefined | string[]>();
  const [messages, setMessages] = useState<undefined | Message[]>();
  const inputComp = useRef<HTMLInputElement | null>(null);
  const messageContainer = useRef<HTMLDivElement | null>(null);

  useEffect(() => {
    messageContainer?.current?.lastElementChild?.scrollIntoView({ behavior: "smooth" });
  }, [messages])


  const addFilePath = async () => {
    let inputdirPaths = (await open({
      multiple: true,
      directory: true,
    }))?.filter(x => !dirPaths?.includes(x));

    // invoke backend command to load the files into vector database
    if (inputdirPaths) {
      try {
        dirPaths ? setDirPaths([...dirPaths, ...inputdirPaths]) : setDirPaths(inputdirPaths);

        await invoke("index_folders", { "folders": inputdirPaths })
      } catch (error) {
        dirPaths?.pop();
      }

    }
  }

  const handleMessage = async (e: React.FormEvent<HTMLFormElement>) => {
    e.preventDefault();
    let text = inputComp?.current?.value.trim();
    let innermessages: Message[] = messages ? [...messages!, ...[{ person: "user", message: text } as Message]] : [{ person: "user", message: text } as Message];
    setMessages(innermessages)
    if (inputComp.current) {
      inputComp.current.value = "";
    }
    // invoke the command to send message
    let response: string = await invoke("prompt", { prompt: text });

    // update the messages with the response
    setMessages(messages => [...messages!, ...[{ message: response, person: "system" } as Message]]);

  }

  return (
    <div className="grid grid-rows-12 w-full h-[100vh] dark:bg-neutral-950 dark:text-white p-2 " onLoad={() => { }}>
      <div className="top row-span-1 w-full h-fit py-2 dark:bg-neutral-900 rounded-full grid grid-cols-12">
        <div className="h-fit flex flex-row  dirtags col-span-9 px-1 my-auto overflow-x-scroll" >
          {
            dirPaths?.map(e => {
              return <span className="text-sm text-gray-300 my-auto px-3 p-2 bg-neutral-800 rounded-full mx-1 cursor-pointer">{e.split("/").pop()}</span>
            })
          }
        </div>
        <div className="col-span-3 flex flex-row-reverse px-2">
          <button className="rounded-full bg-purple-950 px-3 py-2 h-fit my-auto text-purple-200 text-sm " onClick={addFilePath}>Add Folder</button>
        </div>
      </div>
      <div className="center h-full row-span-10 overflow-scroll flex-col" ref={messageContainer} >
        {
          messages?.map(message => {
            return <div className={`px-4 py-2 mx-auto m-2  w-fit flex flex-row ${message.person == "user" ? "mr-0" : "ml-0"}`}>
              {message.person == "system" ? <Bot className="min-w-5" /> : <CircleUser/>}              
              <p className="text-gray-400 px-2"><Markdown>{message.message}</Markdown></p>
            </div>
          })
        }
      </div>
      <form onSubmit={handleMessage} className="buttom row-span-2 my-auto px-2">
        <input ref={inputComp} type="text" placeholder="Start your conversation" className="outline-1 w-full rounded-full outline-neutral-600 shadow-sm shadow-nuetral-300 p-3 px-6 text-sm " />
      </form>
    </div>
  )
}

export default App;

Front-end summary

The front end, allows you to select a folder → calls the index_folder tauri command that vectorizes the contents of this folder.

On prompt from the user, the prompt command is called and the backend uses the closest items in the vector db as context for the completion models Gemini. That is simply how RAG systems work.

Key Tauri Function Calls

1. index_folders: Sends selected folders to the Rust backend for indexing.

2. prompt: Sends user input to the AI and retrieves a response.

Backend Structure

Now we are done with the front end it would be wise to see how the commands are written on the backend, and to be honest, rig.rs makes this easy.

Rig.rs currently has clients for Anthropic, Claude, Gemini, and ChatGPT models, and their API so if you use any of these models this should be relatively the same for you.

Since we would need multiple sessions of vector database we have to be able to persist both the client and the vector database to some sort of state.

Once we have that in the state, we can now move on to implementing the index_folders command.

In Tauri IPC commands are a simple way for the front end and the back end to communicate, so you can leverage some features only available with RUST. Visit Tauri docs to learn more about commands

This index_folder would take a list of folder paths, walk through the dir recursively, and as you could guess from the name embed whatever textual content of the supported documents and save it in the vector database from state.

To learn more about state management in Tauri see

Next, the prompt command would take a prompt string →Get the top 2 closest documents to the prompt, and feed that as context to out GEMINI_COMPLETION_MODEL. Let's dive into code.

Setting up state

Open the file src-tauri/src/lib.rs : this is the entry point to your app. You should see two functions one with signature:

#[cfg_attr(mobile, tauri::mobile_entry_point)]
pub fn run() {...}

And the other with:

#[command]
fn greet(name: String) -> String {...}

Delete the greet command and remove it from the Tauri command handler: You should delete or comment out the function greet, and also remove it from your tauri::generate_handler![] call, found somewhere in your run function.

Setup Global State and Initialize your env variables

Replace your current lib.rs file with the contents of this code block below. We would walk through what you have just done.

// Learn more about Tauri commands at https://tauri.app/v1/guides/features/command

mod commands;
use dotenv;
use rig::providers::gemini;
use rig::vector_store::in_memory_store::InMemoryVectorStore;
use tauri::{
    async_runtime::Mutex,
    Manager,
};

struct AppState {
    vector_store: Mutex<InMemoryVectorStore<String>>,
    client: gemini::Client,
}

#[cfg_attr(mobile, tauri::mobile_entry_point)]
pub fn run() {
    tauri::Builder::default()
        .setup(|app| {
            dotenv::dotenv().ok();
            let vc: InMemoryVectorStore<String> = InMemoryVectorStore::default();
            let client = gemini::Client::from_env();

            let state = AppState {
                vector_store: Mutex::new(vc),
                client,
            };

            app.manage(state);

            Ok(())
        })
        .plugin(tauri_plugin_dialog::init())
        .plugin(tauri_plugin_fs::init())
        .plugin(tauri_plugin_shell::init())
        .invoke_handler(tauri::generate_handler![commands::prompt, commands::index_folders])
        .run(tauri::generate_context!())
        .expect("error while running tauri application");
}

In this piece of code, we declare a structure called AppState, where we store handles to the global store and client. This State is available to all handlers and initialized when the app initiates. We also called the dotenv crate to load all env from our env file into the shell environment.

Now you have an idea of the setup, let's move over to writing the command handlers for indexing folders and handling prompts.

Make your Global Imports:

use anyhow::Context;
use pdf_extract::extract_text;
use rig::{
    completion::Prompt, embeddings::EmbeddingsBuilder,
    vector_store::VectorStoreIndex,
};
use std::{
    fs,
    path::{Path, PathBuf}
};
use tauri::State;

use crate::AppState;
use walkdir::WalkDir;
use rig::providers::gemini::completion::GEMINI_1_5_FLASH;

// You would understand why we need them in the future

Writing the load_folder_contents util function: This function uses WalkDir to Walk through a dir loading only entries that are pdf or txt. Let's look at the code:

pub fn load_folder_contents<P: AsRef<Path>>(dir_path: P) -> anyhow::Result<Vec<String>> {
    let mut contents = Vec::new();

    for entry in WalkDir::new(dir_path) {
        let entry = entry.with_context(|| "Failed to read directory entry")?;
        let path = entry.path();

        if path.is_file() {
            match path.extension().and_then(|ext| ext.to_str()) {
                Some("txt") | Some("srt") | Some("rs") => {
                    let text = fs::read_to_string(path)
                        .with_context(|| format!("Failed to read text file: {:?}", path))?;
                    contents.push(text);
                }
                Some("pdf") => {
                    let pdf_text = extract_text(path)
                        .with_context(|| format!("Failed to extract text from PDF: {:?}", path))?;
                    println!("PDF TEXT \n {:?}", pdf_text);

                    contents.push(pdf_text);
                }
                _ => {
                    // Unsupported file types are ignored
                }
            }
        }
    }

    Ok(contents)
}

Building the index_folders command

This command would take a Vec<PathBuf> and using the earlier defined util load_folder_contents read all the strings from the folders and store the strings in the vector database.

Here is what the code should look like:

#[tauri::command]
pub async fn index_folders(folders: Vec<PathBuf>, state: State<'_, AppState>) -> Result<(), ()> {
    let documents = folders
        .into_iter()
        .filter_map(|f| load_folder_contents(f).ok())
        .collect::<Vec<_>>()
        .into_iter()
        .flatten()
        .collect::<Vec<_>>()
        .join(",");


    let mut vector_store_guard = state.vector_store.lock().await;
    let client = &state.client;

    let model = client.embedding_model("embedding-001");
    let embeddings = EmbeddingsBuilder::new(model.clone())
        .document(documents)
        .unwrap()
        .build()
        .await
        .unwrap();
    vector_store_guard.add_documents(embeddings);
    Ok(())
}

Let us look at an in-detail explanation of what each line does. If you could not figure it out already.

The function scans multiple folders, extracts text, embeds the extracted text using a Gemini model, and stores the embeddings in an in-memory vector store.

Building the prompt command.

In a nutshell, this command should take a prompt to find its k closest neighbor in the vector db and add that as context for a completion model.

Here is what the code may look like:

#[tauri::command]
pub async fn prompt(prompt: String, state: State<'_, AppState>) -> Result<String, ()> {
    let vector_store_guard = state.vector_store.lock().await;
    let client = &state.client;
    let model = client.embedding_model("embedding-001");
    let ind = vector_store_guard
        .clone()
        .index(model.clone())
        .top_n::<String>(&prompt, 5)
        .await
        .unwrap()
        .iter()
        .map(|f| f.2.clone())
        .collect::<Vec<_>>();

    let rag_response = client.agent(GEMINI_1_5_FLASH).preamble("You are a helpful assistant that answers questions based on the given context from documents.").context(&ind.join(",")).build().prompt(&prompt).await.unwrap();

    Ok(rag_response)
}

The function takes a user query, retrieves relevant information from the vector store, passes the data to a Gemini AI agent, and returns the AI-generated response.

Now we have everything setup we can now build our project in dev mode.

Testing the app

Run the following command in your terminal:

bun run tauri dev

The entire project would build, be patient, and let rust do its thing.

If you face any issues down this path just check the code base for the fix or comment on your issues or you could reach out to me personally on my LinkedIn. I believe by now you know how to build a simple RAG system with rust.