Building a Chatbot with Chat History using Langchain JS/TS: A Step-by-Step Guide

In the dynamic landscape of 2024, the creation of chatbots tailored to specific data has become a trend. With the invention of numerous Language Models (LLMs), including both open and closed sources, web developers are increasingly inclined towards integrating chatbots into their projects. However, the problem is to create an optimized chatbot that not only responds intelligently but also maintains a contextual chat history, similar to OpenAI's outstanding capabilities.

What will we be building?

This blog aims to guide you through the process of building a powerful backend-only chatbot. This chatbot will not only respond based on prompts and custom data but will also have access to a valuable chat history. To achieve this, we will leverage the capabilities of Langchain JS/TS SDK, paired with the OpenAI API key and the Qdrant vector database.

For a more in-depth exploration of creating chatbots with custom data, refer to my previous blog on the topic -[Link]

Chat History-

A chatbot equipped with chat history has the remarkable ability to remember and refer back to previous interactions.

For instance, consider asking ChatGPT for your name and then asking it again in the next question. It retains and incorporates this information into its responses. Now, contrast this with a scenario where you directly call the OpenAI API - it lacks the contextual reference from past chats.

Let's start building✨

1. Install the Required Packages:

Begin by installing the necessary packages using the following command:

npm i langchain @langchain/openai @langchain/community

1. Import Dependencies:

Import the essential dependencies at the beginning of your file to set the stage for the implementation:

import { ChatOpenAI } from "@langchain/openai";
import { createStuffDocumentsChain } from "langchain/chains/combine_documents";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { OpenAIEmbeddings } from "@langchain/openai";
import { createRetrievalChain } from "langchain/chains/retrieval";
import { createHistoryAwareRetriever } from "langchain/chains/history_aware_retriever";
import { QdrantVectorStore } from "@langchain/community/vectorstores/qdrant";
import { MessagesPlaceholder } from "@langchain/core/prompts";
import { HumanMessage, AIMessage } from "@langchain/core/messages";
import "dotenv/config";

1. Declare Chat History:

Define the chat history, a crucial component for the chatbot to learn and respond intelligently:

 const chatHistory = [
        new HumanMessage(
          "My name is Abir Dutta. I am a software engineer expertise in MERN stack and Generative AI. I am looking for a job. Can you help me?"
        ),
        new AIMessage("Hi Abir, I can help you in many ways."),
      ];

You can fetch the messages from the database for the tutorial, I am hardcoding the messages. I will suggest adding the top 12 messages (6 AI messages and 6 human messages) if you're using the free API of GPT-3.5-turbo.

Playing with Prompts

Prompts play a very important role in defining the output of any LLM. So we have to adjust the prompt as per our needs and add the chat history to the prompt itself.

We will have two prompts.

1. Retrieval prompt: the prompt used for retrieving specific chunks of relevant documents from a vector database.

2. Main prompt: the prompt that we will be giving open AI at every call.

 const retrieverPrompt = ChatPromptTemplate.fromMessages([
    new MessagesPlaceholder("chat_history"),
    ["user", "{input}"],
    [
      "user",
      "Given the above conversation, generate a search query to look up in order to get information relevant to the conversation",
    ],
  ]);  

const prompt = ChatPromptTemplate.fromMessages([
        [
          "system",
          `You are a helpful assistant.Answer user's question based on following context : {context}, in a simple way. 
          Don;t try to give imaginary answers, if you don;t know something just say, sorry I don't know.`,
        ],
        new MessagesPlaceholder("chat_history"),
        ["user", "{input}"],
      ]);

Retrieval Chains:

The second most significant thing is obtaining documents from vector databases.

There are several ways to do this, including as

• Using native SDKs for the respective database.

• Using the vector databases Langchain wrapper.

• Using predefined Langchain retrieval chains.

  We'll proceed with the third alternative, which is to employ predefined retrieval chains from Langchain.

  Begin with designing the model -> followed by the retriever -> and finally the retrieval chain.

// put the openai api key in a dotenv file using the name OPENAI_API_KEY 
const model = new ChatOpenAI({
    modelName: "gpt-3.5-turbo-0125",
    temperature: 0.7,
  });
// getting the collection where the data is stored from QdrantDB.
 const vectorStore = await QdrantVectorStore.fromExistingCollection(
    new OpenAIEmbeddings(),
    {
      url: process.env.QDRANT_URL,
      collectionName: "lsoxnt98",
    }
  );

// Creating the retriever
 const retriever = vectorStore.asRetriever(
    {
        k: 3,
    }
  );
//At last creating the chain -
const retrieverChain = await createHistoryAwareRetriever({
    llm: model,
    retriever,
    rephrasePrompt: retrieverPrompt,
  });

Here in the retriever the k means the number of relevant chunks you need to fetch from Vector DB.

Conversational Chains

The following stage is to build the main chain, which will be used to send calls to open AI and generate responses. For that-

• First declare the chain

const chain = await createStuffDocumentsChain({
    llm: model,
    prompt: prompt
  });

• Then declare the conversational chain

  const conversationChain = await createRetrievalChain({
    combineDocsChain: chain,
    retriever: retrieverChain,

  });

Invoking the Chain

• Conclude the implementation by making a call to OpenAI with all the configured chains and data-

const response = await conversationChain.invoke({
    chat_history: chatHistory,
    input: "Which tech stack used in video?",
  });

Now explaining the terms:

1. The createStuffDocumentsChain actually stuffs all the relevant chunks of documents fetched from QdrantDB into a single call.

2. The createRetrievalChain will actually create a chain for retrieving relevant chunks of data from vector databases according to the user input query.

3. The conversation chain will actually run the open AI call, using all the other chains and data. The input is the user query that you can get from the front end.

Conclusion

In conclusion, creating a chatbot with chat history with Langchain JS/TS is an interesting trip that allows web developers to design intelligent and context-aware conversational bots. The step-by-step guide takes you through the entire procedure, from installing the necessary packages to creating smart prompts and implementing retrieval chains. The importance of chat history is shown as the chatbot learns and answers intelligently by referencing previous interactions. Developers can utilize the Langchain JS/TS SDK, OpenAI API, and Qdrant vector database to build a backend-only chatbot that improves user experiences with tailored and context-rich responses. Dive into the world of generative AI and discover the possibility for designing chatbots that genuinely comprehend and recall user interactions.

Have fun coding!

Important Links

• Langchain JS/TS - [Docs]

• Qdrant Vector Database - [Docs]

• OpenAI API - [Link]

Still have any questions/doubts? Reach out to me -

• Linkedln

• Twitter

• Github