AI is rapidly changing the world around us, and language processing is no exception. Langchain is a new AI framework that is making it easier to build powerful language processing applications.

Langchain is built on top of large language models (LLMs), which are AI models that have been trained on massive datasets of text. LLMs can be used for a variety of tasks, including text generation, translation, summarization, and question answering. Langchain makes it easy to chain together multiple LLMs to create more complex applications. For example, you could use Langchain to build a chatbot that can translate languages, summarize text, and answer questions. Langchain is still under development, but it has the potential to revolutionize the way we interact with computers. With Langchain, we can build applications that can understand and respond to our natural language in a way that has never been possible before.

Langchain can be used to build powerful language processing applications like

  • Chatbots: Langchain can be used to build chatbots that can understand and respond to natural language. These chatbots can be used for a variety of purposes, such as customer service, education, and entertainment.
  • Translation: Langchain can be used to build translation applications that can translate text from one language to another. These applications can be used for a variety of purposes, such as business, travel, and education.
  • Summarization: Langchain can be used to build summarization applications that can summarize long pieces of text into shorter, more concise versions. These applications can be used for a variety of purposes, such as research, news, and education.
  • Question answering: Langchain can be used to build question answering applications that can answer questions about a variety of topics. These applications can be used for a variety of purposes, such as research, education, and customer service.

Langchain is a powerful new AI framework that is making it easier to build powerful language processing applications. With Langchain, we can build applications that can understand and respond to our natural language in a way that has never been possible before.

You can use OpenAI or HuggingFace to build out your own, but for this example we will demonstrates how to ingest custom text and query it for questions.

I recommend spinning off an AWS EC2 instance. To start with, go ahead and download the 2 models files and place them in models directory

Let take a look at a simple example of how to use Langchain to build a question answering system. The code first imports the necessary modules from Langchain. Then, it defines a list of texts that will be used to train the system. Next, it creates a Chroma vector store and a LlamaCppEmbeddings embedding model. Finally, it calls the ingest() function to add the texts to the vector store and persist the changes and then called query() that takes a question as input and returns the answer to the question.

Firstly import the following modules

from langchain.document_loaders import TextLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.vectorstores import Chroma
from langchain.embeddings import LlamaCppEmbeddings
from sys import argv
from langchain.chains import RetrievalQA

from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain.llms import GPT4All

For this example, we will ingest the below text into LLM and then do a simple Q&A

texts = [
    "Security by design is a concept that emphasizes the importance of incorporating security measures into the design and development of systems, applications, and products from the very beginning of the process. It aims to integrate security as a fundamental part of the development lifecycle, rather than treating it as an afterthought or add-on.",
    "The idea behind security by design is to minimize the risk of security vulnerabilities and threats by designing products and systems with security in mind from the start. This means considering security requirements and best practices at every stage of the design and development process, from the initial planning and architecture design to the final testing and deployment."
]

Define where the data will be persisted and declare your embedding

persist_directory = 'db_1'
llama = LlamaCppEmbeddings(model_path="./models/ggml-model-q4_0.bin",n_ctx = 1024, n_threads=64, n_batch=1024)
db = Chroma(persist_directory=persist_directory, embedding_function=llama)

There are 2 functions ingest and query

def ingest():
    db.add_texts(texts)
    db.persist()
    print("ingestion completed")

def query(q): 
    retriever = db.as_retriever(search_kwargs={"k": 1})
    callbacks = [StreamingStdOutCallbackHandler()]

    llm = GPT4All(model="./models/ggml-gpt4all-j-v1.3-groovy.bin", backend='gptj', callbacks=callbacks, verbose=False)
    qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever, return_source_documents=True)

    print("sending query")
    res = qa(q)
    answer, docs = res['result'], res['source_documents']
    print(answer)
    return answer

Link to Github Repo