In this era of Artificial Intelligence, LLMs, and GenAI, it is essential to understand the tools used in this space. LangChain is a framework designed to simplify the development of GenAI applications and is a highly sought-after skill in today’s AI market. This is not about Natural Language Processing theory, but rather an engineering perspective. Companies are looking for skills in Prompting, LLMs, RAG, LangChain, LlamaIndex, and Vector Databases, in addition to classical machine learning, deep learning theory, coding skills, and domain knowledge in areas like computer vision and NLP. However, these GenAI skills are relatively easy to learn and implement. It is good to have knowledge along with a solid machine-learning foundation. Let’s get started:
What is LangChain?
LangChain is an open-source software framework that integrates Large Language Models (LLMs) into domain-specific applications. Released in October 2022, LangChain became popular in the industry and research for its easy-to-use interface. It is designed to simplify the development, productionization, and deployment of LLM-powered applications. It has a set of building blocks for almost every stage of the LLM application lifecycle. Another framework similar to LangChain is LlamaIndex but I am not going to cover LlamaIndex in this blog.
LangChain
LangChain Features and Open-source Libraries:
langchain-core Base abstractions and LangChain Expression Language (LCEL) for composing Chains.
langchain-community Third-party integrations and partner packages for extensibility such as langchain-openai , lanchain-anthropic , etc.
langchain Chains, Agents, and retrieval strategies for cognitive architecture.
LangGraph: multi-actor applications with LLMs by modeling steps as edges and nodes in a graph.
LangServe: Deploy LangChain Chains as REST APIs.
LangSmith: Platform to debug, test, evaluate, and monitor LLM applications.
There is extensive, detailed documentation on LangChain available on their official site. In this blog, I will only cover high-level, important components and concepts that are sufficient to get started with using LangChain in real-life applications.
Key Concepts and Components
LangChain Components
You can find detailed documentation of key LangChain concepts here
Tool: Tools are utilities designed to be called by a model: their inputs are designed to be generated by models, and their outputs are designed to be passed back to models. Tools are needed whenever you want a model to control parts of your code or call out to external APIs.
Output Parser: Output parsers are classes that help structure language model responses. They are responsible for taking the output of an LLM and transforming it into a more suitable format.
Text Splitter: Text splitters divide a document or text into smaller chunks or segments. LangChain has a number of built-in document transformers that can split, combine, and filter documents.
Prompt: LangChain provides tooling to create and work with prompt templates. Prompt templates are predefined recipes for generating prompts for language models
Model: Traditional LLMs
Chat Model: Language models that use a sequence of messages as inputs and return chat messages as outputs, These are traditionally newer models. Chat models support the assignment of distinct roles to conversation messages, helping to distinguish messages from the AI, users, and instructions such as system messages.
Embeddings: Embeddings class to provide embeddings/vectors for a given document. It provides a standard interface for different embedding model providers such as HuggingFace, OpenAI, etc.
Retriever: Retrievers accept a string query as input and return a list of Documents as output. LangChain provides several advanced retrieval types and also integrates with many third-party retrieval services.
Document Loader: Provides a Load method for loading data as documents from a source.
Vector Store: A vector store stores embedded data and performs vector search. Embedding and storing embedding vectors is one of the most common ways to store and search over unstructured data. They are also used in RAG applications.
Index: A data structure that organizes and stores data to facilitate quick and efficient searches.
Agents: Agents are the decision-making components that decide the plan of action or process.
Chains: They are sequences of calls, whether to an LLM, a tool, or a data preprocessing step. They integrate various components into a user-friendly interface, including the model, prompt, memory, output parsing, and debugging capabilities.
Memory: This feature records past interactions with a language model, providing context for future interactions.
Callbacks: LangChain provides a callbacks system that allows you to hook into the various stages of your LLM application. This is useful for logging, monitoring, and streaming.
LangChain and Retrieval Augmented Generation (RAG)
Retrieval-augmented generation (RAG) is an effective technique for tackling one of the key challenges faced by Large Language Models (LLMs): hallucinations. By incorporating external knowledge sources, RAG systems enable LLMs to access relevant, factual information during the generation process. This results in outputs that are more accurate, reliable, and contextually appropriate. LangChain offers useful abstractions for building RAG systems. With its retrieval components, developers can seamlessly integrate external data sources, such as documents or databases, into their LLM-driven applications. This allows models to retrieve and utilize pertinent information during generation, leading to more precise and reliable outputs. I have shown an example proof of concept of using LLMs with RAG on Databricks that can be found in this blog.
Prompt Template
A prompt template consists of a string template. It accepts a set of parameters from the user that can be used to generate a prompt for a language model. Example PromptTemplate can be found here.
Example Code
I am going to show you a few simple and common example use cases using LangChain. They are mainly focused on the Summarization chain and Question-Answer chain. You’ll need to have free credits in your OpenAI account. You can also use models from Meta or Anthropic.
You will need to install LangChain on your system by simply using pip install langchain it in your Python environment. You can find more details here.
You will need to have an OpenAI account. I am using one OPENAI_API_KEYthat is stored in environment variablesin my system. It is easy to generate a key from an OpenAI account. You can also externally define your key as a string as shown below in code. Some of the code snippets are copied from docs.
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
chat = ChatOpenAI(model="gpt-3.5-turbo-1106", temperature=0.2)
prompt = ChatPromptTemplate.from_messages(
[
(
"system",
"You are a helpful assistant. Answer all questions to the best of your ability.",
),
MessagesPlaceholder(variable_name="messages"),
]
)
chain = prompt | chat
chain.invoke(
{
"messages": [
HumanMessage(
content="Translate this sentence from English to French: I love programming."
),
AIMessage(content="J'adore la programmation."),
HumanMessage(content="What did you just say?"),
],
}
)
OUTPUT
AIMessage(content='I said "J\'adore la programmation," which means "I love programming" in French.')
### Summarization Chain Example
from langchain.chains.summarize import load_summarize_chain
from langchain_community.document_loaders import WebBaseLoader
from langchain_openai import ChatOpenAI
loader = WebBaseLoader("https://lilianweng.github.io/posts/2024-04-12-diffusion-video/")
docs = loader.load()
llm = ChatOpenAI(temperature=0, model_name="gpt-3.5-turbo-1106")
chain = load_summarize_chain(llm, chain_type="stuff")
chain.run(docs)
OUTPUT
The blog post by Lilian Weng titled "Scaling Diffusion Models for Video Generation" (April 12, 2024) explores how diffusion models are adapted to create high-quality video content. It discusses techniques like frame interpolation, temporal consistency, and how latent space and attention mechanisms are used to maintain coherence across frames. The post also highlights challenges like long-term dependencies and introduces advanced architectures that scale diffusion models to handle the complexity of video data. It offers insights into advancements in generative AI for video creation.
For more details, visit: Lilian Weng's post.
### Question-Answer Chain Example
from langchain_community.chat_models import ChatOpenAI
from langchain.chains import LLMChain
from langchain_core.prompts import PromptTemplate
OPENAI_SECRET = "your_openai_key" #if not stored in env variable
prompt = PromptTemplate(template="Question: {question}\nAnswer:",
input_variables=["question"]) # PromptTemplate
llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0, api_key=OPENAI_SECRET) #Call LLM
chain = LLMChain(llm=llm, prompt=prompt) #Create Chain
chain.run("What is Halloween?") #Chain Run
OUTPUT
Halloween is a holiday celebrated annually on October 31, originating from ancient Celtic festivals like Samhain. It marks the end of the harvest season and the beginning of winter, historically seen as a time when the boundary between the living and the dead thins. Modern Halloween traditions include dressing in costumes, trick-or-treating, carving pumpkins into jack-o'-lanterns, and attending parties. It's a fun, spooky celebration known for embracing themes of ghosts, witches, and the supernatural.
Note: This code was developed in version 0.3.4 and some of the LangChain components may become redundant or deprecated in future releases.
Top comments (0)