Prompt Chaining Introduction and Coding Tutorials

Have you encountered a situation where an LLM might not be giving you your expected outputs? Are you trying to enhance your prompts with a tried and tested prompt engineering technique? If the answer to both these questions is a ‘yes,’ you’re in the right place.

One of the best ways to tackle bigger tasks is to break them down and solve them in smaller chunks. The same approach is applied when working with LLMs. This prompt engineering technique is known as prompt chaining.

This article explores the key concepts and best practices in prompt chaining.

Summary of prompt chaining concepts

Why is prompt chaining important?

Without context, a conversation could become difficult to understand or respond to. The same applies to LLMs, where introducing the context and directing their reasoning enhances their performances. Prompt chaining is a technique in conversational AI that maintains context over a series of prompts and systematically guides the LLM. 

In simple terms, prompt chaining is breaking down larger prompts into smaller, sequential ones. LLM response from the first small prompt is used to enhance input in the next small prompt, guiding the LLM to perform the more complex task accurately. It lets you maintain context throughout a conversation thread with your LLM. 

Overview of how prompt chaining works

Prompt chaining allows AI models to conduct more meaningful, context-aware dialogs over extended interactions. It enables developers and end-users to improve the quality, controllability, and customization of every response generated by the AI model. We give some example benefits below.

Limited context-length

Every model has a limit on the input length it can process in a single prompt. For complex scenarios, providing all your instructions in one go becomes challenging. This is one of the main reasons why prompt chaining exists. The model architecture and design determine the input limit, but our prompts can be changed and tweaked to form a prompt chain that retains context over a series of inputs. 

Context hallucination

There are some complex problems where the probability of the initial solution to derail or divert the direction of the conversation is high. Your prompt might be dependent on the model’s previous output. If the model is inaccurate or inconsistent with the context set, it results in hallucinations (imagined false outputs) by the LLM. Maintaining context via a prompt chain overcomes this significant challenge.

Fault analysis

Prompt chaining makes fault control, detection, and correction much more accessible. A prompt chain indirectly isolates the problem statement into multiple sections. This makes root cause analysis faster as you can quickly find the prompt creating the problem.

Prompt chaining basics

At its core, prompt chaining utilizes tokenization and embeddings in each prompt in the chain. This might not be visible to the end user, but gaining an overarching idea of these concepts enhances the understandability of the remainder of the article.

Powering data engineering automation for AI and ML applications

Learn how Nexla helps enhance LLM models

• 
  
  
  Enhance LLM models like GPT and LaMDA with your own data
• 
  
  
  Connect to any vector database like Pinecone
• 
  
  
  Build retrieval-augmented generation (RAG) with no code

Tokens

Machines understand and operate with numerical data, but we interact with LLMs as raw text. We use tokens to bridge the gap between raw text and numerical data. Tokens can be represented as individual characters, words, or punctuation. The process of converting raw text into tokens is called tokenization.

Tokens transform complex, variable-length text into a uniform format that the model can understand and process.

Vector database

After tokenization, each token is converted to a numerical representation called a vector using embedding models. An embedding represents a mapping of a discrete categorical variable to a vector of continuous numbers. The resulting vectors are then stored in a vector database, allowing easy accessibility and retrieval.

Image showing data to vector conversion

Preparing, transforming, and loading the vectors in a vector database can be quite complex if not carried out properly. You need to use tools that automatically convert data into vector format. Vectors are key components of your prompt chaining stack, so you need robust and reliable tools for conversion. Nexla is a no-code data integration platform that lets you move data from any source to any vector database with just a few clicks.

Prompt chaining tutorial in Langchain

Langchain is a framework for developing applications using LLMss. It is a powerful tool for creating sophisticated prompt chains. To get started, set up an environment with the Langchain library in Python. Next, outline the sequence of your prompts and the interaction flow. Feel free to pick prompts from prompt templates developed by the Langchain community. 

After creating your prompts, define the sequence of your prompts in a chain. Before running the prompt chain, you can add a persona, style, and tone via functions. Test and refine your prompts to create an effective prompt chain. 

Let’s look at the steps in the code.

Step 1 – Environment setup and dependency import

from langchain_openai import ChatOpenAI
from langchain_core.prompts import PromptTemplate
from langchain_core.runnables import RunnablePassthrough, RunnableLambda
from langchain_core.output_parsers import StrOutputParser
import os

# Set up your OpenAI API key
os.environ["OPENAI_API_KEY"] = ""

We are using langchain_openai and langchain_core modules, which are part of the LangChain library (version 0.1.0 or later). We are importing:

• ChatOpenAI: Wrapper for OpenAI’s chat models (e.g., GPT-3.5-turbo, GPT-4).
• PromptTemplate: Dynamic prompt creation with variable substitution.
• RunnablePassthrough and RunnableLambda: Core components for building sequential chains in LangChain.
• StrOutputParser: Ensures consistent string output from LLM responses.
• API Key: Stored as an environment variable for security. *In production, consider using a secure secret management system.

Step 2 – Initialize the language model and parser

# Initialize the language model
llm = ChatOpenAI(temperature=0.7)

# Use StrOutputParser to get string output from LLM
output_parser = StrOutputParser()

ChatOpenAI(temperature=0.7):

• Temperature controls randomness in output. Range: 0.0 – 1.0.
• 0.7 balances creativity and coherence. Adjust as needed.

StrOutputParser():

• Ensures consistent string output, simplifying downstream processing.
• Helpful when dealing with varied LLM response structures.

Step 3 – Define the prompt templates.

idea_prompt = PromptTemplate(
    input_variables=["industry", "target_market"],
    template="Generate an innovative product idea for the {industry} industry, targeting {target_market}. Provide a brief description in 2-3 sentences."
)

market_analysis_prompt = PromptTemplate(
    input_variables=["product_idea", "target_market"],
    template="Conduct a brief market analysis for the following product idea: {product_idea}\nConsider the target market: {target_market}\nProvide insights on market size, competitors, and potential challenges."
)

feature_prompt = PromptTemplate(
    input_variables=["product_idea", "market_analysis"],
    template="Based on the product idea: {product_idea}\nAnd the market analysis: {market_analysis}\nOutline 3-5 key features or functionalities that would make this product competitive and appealing to the target market."
)

development_plan_prompt = PromptTemplate(
    input_variables=["product_idea", "features", "market_analysis"],
    template="Create a high-level product development plan for: {product_idea}\nKey features: {features}\nConsidering the market analysis: {market_analysis}\nOutline the main stages of development, potential timelines, and key considerations for bringing this product to market."
)

In the code above, each prompt is designed for a specific step in the product development process. The template is the actual prompt text with placeholders for variables. input_variables is the list of variables to be substituted in the template. Their values can be taken from the user input.

Modifying these templates allows easy customization of the AI’s focus and output style.

Step 4 – Create individual chain nodes with print functions

def create_chain_with_print(name, chain):
    return RunnableLambda(lambda x: print(f"\n{name} Output:\n{chain.invoke(x)}") or chain.invoke(x))

idea_chain = create_chain_with_print("Product Idea", idea_prompt | llm | output_parser)
market_analysis_chain = create_chain_with_print("Market Analysis", market_analysis_prompt | llm | output_parser)
feature_chain = create_chain_with_print("Key Features", feature_prompt | llm | output_parser)
development_plan_chain = create_chain_with_print("Development Plan", development_plan_prompt | llm | output_parser)

In the code above, create_chain_with_print is a higher-order function that adds logging to any chain. RunnableLambda wraps a Python function to make it compatible with LangChain’s runnable interface.

The | operator in idea_prompt | llm | output_parser:

• Pipes the output of each step to the input of the next.
• Equivalent to output_parser(llm(idea_prompt(x))).

This pattern allows for easy insertion of additional processing steps if needed.

Step 5 – Assemble the overall prompt chain

product_development_chain = (
    RunnablePassthrough.assign(
        product_idea=idea_chain
    )
    | RunnablePassthrough.assign(
        market_analysis=lambda x: market_analysis_chain.invoke({
            "product_idea": x["product_idea"],
            "target_market": x["target_market"]
        })
    )
    | RunnablePassthrough.assign(
        features=lambda x: feature_chain.invoke({
            "product_idea": x["product_idea"],
            "market_analysis": x["market_analysis"]
        })
    )
    | RunnableLambda(lambda x: development_plan_chain.invoke({
        "product_idea": x["product_idea"],
        "features": x["features"],
        "market_analysis": x["market_analysis"]
    }))
)

RunnablePassthrough.assign adds new key-value pairs to the chain’s running dictionary, allowing the accumulation of results from each step.

Lambda functions invoke each chain with the appropriate inputs from previous steps, enabling dynamic input preparation for each step. The chain structure ensures that each step has access to all previous outputs, allowing for easy addition or modification of steps in the process.

Step 6 – Execute the prompt chain

print("\nStarting the product development strategy generation process...\n")
result = product_development_chain.invoke({
    "industry": "Data and AI",
    "target_market": "Data engineers and AI research scientists"
})

print("\nFinal Result (Development Plan):")
print(result)

The invoke function runs the entire chain with the provided initial inputs. The chain processes each step sequentially, passing accumulated data between steps.

Prompt chaining tools for everyone

The following tools are useful for non-coders.

PromptChainer

Promptchainer.io is an emerging prompt-chaining tool designed to streamline the process of creating and managing prompt chains. It simplifies the creation of prompt chains with a visual builder, enables advanced context management, and supports various API integration capabilities. The visual builder offers user-friendly drag-and-drop functionality, making it accessible to all users irrespective of their technical know-how. It is one of the best tools to start your prompt chaining journey. 

PromptLayer

Promptlayer.com is a cutting-edge tool used to develop and maintain prompts. It offers robust capabilities for creating, organizing, and optimizing prompts. Promptlayer lets you create new prompts in a systematic and operationally efficient way and test prompts with the help of human and AI graders.

Brainglue

Brainglue is a flexible AI assistant built to augment your strategic thinking. You can create a chain that represents your prompt sequence. Brainglue lets you define a persona as a chain field. You can specify the persona’s name and tone and define traits like empathy or technical knowledge. This persona sets the tone and influences all of the model’s responses. This makes it easy to develop a prompt chain that maintains context, style, and tone.

Recommendations

Some prompt chaining best practices are given below.

Frame well-defined prompts

LLMs work on the concept of next-token prediction and are prone to hallucination. If allowed to assume, they do so creatively and effortlessly generate output in a completely different tangent.

Hence, it is essential to have structured and precise prompts. Be specific in your instructions to minimize the element of surprise in your model’s output. Setting a descriptive context is highly beneficial if you have clearly defined output expectations.

Add self-diagnosing prompts

This is similar to applying auto-correct functionality to LLM output. There is always a probability of the model generating an output that is incorrect and inconsistent with your prompts. Hence, adding a section in your prompt to self-diagnose its response adds a consistency-check layer. 

The simplest way to start with this is to instruct the model to debug its generated output and ensure the output is accurate. This is possible since LLMs can reason to a certain extent.

Avoid cognitive overload

Overly detailed prompts may cause adverse results since all LLMs lack strong cognitive capabilities and can falter without strong reasoning. This is especially true for smaller models that struggle to maintain coherence and relevance when tasked with complex or overly specific instructions. If, at any point in time, a model does not work well with a prompt chain, it is most likely confused. Start with a zero-shot prompt (a single prompt), and once that is successful, look at chaining your prompts, if required. 

Conclusion

Prompt chaining is one of the best prompt engineering techniques, and it has been thoroughly tried and tested. Its systematic approach makes it ideal for dealing with complex prompts in LLMs. If implemented effectively, it can significantly improve LLM output.