← All tutorials
Build LLM Apps

Building LLM Applications

A hands-on tour of the tools you use to turn a raw language model into a real application β€” organized around the three problems you hit as your app grows up.

Problem 1

Too much glue code

Prompts, parsing, tools, retrieval β€” LangChain gives you reusable building blocks.

Problem 2

Logic isn't a straight line

Loops, branches, multiple agents β€” LangGraph gives you stateful orchestration.

Problem 3

It's a black box

Can't see or measure it β€” LangSmith gives you tracing & evaluation.

How to use this page. Read top to bottom β€” each section builds on the last. Code blocks with tabs let you compare options; hit Copy to grab any snippet. There's a short quiz at the end to check what stuck.

Problem 1  LangChain β€” building blocks

Calling a model directly is easy for one prompt. A real feature needs prompt templating, output parsing, tool-calling, memory, and β€” to ground answers in your own data β€” document loading, chunking, embedding, and retrieval. LangChain provides standard, composable pieces so you write that glue once and swap providers with a one-line change.

input→ retrieve→ prompt→ model→ parse

The LCEL pipe (|) composes these into a chain with streaming, batching, and async for free. Here's the same minimal chain across two providers:

chain.py
# pip install langchain-anthropic
from langchain_anthropic import ChatAnthropic
from langchain_core.prompts import ChatPromptTemplate

model  = ChatAnthropic(model="claude-opus-4-8")
prompt = ChatPromptTemplate.from_template("Summarize in one sentence: {text}")
chain  = prompt | model

print(chain.invoke({"text": "..."}).content)

Notice only the import and class name changed β€” the chain is identical. That provider-agnostic interface is LangChain's core value.

2 Β· RAG β€” grounding answers in your data

Retrieval-Augmented Generation feeds the model only the chunks of your documents relevant to a question, so it answers from your data instead of guessing. The pipeline is a straight line β€” no loops β€” which makes it a perfect LangChain chain.

documents→ split→ embed→ store→ retrieve→ answer
rag.py
# load β†’ split β†’ embed β†’ store β†’ retrieve β†’ grounded answer
from langchain_community.document_loaders import TextLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.vectorstores import FAISS
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_anthropic import ChatAnthropic
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser

docs   = TextLoader("kb.txt").load()
chunks = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50).split_documents(docs)
store  = FAISS.from_documents(chunks, HuggingFaceEmbeddings(
             model_name="sentence-transformers/all-MiniLM-L6-v2"))
retriever = store.as_retriever(search_kwargs={"k": 4})

prompt = ChatPromptTemplate.from_template(
    "Answer using ONLY this context:\n{context}\n\nQuestion: {question}")
model  = ChatAnthropic(model="claude-opus-4-8", temperature=0)

chain = ({"context": retriever, "question": RunnablePassthrough()}
         | prompt | model | StrOutputParser())

print(chain.invoke("What is our refund policy?"))
Where do the chunks live? In a vector database. This example uses an in-memory FAISS index; section 5 covers the real storage options.

Problem 2  LangGraph β€” stateful orchestration

A chain is a one-way pipe. The moment your app must loop ("keep using tools until you can answer"), branch ("refund vs. escalate"), coordinate multiple agents, or pause for a human, you need a graph. LangGraph models your app as nodes (steps) connected by edges (control flow) that share a state object β€” and edges may form cycles.

STARTβ†’ agent⇄ toolsβ†’ END

The agent ⇄ tools cycle is the reasonβ†’actβ†’observe loop a chain can't express.

agent.py
from typing import Annotated, TypedDict
from langgraph.graph import StateGraph, START
from langgraph.graph.message import add_messages
from langgraph.prebuilt import ToolNode, tools_condition
from langchain_anthropic import ChatAnthropic
from langchain_core.tools import tool
from langchain_core.messages import HumanMessage

@tool
def calculator(expression: str) -> str:
    """Evaluate arithmetic, e.g. '15 * 0.2'."""
    return str(eval(expression, {"__builtins__": {}}))

model = ChatAnthropic(model="claude-opus-4-8").bind_tools([calculator])

class State(TypedDict):
    messages: Annotated[list, add_messages]

def call_model(state): return {"messages": [model.invoke(state["messages"])]}

g = StateGraph(State)
g.add_node("agent", call_model)
g.add_node("tools", ToolNode([calculator]))
g.add_edge(START, "agent")
g.add_conditional_edges("agent", tools_condition)  # branch: tool or done?
g.add_edge("tools", "agent")                        # loop back
graph = g.compile()

out = graph.invoke({"messages": [HumanMessage("What's 15% of 240?")]})
print(out["messages"][-1].content)
What does the graph buy you over a chain?

The loop lets the model call several tools in sequence, reacting to each result. The conditional edge decides at runtime whether to act or answer. Add a checkpointer and the run becomes durable, resumable, and human-in-the-loop-ready (pause before a tool, wait for approval, resume).

Problem 3  LangSmith β€” see & measure it

LLMs are non-deterministic. When an answer is wrong, was it bad retrieval, a weak prompt, or the model? LangSmith gives you a trace β€” a nested tree of every step with latency, tokens, and cost β€” plus evaluation to catch regressions before shipping. It auto-instruments LangChain & LangGraph with zero code change:

terminal
# the RAG chain and agent above are now fully traced
export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY="..."

What a trace looks like

Running the agent on "What's 15% of 240?" produces a run tree — the three LLM calls are the reason→act→observe loop made visible:

β–Έ LangGraph run Β· agent            3.1s Β· 1,240 tok Β· $0.021 βœ…
   β”œβ”€ agent Β· LLM call #1          0.9s Β· 320 tok Β· calls calculator('240*0.15')
   β”œβ”€ tools Β· calculator          0.02s Β· returns '36'
   └─ agent Β· LLM call #2          0.8s Β· 360 tok Β· final answer βœ…

Evaluation β€” did my change help?

eval.py
from langsmith import Client, evaluate

client = Client()
client.create_dataset("qa")  # add {question β†’ reference answer} examples

def target(inputs):  # the system under test (your RAG chain)
    return {"answer": rag_chain.invoke(inputs["question"])}

def correct(outputs, reference_outputs):  # a grader
    return reference_outputs["answer"].lower() in outputs["answer"].lower()

evaluate(target, data="qa", evaluators=[correct], experiment_prefix="rag-v1")
# change a prompt β†’ run "rag-v2" β†’ compare scores side by side

5 Β· Vector databases β€” where chunks live

An embedding turns text into a vector positioned so similar meanings sit close together. A vector database stores those vectors + metadata and finds the nearest ones to a query β€” fast, over millions of items. That "semantic search" is the retrieval engine behind RAG.

DatabaseTypeBest for
ChromaOSS, embedded/serverPrototyping, local RAG
FAISSOSS library, in-processStatic, read-heavy, research
QdrantOSS server (Rust)Balanced prod, rich filters
PineconeManaged cloudHands-off production
pgvectorPostgres extensionAlready on Postgres

The storage flow never changes β€” only the client does. Compare storing the same chunks in two backends:

store_chunks.py
# pip install chromadb
import chromadb
from chromadb.utils import embedding_functions

client = chromadb.PersistentClient(path="./chroma_db")
ef = embedding_functions.SentenceTransformerEmbeddingFunction("all-MiniLM-L6-v2")
col = client.get_or_create_collection("policies", embedding_function=ef)

col.add(                       # store: Chroma embeds for you
    ids=[f"chunk-{i}" for i in range(len(chunks))],
    documents=chunks,
    metadatas=[{"source": "policy.txt"} for _ in chunks])

print(col.query(query_texts=["Can I get money back?"], n_results=2)["documents"])
Rule of thumb: start local with Chroma/FAISS to build the pipeline, then swap in a server (Qdrant/Pinecone/pgvector) when you need scale β€” the chunk β†’ embed β†’ upsert β†’ query flow is identical.

6 Β· create-llm-app β€” scaffold it all

Everything above is wired together by create-llm-app, a CLI that generates a runnable project. You pick a provider, an app type (chat / RAG / agent), and whether to enable LangSmith tracing β€” it emits main.py, requirements.txt, .env.example (loaded via python-dotenv), and more.

terminal
# install
pip install -e .

# interactive β€” prompts for provider, app type, tracing
create-llm-app new my-assistant

# or non-interactive with flags
create-llm-app new my-rag --provider anthropic --type rag --no-tracing

cd my-rag
pip install -r requirements.txt
cp .env.example .env     # add your API key(s)
python main.py
pick provider→ pick app type→ tracing on/off→ runnable project

Under the hood it maps your choices to a provider registry and renders Jinja2 templates β€” the same chat / RAG / agent code you saw in sections 1–3.

7 Β· Agentic patterns & techniques

Once a single agent isn't enough, you compose multiple β€” and there's a small catalogue of proven shapes for doing so. This repo has a runnable scaffold for each, following Google Cloud's Choose a design pattern for your agentic AI system, built on the same LangChain / LangGraph / LangSmith stack from this tutorial.

The doc's key axis is who decides the route:

predetermined route β€” code decides β€” sequential Β· parallel Β· loop vs AI decides β€” model routes β€” coordinator Β· hierarchical Β· swarm

Design patterns (Google Cloud)

#PatternWhen to useStack
01Single-AgentModerate multi-step tasks, no delegationLangGraph (prebuilt ReAct)
02SequentialStructured, repeatable pipelinesLangChain (LCEL)
03ParallelIndependent sub-tasks; diverse perspectives at onceLangChain (RunnableParallel)
04LoopIterative refinement with an exit conditionLangGraph (cycle)
05Review & CritiqueHigh accuracy / compliance validationLangGraph (generator + critic)
06Iterative RefinementOutput hard to nail in one shot; quality gateLangGraph (self-score loop)
07CoordinatorAdaptive routing of varied inputsLangGraph (router)
08HierarchicalAmbiguous, open-ended problemsLangGraph (orchestrator–worker)
09SwarmComplex problems that benefit from debateLangChain (multi-persona)
10ReActDynamic tasks needing tools + adaptationLangChain (manual loop)
11Human-in-the-LoopHigh-stakes / safety / compliance approvalsLangGraph (interrupt)
12Custom LogicMixed-pattern, bespoke branchingLangChain (plain Python)
13RAGGround answers in your own dataLangChain + FAISS
14RAG + CAGStable facts cached + volatile corpus retrievedLangChain + FAISS + prompt caching

Reasoning & agentic techniques

Prompting, reasoning, and retrieval techniques that make an individual agent smarter β€” complementary to the orchestration patterns above.

#TechniqueWhat it addsStack
15Chain-of-ThoughtReason step by step before answeringLangChain
16Self-ConsistencySample many CoT paths, majority-voteLangChain
17Tree of ThoughtsBranch, score, expand the best (search)LangChain
18ReflexionRetry with accumulated self-feedbackLangGraph
19Plan-and-ExecutePlan up front, then run each stepLangGraph
20ReWOOPlan all tool calls once, then solveLangChain
21Chain-of-VerificationDraft β†’ verify β†’ revise (anti-hallucination)LangChain
22Agentic RAGRetrieve on demand as a toolLangGraph + FAISS
23Corrective RAGGrade retrieval, fall back if weakLangChain + FAISS
24Self-RAGModel gates its own retrieval + groundingLangChain + FAISS
25Memory AgentRecall earlier turns (checkpointer)LangGraph
All 25 are syntax-verified in CI (py_compile on Python 3.11–3.13). Each folder has a README, a runnable main.py, requirements.txt, and .env.example. Browse them: experiments/ on GitHub.

8 Β· Full-stack example: a support agent

Everything in this tutorial, wired into one runnable app: a customer-support agent that answers questions grounded in a knowledge base and escalates to a human when it can't help. It combines all six pieces:

LangGraph

The agent ⇄ tools loop + memory across turns.

OpenAI

gpt-4o reasoning + text-embedding-3-small embeddings.

Chroma

Persistent vector database for the knowledge base.

RAG

search_kb retrieves grounding context on demand.

LangSmith

Traces every model + tool + retrieval step.

userβ†’ agent Β· gpt-4o⇄ tools Β· search_kb / escalateβ†’ grounded reply

Retrieval is a tool the agent chooses to call (agentic RAG), and a sensitive request routes to escalate_to_human instead of a guess.

agent.py (trimmed)
@tool
def search_kb(query: str) -> str:
    """Search the support KB β€” RAG over the Chroma vector store."""
    docs = store.similarity_search(query, k=4)
    return "\n\n".join(d.page_content for d in docs)

model = ChatOpenAI(model="gpt-4o", temperature=0).bind_tools([search_kb, escalate_to_human])

builder = StateGraph(State)
builder.add_node("agent", agent)                      # gpt-4o reasons
builder.add_node("tools", ToolNode(TOOLS))            # search_kb (RAG) + escalate
builder.add_edge(START, "agent")
builder.add_conditional_edges("agent", tools_condition)  # call a tool, or answer?
builder.add_edge("tools", "agent")                    # loop back with the result
graph = builder.compile(checkpointer=MemorySaver())   # memory across turns
run it
pip install -r requirements.txt
cp .env.example .env      # add your OPENAI_API_KEY
python ingest.py          # load KB -> embed -> store in Chroma (once)
python agent.py           # grounded answer, then an escalation

Complete source β€” copy each file

The whole example, ready to build. Hit Copy setup script and paste it into a terminal β€” it writes every file into a new support-agent/ folder. Or switch tabs and Copy file one at a time.

examples/support-agent/
python-dotenv
langchain
langchain-core
langchain-openai
langchain-chroma
langchain-text-splitters
chromadb
langgraph
langsmith
Full multi-file source (config Β· ingest Β· agent Β· sample KB) with an architecture diagram: examples/support-agent on GitHub. It's syntax-checked in the same CI as the experiments.

9 Β· Knowledge check

Five quick questions. Pick an answer for each, then check your score.

1. Your app needs to loop β€” call tools repeatedly until it can answer. Which tool fits?

2. An answer came back wrong and you want to see exactly which step failed. You reach for…

3. In RAG, why do we split documents into overlapping chunks before embedding?

4. You're prototyping RAG locally with no infrastructure. Best first vector store?

5. A human must approve a refund before the agent sends it. Which agentic pattern?

Next steps

🧠 Transformers

How LLMs work under the hood β€” tokens, embeddings, attention and sampling.

✍️ Prompt engineering

Steer the model well β€” prompt anatomy, few-shot, structured output & context.

πŸ“˜ LangChain guide

The full LangChain / LangGraph / LangSmith write-up with diagrams and comparisons.

πŸ—„οΈ Vector databases

Deep dive on storing chunk data across Chroma, FAISS, Qdrant, Pinecone, pgvector.

πŸ”Ž RAG vs CAG vs GraphRAG

Grounding techniques compared β€” when to retrieve, when to preload & cache, when to mix, and GraphRAG.

βš–οΈ LangChain vs LlamaIndex

Which framework? β€” LangChain's breadth vs LlamaIndex's data/RAG depth, and when to use each.

πŸ› οΈ create-llm-app

The CLI on GitHub β€” scaffold a runnable app in seconds.

πŸ§ͺ Agentic experiments

25 runnable scaffolds β€” agentic design patterns plus reasoning techniques.

πŸ€– Support agent

The full-stack example β€” LangGraph + OpenAI + Chroma RAG + LangSmith.

☁️ AWS services guide

An architect's field guide to 40 core AWS services β€” the problem each solves and its limits (pairs with Bedrock).

🧰 Agent Development Kits

Google ADK vs AWS Strands + Bedrock AgentCore vs LangGraph β€” problem, solution & limits for each.

πŸ“‘ IoT Γ— Agents

A smart fridge that auto-orders groceries β€” AWS IoT Core wired to a Strands agent on Bedrock AgentCore.