Advanced

Best Practices

Production-ready patterns for LangChain applications - from project structure and error handling to cost optimization, testing, deployment, and migration from legacy chains.

Project Structure

Organize your LangChain project for maintainability and scalability:

Recommended Structure

my-langchain-app/
  .env                      # API keys (never commit)
  .gitignore
  pyproject.toml             # Dependencies
  src/
    app/
      __init__.py
      config.py              # Settings, model configs
      chains/
        __init__.py
        rag_chain.py         # RAG chain definitions
        analysis_chain.py    # Analysis chains
      prompts/
        __init__.py
        templates.py         # All prompt templates
      tools/
        __init__.py
        search.py            # Custom tools
        database.py
      agents/
        __init__.py
        research_agent.py    # Agent definitions
      models.py              # Model initialization
  data/
    documents/               # RAG documents
  tests/
    test_chains.py
    test_prompts.py
    eval/
      datasets/              # Evaluation datasets
      evaluators.py          # Custom evaluators
  scripts/
    ingest.py                # Document ingestion script
    evaluate.py              # Run evaluations

LCEL Best Practices

Python

# DO: Use LCEL for all new chains
chain = prompt | model | parser

# DON'T: Use legacy chain classes
# chain = LLMChain(llm=model, prompt=prompt)  # Deprecated!

# DO: Name your chain steps for debugging
chain = (
    prompt.with_config(run_name="format_prompt")
    | model.with_config(run_name="generate")
    | parser.with_config(run_name="parse_output")
)

# DO: Use RunnablePassthrough.assign for adding computed fields
from langchain_core.runnables import RunnablePassthrough

chain = RunnablePassthrough.assign(
    context=retriever | format_docs,
) | prompt | model | parser

# DO: Use with_structured_output for typed responses
structured_llm = model.with_structured_output(MySchema)

Error Handling Patterns

Python

from langchain_core.runnables import RunnableLambda

# Pattern 1: Fallback chains
primary_chain = prompt | ChatOpenAI(model="gpt-4o") | parser
fallback_chain = prompt | ChatAnthropic(model="claude-sonnet-4-20250514") | parser
safe_chain = primary_chain.with_fallbacks([fallback_chain])

# Pattern 2: Retry with backoff
resilient_chain = chain.with_retry(
    stop_after_attempt=3,
    wait_exponential_jitter=True,
)

# Pattern 3: Graceful error handling
def safe_invoke(input_data):
    try:
        return chain.invoke(input_data)
    except Exception as e:
        logger.error(f"Chain failed: {e}")
        return {"error": str(e), "fallback": "Unable to process request"}

# Pattern 4: Timeouts
from langchain_openai import ChatOpenAI
model = ChatOpenAI(model="gpt-4o", timeout=30, max_retries=2)

Cost Optimization

✅

Key cost optimization strategies:

Use cheaper models for simple tasks - gpt-4o-mini for classification, gpt-4o only when quality matters
Cache responses - SQLite cache for development, Redis for production
Batch requests - use chain.batch() with max_concurrency to reduce overhead
Optimize prompts - shorter prompts = fewer tokens = lower cost
Use streaming - stream responses to reduce perceived latency without buffering full responses
Track usage with LangSmith - identify expensive chains and optimize them

Python

# Use callback handlers to track token usage
from langchain_community.callbacks import get_openai_callback

with get_openai_callback() as cb:
    result = chain.invoke({"topic": "AI"})
    print(f"Tokens: {cb.total_tokens}")
    print(f"Cost: ${cb.total_cost:.4f}")

Testing LangChain Applications

Python

import pytest
from unittest.mock import patch, MagicMock
from langchain_core.messages import AIMessage

# Test 1: Unit test prompts (no LLM needed)
def test_prompt_template():
    prompt = ChatPromptTemplate.from_template("Hello {name}")
    result = prompt.invoke({"name": "Alice"})
    assert "Alice" in result.messages[0].content

# Test 2: Mock the LLM for fast tests
def test_chain_with_mock():
    mock_llm = MagicMock()
    mock_llm.invoke.return_value = AIMessage(content="Mocked response")

    chain = prompt | mock_llm | parser
    result = chain.invoke({"topic": "test"})
    assert result == "Mocked response"

# Test 3: Integration test with real LLM (use sparingly)
@pytest.mark.integration
def test_chain_integration():
    chain = prompt | ChatOpenAI(model="gpt-4o-mini") | parser
    result = chain.invoke({"topic": "Python"})
    assert len(result) > 0
    assert "python" in result.lower()

# Test 4: Evaluation with LangSmith datasets
# See LangSmith lesson for dataset-based evaluation

Production Deployment

LangServe (Quick API)

Python

# pip install langserve[server]
from fastapi import FastAPI
from langserve import add_routes

app = FastAPI(title="My LangChain API")

# Expose your chain as a REST API
add_routes(app, chain, path="/chat")

# Run: uvicorn app:app --host 0.0.0.0 --port 8000
# Endpoints created:
# POST /chat/invoke    - single invocation
# POST /chat/batch     - batch processing
# POST /chat/stream    - streaming
# GET  /chat/playground - interactive UI

FastAPI (Custom API)

Python

from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from pydantic import BaseModel

app = FastAPI()

class ChatRequest(BaseModel):
    message: str
    session_id: str = "default"

@app.post("/chat")
async def chat(request: ChatRequest):
    result = await chain.ainvoke({"input": request.message})
    return {"response": result}

@app.post("/chat/stream")
async def chat_stream(request: ChatRequest):
    async def generate():
        async for chunk in chain.astream({"input": request.message}):
            yield chunk
    return StreamingResponse(generate(), media_type="text/plain")

Streaming Best Practices

Python

# Always use streaming for user-facing applications
# It dramatically improves perceived latency

# For chains: use .stream() or .astream()
for chunk in chain.stream({"input": "Hello"}):
    send_to_client(chunk)

# For agents: use .astream_events() for granular events
async for event in agent.astream_events(
    {"input": "Search and analyze"},
    version="v2",
):
    kind = event["event"]
    if kind == "on_chat_model_stream":
        token = event["data"]["chunk"].content
        send_to_client(token)
    elif kind == "on_tool_start":
        send_status("Using tool...")

Common Mistakes

⚠

Avoid these common mistakes:

Using legacy chains - LLMChain, SequentialChain, ConversationChain are deprecated. Use LCEL.
Hardcoding API keys - always use environment variables or a secrets manager.
Not setting temperature=0 for deterministic tasks - classification, extraction, and structured output should use temperature=0.
Ignoring token limits - always check if your prompt + context fits in the model's context window.
Not using streaming - users perceive streaming responses as much faster.
Over-engineering with agents - if you can solve it with a simple chain, do not use an agent.
No error handling - LLM APIs fail. Always add fallbacks and retries.
Not monitoring with LangSmith - you cannot improve what you cannot measure.

Migration Guide: Legacy to LCEL

Python

# BEFORE (Legacy - Deprecated)
from langchain.chains import LLMChain
chain = LLMChain(llm=model, prompt=prompt)
result = chain.run(topic="AI")

# AFTER (LCEL - Recommended)
chain = prompt | model | StrOutputParser()
result = chain.invoke({"topic": "AI"})

# BEFORE (Legacy ConversationChain)
from langchain.chains import ConversationChain
chain = ConversationChain(llm=model, memory=memory)

# AFTER (LCEL with message history)
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are helpful."),
    MessagesPlaceholder("history"),
    ("human", "{input}"),
])
chain = prompt | model | StrOutputParser()

# BEFORE (Legacy AgentExecutor)
# Still works but consider LangGraph for new agents
agent_executor = AgentExecutor(agent=agent, tools=tools)

# AFTER (LangGraph - Recommended for new agents)
from langgraph.prebuilt import create_react_agent
app = create_react_agent(model, tools)

Frequently Asked Questions

Use LangChain when you need to chain multiple steps, use agents, implement RAG, or switch between providers. Call APIs directly for simple, single-call use cases where you want minimal dependencies.

LangChain v0.3 addressed this by splitting into langchain-core (minimal, ~5 dependencies) and optional provider packages. You only install what you need. The core package is lightweight.

Use LangChain LCEL for linear chains (prompt → model → parser). Use LangGraph for agents, multi-step workflows, cycles, human-in-the-loop, or anything that needs state management and conditional routing.

Use max_retries on the model, .with_retry() on chains, max_concurrency in batch calls, and consider adding a rate limiter middleware. LangSmith helps you monitor usage patterns.

Use LangServe for quick REST API deployment, or build a custom FastAPI app for more control. Deploy to any cloud provider (AWS, GCP, Azure) using Docker. Use LangSmith for production monitoring.

Course Complete!

📚

Congratulations! You have completed the LangChain course. You now know how to build chains, agents, RAG systems, and multi-agent workflows using the LangChain ecosystem. Continue learning by building real projects and exploring the LangChain documentation.