Intermediate

Memory & State

Learn how AI systems remember information across conversations: conversation history, session state, long-term memory, and practical memory management patterns.

AI Memory Types

AI models do not inherently remember anything between API calls. Every form of "memory" is actually context engineering - information stored externally and injected into the prompt. There are three levels:

Memory TypeScopeDurationExample
Conversation HistorySingle conversationSessionPrevious messages in current chat
Session StateUser sessionHours/daysUser preferences, current task context
Long-Term MemoryUser accountMonths/yearsUser facts, past interactions, preferences

Conversation History

The simplest form of memory is maintaining conversation history - sending all previous messages with each new request.

Conversation History Management
class ConversationManager:
    def __init__(self, max_messages=20):
        self.messages = []
        self.max_messages = max_messages

    def add_user_message(self, content: str):
        self.messages.append({
            "role": "user",
            "content": content
        })
        self._trim()

    def add_assistant_message(self, content: str):
        self.messages.append({
            "role": "assistant",
            "content": content
        })
        self._trim()

    def _trim(self):
        # Keep only the most recent messages
        if len(self.messages) > self.max_messages:
            self.messages = self.messages[
                -self.max_messages:
            ]

    def get_messages(self):
        return self.messages.copy()

Memory in Popular AI Platforms

ChatGPT Memory

OpenAI's ChatGPT stores user facts and preferences across conversations. Users can view and delete memories. Automatically extracts key information.

Claude Projects

Anthropic's Claude uses Projects to maintain context. You upload documents and set project instructions that persist across conversations within the project.

Gemini Gems

Google's Gemini uses Gems for persistent personas and context. Each Gem maintains its own instructions and can access Google Workspace data.

Implementing Custom Memory

Summary-Based Memory

Periodically summarize the conversation and use the summary as context instead of the full history.

Summary-Based Memory
async def update_summary(client, history, summary):
    """Update running summary with new messages."""
    new_messages = history[-4:]  # Last 2 exchanges
    formatted = "\n".join(
        f"{m['role']}: {m['content']}"
        for m in new_messages
    )

    response = await client.messages.create(
        model="claude-haiku-4-20250514",
        max_tokens=500,
        messages=[{
            "role": "user",
            "content": f"""Update this conversation
summary with the new messages below.

Current summary:
{summary}

New messages:
{formatted}

Updated summary:"""
        }]
    )
    return response.content[0].text

Entity-Based Memory

Extract and store key entities (people, projects, preferences) mentioned in conversations.

Entity-Based Memory
class EntityMemory:
    def __init__(self):
        self.entities = {}  # {name: {facts}}

    async def extract_entities(self, client, text):
        """Extract entities and facts from text."""
        response = await client.messages.create(
            model="claude-haiku-4-20250514",
            max_tokens=500,
            messages=[{
                "role": "user",
                "content": f"""Extract key entities and
facts from this text. Return JSON:
{{"entities": [{{"name": "...",
"type": "person|project|preference",
"facts": ["fact1", "fact2"]}}]}}

Text: {text}"""
            }]
        )
        # Parse and store entities...

    def get_context(self) -> str:
        """Format entities as context."""
        lines = []
        for name, data in self.entities.items():
            facts = "; ".join(data["facts"])
            lines.append(f"- {name}: {facts}")
        return "\n".join(lines)

Memory Management Patterns

  1. Sliding Window + Summary

    Keep the last N messages in full detail, maintain a running summary of everything before that.

  2. Importance-Based Retention

    Score each piece of information by importance. Drop low-importance items first when space is limited.

  3. Topic-Based Retrieval

    Store memories indexed by topic. Only retrieve memories related to the current conversation topic.

  4. Time-Based Decay

    Gradually reduce the detail of older memories. Recent memories are verbatim; older memories are summarized.

Stateful vs Stateless Applications

AspectStatelessStateful
Each requestIndependent, no memoryBuilds on previous interactions
ComplexitySimple to buildRequires memory infrastructure
Use casesOne-off tasks, APIsChatbots, assistants, agents
ScalabilityHorizontally scalableRequires session management
CostPay per request onlyStorage + retrieval costs added
✅
Design guideline: Start stateless and add memory only where it creates clear value. Not every AI application needs long-term memory. For many use cases, well-designed context (RAG + system prompt) provides better results than attempting to simulate memory.

Ready to Go Deeper?

Live instructor-led courses from our partners. Affiliate disclosure.

🧠

AI & ML Courses - 30% Off

Live instructor-led AI, machine learning, data science, and cloud courses for working professionals. Use code Limited30 at checkout.

Edureka
📊

DataCamp - AI & Data Science

Hands-on Python, machine learning, and AI courses with interactive exercises and real projects.

DataCamp
🎓

edX - Top AI Courses

University-level AI courses from MIT, Harvard, Stanford. Earn certificates that employers recognize.

edX