Graph-Based Context Structures

Flat prose is the most token-expensive way to express structured knowledge. Learn how representing the same information as a graph cuts token cost by 40-70%, why LLMs handle graph-serialized context well, and how to apply this technique to real pipelines.

The Token Cost of Prose

Natural language is designed for human communication, not for minimum-token information transfer. When you write a sentence like "The user account belongs to the enterprise plan and has three active sub-users, one of whom is an admin," you use 28 tokens to express three facts: account tier, sub-user count, and admin count. The words "The," "belongs to," "and has," "of whom is" are syntactic glue that carries no information. In a single sentence, this overhead is negligible. Across a knowledge base of hundreds of facts passed to an LLM, it becomes significant.

Flat-text context also repeats structural context continuously. If you have ten sentences about accounts, each one uses variations of "the account" or "this user" to refer back to the entity. A graph representation names each entity once and then lists its attributes and relationships without repeating the entity name. The savings come from eliminating both syntactic glue and repeated entity references.

What a Knowledge Graph Is in This Context

For LLM token optimization, a knowledge graph is a structured representation of entities and their relationships. It does not need to be a formal RDF triple store or a property graph database. The minimal useful definition is:

• Nodes: Named entities (user accounts, products, documents, concepts)
• Edges: Typed relationships between nodes (belongs_to, created_by, depends_on, replaces)
• Properties: Attributes attached to nodes or edges (plan: enterprise, created_at: 2025-03-15)

The key insight is that this structure can be serialized into a compact text format that LLMs read accurately while consuming far fewer tokens than equivalent prose.

Flat Text vs. Graph Serialization: A Comparison

Consider a small knowledge base about a software product's dependencies and configuration. Here is the flat-text version a typical RAG system might inject:

The graph serialization expresses the same five facts in roughly 44% of the tokens. At scale, across hundreds of such fact clusters, the savings compound dramatically.

Why LLMs Handle Graph-Serialized Context Well

A reasonable concern is whether models trained on natural language prose will reason accurately from graph-serialized input. In practice, modern LLMs handle structured notations well because their training data includes substantial code, configuration files, JSON, YAML, and other non-prose formats. They have strong priors for reading entity:attribute patterns.

The main caution is consistency: the model performs best when the serialization format is consistent throughout the context. A mix of JSON, YAML-like notation, and custom delimiters in the same prompt forces the model to switch parsing modes. Pick one format and apply it uniformly.

Common serialization formats and their trade-offs:

Techniques for Graph-Based Compression

Entity Deduplication

When multiple retrieved documents mention the same entity, flat RAG injection repeats that entity's context in each document. Graph extraction lifts the entity out of each document and merges its attributes into a single node. The node appears once in the context; each document's reference becomes a pointer rather than a full description.

Example: three retrieved support documents all mention that "Account #4821 is on the enterprise plan with 10 seats." In flat text, this appears three times (roughly 60 tokens for three repetitions). In a graph, the account node has plan=enterprise, seats=10 once, and the three documents have edges pointing to it (roughly 25 tokens total).

Implicit Relationship Compression

Prose uses many words to express what a typed edge expresses in a few characters. "The order was placed by the user and fulfilled by the warehouse" becomes order --placed_by--> user; order --fulfilled_by--> warehouse. The relationship type is now explicit and compact rather than buried in a verb phrase.

Attribute Flattening

Long prose descriptions of an entity's properties often repeat the entity name as a subject. "The product has a price of $49. The product is currently in stock. The product was added to the catalog on January 3." In graph form: product: price=$49, in_stock=true, added=2026-01-03. One entity reference, three attributes, approximately one-third of the tokens.

Hierarchical Path Compression

Deep nesting relationships (A is part of B which is part of C which belongs to D) are verbose in prose. Edge-list notation expresses them compactly: A -> B -> C -> D or with typed edges A[part_of]B[part_of]C[owned_by]D. This is particularly useful for organizational structures, file system hierarchies, and software dependency chains.

When Graph Compression Works Best

Graph compression produces the largest gains in contexts with these properties:

• High entity density: Many distinct named entities with attributes and relationships. Product catalogs, user permission systems, knowledge bases, infrastructure inventories.
• Repeated entity references: The same entity appears across multiple retrieved documents or across multiple turns of a conversation.
• Stable structure: The entity types and relationship types are known and consistent. Random freeform documents compress less well than structured domain data.
• Retrieval-heavy pipelines: RAG systems that pull many chunks benefit more than single-document summarization tasks.

Graph compression produces less gain for:

• Long-form narrative content (articles, stories, emails) where prose structure is part of the content
• Code that must be read exactly as-is
• Contexts where the model needs to reason about the original phrasing or wording, not just the facts

Extracting a Graph from Existing Documents

You do not always start with structured data. Often the knowledge is in documents. The extraction process has three stages:

1. Entity recognition: Identify the named entities in each document. This can be done with a lightweight NLP model, a simple LLM call with a structured-output schema, or rule-based pattern matching for well-structured domains.
2. Relationship extraction: Identify the typed relationships between entities. An LLM call with a schema like {"entity_1": str, "relationship": str, "entity_2": str} is effective and cheap for most domains.
3. Serialization: Render the extracted graph into your chosen format (JSON, YAML-like, edge-list) and inject it into the context instead of the original document text.

Combining Graph Compression with Caching

Graph compression and prompt caching are complementary. A knowledge graph extracted from stable documents changes infrequently. Once extracted, the serialized graph can be placed in the cached prefix of the system prompt, where it costs a fraction of its uncached price on every request that uses it. The combination of compression (fewer tokens total) and caching (lower per-token price) compounds the savings.

The workflow:

1. Extract the graph from your knowledge base or documents (one-time or periodic)
2. Serialize it compactly (40-70% of the original token count)
3. Place it in the cached portion of your system prompt
4. At runtime, include only the specific sub-graph relevant to the current query rather than the full graph (further reduction)

Lesson 6 walks through the full Graphify workflow that ties these techniques together into an end-to-end optimization cycle.