Intermediate

Graphify in Practice

The load-map-compress-measure cycle is the core Graphify workflow. This lesson walks through each step, shows what you are looking at and deciding at each stage, and covers how the visual output connects to the compression techniques from Lesson 5.

The Four-Step Cycle

Graphify's workflow is a repeating cycle: you load a context, map its token distribution, apply compression based on what you see, and measure the result. On the first pass, you go through all four steps. After that, you only re-run the steps relevant to what you changed. The cycle is designed to be fast enough to run as a normal part of development, not a periodic audit.

Step	What You Do	What You Produce
1. Load	Capture the assembled runtime context payload for a representative request	The actual token payload, not the template source
2. Map	Run Graphify's visualization to see token distribution by component	Token heat map, component breakdown, density chart
3. Compress	Apply the highest-priority optimization from the map (cache, deduplicate, graph-compress)	Modified context or pipeline configuration
4. Measure	Re-run Graphify on the new context and compare before/after	Token delta, cost delta, before/after comparison view

Step 1: Loading the Context

The most important thing to get right in Step 1 is loading the assembled runtime context, not the template. A Jinja template or Python f-string for a prompt looks small. The context that gets sent to the model after variable substitution, history injection, RAG chunk insertion, and tool result appending can be 10x larger. Optimizing the template without seeing the runtime payload misses the real problem.

Practical approaches for capturing runtime context:

• Request logging: Add a middleware layer that captures the full messages array before it is sent to the LLM API. Log to a file or a local store for analysis. This is the simplest and most reliable approach for any pipeline.
• Sampling: You do not need to log every request. A sample of 50-100 representative requests gives enough signal to see the distribution, find the outliers, and identify the dominant components.
• Staging replay: For pipelines with complex runtime assembly, replay a representative set of inputs through the assembly pipeline in a staging environment where you can capture the full payload without impacting production.

Step 2: Mapping Token Distribution

Once you have the runtime context, Graphify renders the token distribution. The primary output is a token heat map showing which sections of the context carry the highest token density. Secondary outputs include:

• Component breakdown: A percentage chart showing what fraction of total tokens came from each labeled component: system prompt, conversation history, retrieved context, tool results, user message.
• Redundancy flags: Sections where the same content (or near-duplicate content) appears in more than one component, flagged as deduplication candidates.
• Static vs. dynamic ratio: What percentage of the context is the same across requests (cacheable) vs. varies per request (dynamic). A high static ratio is a caching opportunity.

At the end of this step, you should have a ranked list of the top 2-3 cost drivers in the context, each with an estimated token count and a preliminary action: cache, deduplicate, compress to graph, truncate, or remove.

Step 3: Compressing

Compression in this context means applying the technique that best matches the waste pattern identified in Step 2. The mapping from pattern to technique was covered in earlier lessons; this step is about applying that technique correctly and verifying it does not break the pipeline's task performance.

Applying the Technique Without Breaking Quality

Every compression change should be tested against a fixed evaluation set before deployment. The evaluation does not need to be elaborate. A set of 20-50 representative queries with expected outputs (or a quality rubric) is enough to catch regressions from most compression changes. Common issues to check:

• History truncation: Does the model still correctly reference context from earlier in the conversation after truncation? Test multi-turn tasks that rely on earlier turns.
• Graph compression: Does the model answer entity-attribute questions correctly from graph-serialized input? Test the specific relationship types and attribute names you compressed.
• RAG reduction: Does answer accuracy hold after reducing top-k? Test queries that previously relied on the removed chunks.
• Cache placement: Does moving content to the cached prefix change how the model interprets it? Usually no, but test if the cached content includes instructions.

Incremental Application

Apply one compression at a time and measure after each. Two changes applied simultaneously make it impossible to attribute a quality regression to one or the other. The incremental approach is slower but produces a cleaner record of what each change contributed.

Step 4: Measuring the Result

After applying a compression, re-load the modified context and run Graphify again. The before/after comparison view shows:

• Token count before and after, absolute and percentage
• Cost estimate before and after (using current model pricing)
• Component-level breakdown showing which component changed and by how much
• Whether any new waste patterns appeared (sometimes a compression in one area reveals a new dominant component)

Record the result and add it to your token savings log. Lesson 7 covers how to build and maintain that log as a dashboard.

A Worked Example: RAG Pipeline Optimization

Here is a concrete end-to-end example of the four steps applied to a document Q&A pipeline.

Baseline (Step 1)

The pipeline retrieves the top-5 document chunks from a vector store and assembles a context with: system prompt (450 tokens), 5 retrieved chunks (average 380 tokens each, total 1,900 tokens), user question (80 tokens). Average context: 2,430 tokens.

Map (Step 2)

The heat map shows retrieved chunks at 78% of total tokens. The component breakdown flags that chunks 3, 4, and 5 have high overlap with chunks 1 and 2 (the same source document appears in multiple chunks). The static/dynamic ratio is 19% (only the system prompt is static; everything else varies per query).

Compress (Step 3)

Two changes applied in sequence:

1. Reduce top-k from 5 to 3. Test: run 30 Q&A pairs, measure answer accuracy. Result: accuracy unchanged (the removed chunks were the low-relevance duplicates).
2. Extract entity-attribute facts from the 3 remaining chunks into graph format. Test: run the same 30 Q&A pairs. Result: accuracy unchanged on factual queries, equivalent on reasoning queries.

Measure (Step 4)

New average context: system prompt 450 tokens, 3 graph-compressed chunks averaging 140 tokens each (420 tokens), user question 80 tokens. Total: 950 tokens. Reduction: 61% from 2,430 to 950. At a hypothetical $15/M input tokens, this saves approximately $0.022 per 1,000 requests - $22 per million requests from a single optimization pass.

Iteration Cadence

The load-map-compress-measure cycle is most effective when it becomes part of the development and deployment rhythm, not a one-time event. A practical cadence:

• Before each new pipeline goes to staging: Run one cycle on the assembled context. Fix any obvious structural problems before they reach production.
• After each significant feature change: New features often add new context components. Re-run the map step to see if the new component is a dominant cost driver.
• Monthly cost review: Pull the token distribution for the past month, identify any pipelines where cost increased disproportionately, and run a full cycle on those pipelines.

Lesson 7 covers the measurement infrastructure that makes this cadence sustainable: the metrics you track, how to build a dashboard, and how to set up alerts that tell you when a pipeline needs re-optimization.