Causal Memory Policy: Making Memory Utility Identifiable by Intervening on Retrieval

Memory-augmented large language models must decide which memories to retain, and recent systems do so by estimating each memory's effect on task performance. However, these estimates rely entirely on retrieved memories. When a memory is never retrieved, store-level interventions produce identical outcomes, leaving its utility unidentified. This is a retrieval-level positivity violation, invisible to diagnostics that examine only memory operations.

Causal factorization of memory utility through retrieval

Memory-augmented large language models maintain external memory stores that persist across interactions, enabling long-horizon reasoning, user personalization, and incremental knowledge accumulation. Deciding what to retain requires estimating the utility of each memory. Recent systems estimate this quantity from per-memory success and failure counts, paired trajectories with and without a candidate memory, controlled interventions on the candidate set, and explicit retention budgets. All of these estimators rely solely on memories already retrieved by the logging policy.

However, retrieval mediates the effect of the memory state on task output. A memory influences an answer only if it is retrieved into the context the model reads, so intervening on the memory store affects the output only through what retrieval subsequently surfaces. Memory utility therefore factorizes into two components: the probability of retrieval and the contribution once retrieved. Logged interactions cannot distinguish a memory with genuinely low utility from one that was never given the opportunity to contribute. This is a positivity violation, the causal inference condition requiring every unit to have a nonzero probability of receiving each treatment. Because the violation occurs at the level of retrieval, diagnostics that verify exploration only over memory operations fail to detect it.

We formalize a memory-augmented LLM as a structural causal model where exogenous variables consist of user queries and observed context, and endogenous variables are the memory state, retrieved subset, memory operation, and task output. The structural equations show that retrieval mediates the effect of memory on output, and that the two mediators form the causal chain from operation to memory state to retrieval to output. The reverse edge captures the policy selecting an operation based on current memory state. The graph remains acyclic as forward and reverse edges connect different time steps.

Retrieval-mediated identification failure

Estimating memory utility requires two conditions: exchangeability and overlap. Exchangeability requires that timesteps where a memory is present are comparable to those where it is absent, so outcome differences are attributable to the memory rather than to the circumstances under which it appears. Overlap requires that both cases occur, since an effect cannot be estimated from one alone. Since retrieval mediates the effect of memory on the output, memory utility factorizes into the probability of retrieval and utility conditional on retrieval.

Proposition 1 (Retrieval-Mediated Factorization) factorizes memory utility into two terms: retrievability, the probability that a memory is retrieved for a query; and conditional utility, the change in expected outcome caused by including the memory in the retrieved context while holding context size fixed. Their product is the contribution of the memory for a given query-context pair, and the utility averages it over the query distribution. The factorization also identifies where overlap must hold: in the retrieval variable, not in the operation variable.

Assumption 1 (Retrieval Positivity) requires that for all memories and all query-context pairs in the support, the probability of retrieval is greater than zero. Theorem 1 (Non-Identifiability under Retrieval Support Failure) shows that when the retrieval probability is zero, the product collapses to zero regardless of the conditional utility, and the conditional utility is not identified by any design that holds the memory state fixed while randomizing only whether the memory is included. Prior methods that estimate memory utility from logged interactions additionally require inclusion-level ignorability and feasibility-restricted positivity conditions, which are stricter than CMP's single retrieval-positivity assumption.

Causal Memory Policy (CMP)

Causal Memory Policy (CMP) restores identification by intervening on retrieval itself, reserving a fixed number of context slots for memories sampled with known propensity scores. CMP estimates memory utility by self-normalized inverse propensity weighting under a balanced assignment design with fixed sampling propensities. The framework has three components: (i) a randomized retrieval exposure design that guarantees retrieval positivity by construction, (ii) utility estimation via self-normalized inverse-propensity weighting, and (iii) a Bayes-optimal decision rule under irreversible operations.

The exposure design fills the B context slots by two mechanisms: the top-(B−k) memories from the deterministic retriever, and k randomly exposed memories sampled from an exposure pool. Across timesteps, the Tk exposure slots are allocated as evenly as possible across the exposure pool, so each memory receives exactly ⌊Tk/|𒫃|⌋ or ⌈Tk/|𒫃|⌉ exposures, and its propensity nm = n_m/T is known exactly. Balanced assignment guarantees every memory exactly five or six exposures, unlike independent inclusion which leaves many memories underexposed.

Utility estimation uses the self-normalized (Hájek) inverse-propensity weighting estimator. Since the propensity is known and balanced assignment fixes the exposure count in advance, the weight sums are constants, the estimator reduces to a difference of arm means, and its variance is exact rather than conditional on realized arm sizes. Self-normalization removes the propensity-dependent scaling of ordinary inverse-propensity weighting, making estimates directly comparable across memories.

From estimates to operations and decision rules

Component (iii) derives the Bayes-optimal decision rule under the asymmetry between reversible and irreversible operations. The forget operation permanently removes a memory, preventing any future re-estimation of its utility, while noop is reversible: a memory retained in error can be evicted later. Decision rules that treat the two errors symmetrically are therefore misspecified. Under asymmetric reversible operations, the Bayes-optimal policy retains a memory if its estimated utility exceeds a threshold that depends on the cost of forgetting versus the cost of retaining. Under irreversible operations, the optimal decision rule retains a memory only if its utility is identified and exceeds the threshold, since there is no second chance.

Finally, the framework shows that no aggregation over observed queries available to a retention policy predicts a memory's value on unseen queries. Per-query estimation reaches 0.78 AUC, but this information cannot be aggregated across queries to support retention decisions. The remaining limitation lies in the estimand itself, not in the estimation procedure.

Empirical identification failure and CMP's improvement

Empirically, identification fails for 54% of required memories on LongMemEval and 67% on LoCoMo, and the failure persists in a deployed memory system (Mem0), where unretrieved evidence has a median retrieval rank of 139 among 291 stored memories. Store-level randomization leaves the retrieved context unchanged on 52.2% of perturbations, causing utility-based eviction to perform no better than chance (lift 0.97). Restricting evaluation to retrieved memories eliminates the gap between the two estimators (0.655 vs. 0.658 AUC), localizing the failure to positivity.

CMP improves discrimination between required and non-required memories from 0.54 to 0.66 AUC. Per-query estimation raises discrimination to 0.78 AUC. However, identified memory utility alone is insufficient for retention decisions: no aggregation available to a retention policy predicts a memory's value on unseen queries, indicating that the remaining limitation lies in the estimand, not in the estimation procedure.

Experimental evaluation

CMP was evaluated on LongMemEval (long-horizon retention over ∼490-turn histories), LoCoMo (multi-session dialogue with many queries over a single memory store), HotpotQA and MuSiQue (2–4-hop reasoning, as replication), and Mem0 (a deployed memory system, as transfer). The exposure design parameters and statistical procedures are detailed in the appendix. Code is available at: https://anonymous.4open.science/r/cmp-release-D0C3/.

Code and reproducibility

Code is available at: https://anonymous.4open.science/r/cmp-release-D0C3/.