Split learning divides a neural network between two nodes, keeping private data on a trusted local machine while offloading computation to a cloud server. A new paper demonstrates that a privacy mechanism designed for this architecture, mixing real training rows with decoy rows, fails completely because the gradient signal leaks exactly which rows were real.
How split-LLM training works
In a two-node split-LLM system, the Trusted Local Node holds the private training data and the early layers of the model. It computes activations on the training rows and sends them to the Untrusted Cloud Node, which runs the later layers and returns its output. The local node, holding the loss and the private labels, computes the gradient and sends it back to the cloud node for weight updates.
The privacy mechanism tested in the paper mixes real training rows with decoy rows before sending activations to the cloud. The idea is that the cloud cannot distinguish real rows from decoys, and the loss function ignores the decoys entirely. The cloud sees a frame containing many rows but does not know which ones matter.
Where the leak happens
The problem is that decoy rows produce exactly zero gradient. When the local node computes gradients, the real rows produce non-zero gradients that flow back to the cloud node. The decoy rows, being ignored by the loss, produce gradients that are precisely zero. The pattern of zeros in the returned gradient tells the cloud node exactly which rows were real and which were decoys.
The researchers measured this across nine random seeds and found that the zero pattern identified the real rows on every frame. Across all runs, 4,096 out of 4,096 frames per run were correctly identified. The decoy mechanism, which was supposed to provide privacy, provided none.
Recovering training data from the leak
Beyond identifying which rows were real, the researchers tested whether the gradient leak could recover actual training data. They conducted an attack on the frame contents and recovered approximately one extra token per hundred over a constant-guess baseline, an improvement of 0.65 to 1.50 percentage points. The shuffled-label controls, where the privacy mechanism should have been effective, recovered nothing.
The attack is not dramatic in isolation. Recovering one additional token per hundred does not sound like a catastrophic failure. But in the context of training on sensitive data, where each token might be a name, an address, or a medical term, the cumulative effect across many training steps could expose meaningful information. The researchers emphasize that five classes of attack, including those that accumulate observations across multiple training steps, were never measured, and the actual exposure could be significantly worse than what this single-step analysis shows.
The privacy check that passed anyway
The most concerning finding is that every run in the study passed the forward-channel privacy check and the quality check. The system was evaluated on whether the activations sent to the cloud node leaked information, and it did not. The quality check confirmed that the model trained correctly and produced outputs within acceptable bounds. Both checks passed.
The failure only became visible when the returned gradient was included in the analysis. The forward-channel check evaluated what the cloud received from the local node. It did not evaluate what the local node sent back to the cloud. The gradient, which flows in the opposite direction, contained the leak. The standard privacy evaluation missed it because it only looked at one direction of the communication.
This is a systems-security failure, not a cryptographic one. The privacy mechanism is sound in theory. The decoy rows do obscure the activations. But the gradient computation undoes that obscurity, and the privacy evaluation did not account for the return channel.
What the fix costs and what remains unknown
The researchers tested a mitigation: clipping and noising each row of the gradient before sending it back to the cloud. This closed the leak at a cost of approximately 0.01 nats of held-out cross-entropy, a small degradation in model quality. The fix works, but it is not free, and the paper notes that the system is not thereby safe.
Five classes of attack were never measured. The paper's analysis covers single-step gradient leakage, but real attackers can accumulate observations across training steps, correlate gradients across epochs, or combine gradient information with other side channels. The clipping and noising mitigation may or may not hold up against these more sophisticated attacks. The paper does not claim it does.
For teams building split-LLM systems, the practical takeaway is that privacy evaluation must include the full communication loop, not just the forward channel. The gradient is not a neutral signal. It carries information about the training data, and mechanisms that obscure the activations do not necessarily obscure the gradients. Any privacy analysis that ignores the return path is incomplete, and the fact that a system passes a forward-channel privacy check is not evidence that it is private.