News
AI, cybersecurity, machine learning, scraping & programming news.
76 results on this page · clear filters
Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video Generation
Why Video Diffusion Hits a Wall at Long Sequences Video diffusion transformers process spatiotemporal token sequences that grow with both spatial resolution and temporal duration. A 14.3-second, 768p video produces thousands of la...
Don't Mask the Environment: Observation Supervision Changes How Agents Explore Under RL
The Default Training Recipe for Language Agents Has a Hidden Blind Spot When you fine-tune a language model on agent trajectories, the training loss is applied only to the tokens the agent wrote. The environment responses, termina...
Probabilistic Linear Explanations
When a machine learning model denies someone a loan or flags a medical scan, the most natural question is "why?" Formal explainability has spent years trying to answer that question with mathematical rigor. The dominant approach f...
Double descent is the principle of least action
The double descent phenomenon is one of the most counterintuitive findings in modern machine learning. Train a model with a small number of parameters and test error falls as you add more, following the classical bias-variance cur...
Knowledge as Orbit: Finite Collections as Phases of an Exactly Periodic Latent Generator
Machine learning stores knowledge extensionally. Every item in a collection gets its own code, its own latent vector, its own row in a codebook. For X objects, you need X entries. The question this paper asks is whether a finite c...
Type Diversity Enables Transformers to Generalise Compositionally
The compositional generalisation problem has haunted Transformer research for years. Train a seq2seq model on "John drinks coffee" and it learns to translate that just fine. But give it "John drinks coffee on a mat" when only "on...
General Quantification of Covariate and Concept Shifts
Distribution Shift Is a Broken Promise Every deployed model makes an implicit bet: that the data it sees at inference will look like the data it trained on. When that bet fails, performance degrades silently and sometimes catastro...
Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data
When Running Out of Data Breaks Sparse Models The internet's text is finite. As language model training scales toward and beyond the limits of human-written corpora, repeating data has become standard practice. Muennighoff et al....
Distance generalization in transformers: why bother with positional encoding?
Positional Encoding May Be Hurting Your Transformer Transformers are everywhere, and positional encoding is considered foundational. Every major production model, from Llama to DeepSeek, uses RoPE (Rotary Position Embeddings) to t...
TART: A Modular Tool for Technique-Aware Audio-to-Tablature Guitar Transcription
Automatic Music Transcription for guitar has long been stuck on the wrong side of the difficulty curve compared to piano. Piano AMT hit high note-level accuracy years ago through CRNN models and large paired datasets, but guitar t...