Thevoxium published bare-lm, a minimal autograd tensor library written in C. The library implements automatic differentiation, common neural network layers, and optimizer updates in roughly 256 MB of arena-allocated memory, with zero individual malloc or free calls during training. The entire forward-backward-update cycle runs through two memory arenas that reset in constant time.
How the Memory Model Works
Every allocation in bare-lm goes through a single function: allocate_mem. The library maintains two arenas per Memory object, one permanent and one temporary. Permanent tensors, like weights, biases, and input data, live until free_global_mem is called. Temporary tensors, which include every intermediate result from forward and backward passes, live until reset_temp_mem zeros the arena pointer back to zero.
The training loop exploits this structure directly. Forward pass fills the temp arena. Backward pass reads from it. After both complete, sgd_step updates parameters, and reset_temp_mem empties the temp arena in O(1). The next epoch starts with a clean arena. There is no garbage collection, no reference counting, no individual deallocation. The cost of cleaning up after a training step is a single pointer reset.
The example XOR network in the README demonstrates the full cycle: create a 256 MB Memory object, allocate input and output tensors as permanent, build two linear layers that auto-register their weights, run 500 epochs of forward-backward-update, and call free_global_mem once at the end. Between epochs, reset_temp_mem keeps the temp arena from growing.
What the Library Actually Contains
bare-lm implements the operations you need for a small neural network, not the ones you might want for research. Linear layers, layer normalization, and embedding lookup handle the common building blocks. Activations include ReLU, GELU, sigmoid, tanh, and softmax. Loss functions cover mean squared error and cross-entropy. Optimizers include SGD with gradient clipping, Adam, and AdamW. Matrix multiplication delegates to OpenBLAS, which handles the performance-critical GEMM calls.
The tensor API is straightforward. tensor_init creates zero-initialized tensors. tensor_randn uses Box-Muller for random normal sampling. tensor_xavier provides Xavier initialization. Reshape, squeeze, unsqueeze, broadcast, permute, concat, and slice cover shape manipulation. The library does not include convolution, attention, or dropout. It does not include a dataset loader, a training loop abstraction, or a model serialization format beyond raw binary checkpoint save and load.
The Constraint Is the Design
The library's limitations are intentional. There is no GPU support. There is no automatic shape checking beyond what the C type system enforces. Batch matrix multiply assumes a specific shape convention. The softmax implementation is not numerically stable in the way a production library would be. These are tradeoffs that keep the code small and the memory model simple.
The checkpoint system is similarly minimal. save_checkpoint writes all parameters in the ParameterList to a binary file. load_checkpoint reads them back. There is no versioning, no metadata, no hash verification. The file is a raw dump of tensor data. This is enough to resume training or inspect weights, but not enough to share checkpoints across different builds of the library.
Who This Is For
bare-lm is not a replacement for PyTorch or JAX. It is a tool for understanding how autograd works at the implementation level. The arena memory model makes the cost of every allocation visible. The backward pass builds a topological sort of the computation graph from scratch. The optimizer step is a single function call over a parameter list. Nothing is hidden behind abstraction layers.
For developers who have used PyTorch but never looked at how automatic differentiation actually propagates gradients, bare-lm provides a complete implementation you can read in an afternoon. For embedded systems or environments where a runtime like Python is unavailable, the library offers a path to small neural networks with no external dependencies beyond OpenBLAS. The XOR example trains in under 500 lines of C, and the entire library compiles with a single make command.