Base64 Decoding Performance in Python

Base64 converts binary data to text using 64 ASCII characters. The encoding represents three bytes with four characters, making it more efficient than a one-character-per-bit approach. The format appears in email attachments, certificates, data URLs, and JSON-embedded binary data.

Python includes a base64 module written in C, but it processes data one character at a time. A popular third-party alternative, pybase64, offers improved performance. For years, researchers including Daniel Lemire and Wojciech Muła have explored SIMD-accelerated base64 decoding. More recently, forgiving decoding — the variant web browsers use to skip spaces and line breaks — was implemented in the simdutf library. Because simdutf lacked a Python binding, fastpybase64 was created as one.

Installation and Basic Usage

Installation is straightforward:

pip install fastpybase64
import fastpybase64 as base64
base64.b64encode(b"hello")      # b'aGVsbG8='
base64.b64decode("aGVsbG8=")    # b'hello'

The module serves as a drop-in replacement for the standard library. Encoding and decoding use the same API, so existing code requires no changes.

Benchmark Results

Performance was measured on an Intel Xeon Gold 6548N (Emerald Rapids) processor pinned to a single core, running Python 3.14.6. The benchmark decoded base64-encoded random bytes and reported speed in gigabytes decoded per second. Real-world base64 often contains line breaks; MIME format, used for email attachments, segments text into 76-character lines. The benchmark used this MIME-style format as its example. A single thread was used.

Results showed fastpybase64 significantly outpaces other implementations. It runs four times faster than pybase64 and 27 times faster than the standard library module. The fastpybase64 package is available on PyPI for Linux, macOS, and Windows, supporting Python 3.10 through 3.14, including free-threaded Python 3.14.

Technical Approach

The speed improvements come from SIMD instruction usage and forgiving decoding that tolerates whitespace and line breaks in input. The underlying ideas were previously demonstrated in academic publications and the simdutf library, which mainstream browsers rely upon. The Python binding makes these techniques accessible to general application code.

Further Reading

  • Fast Forgiving Base64 Decoding with SIMD Instructions, ACM Transactions on Architecture and Code Optimization, 2026
  • Base64 encoding and decoding at almost the speed of a memory copy, Software: Practice and Experience 50 (2), 2020
  • Faster Base64 Encoding and Decoding using AVX2 Instructions, ACM Transactions on the Web 12 (3), 2018