Not every file is text. Images, audio, custom binary protocols, and file formats without a text encoding all need to be handled as raw bytes, not decoded strings. This lesson covers binary mode I/O and the struct module for interpreting structured binary data.
1Binary Mode: When 'r'/'w' Actively Corrupts Data
Python's default file mode ('r' for reading, 'w' for writing) is text mode: open() automatically decodes the file's raw bytes into a str using a specified or platform-default encoding (commonly UTF-8), and separately performs newline translation (converting \r\n to \n on read, and back on write, on Windows). Both of these transformations are specifically designed for and appropriate to genuine text data ā they assume the file's bytes represent valid, decodable characters in some known encoding.
A PNG image, an audio file, a compiled binary, or any file format not defined in terms of a text encoding has no such property ā its bytes are meaningful as raw binary data, not as encoded characters. Opening such a file in text mode either fails outright with UnicodeDecodeError (the common, at-least-loud outcome, since most binary data isn't valid UTF-8) or, in less common cases, happens to decode 'successfully' while the newline translation step silently alters specific byte sequences that coincidentally resemble \r\n ā a genuinely corrupting, silent failure mode, not just an inconvenient crash.
'rb'/'wb' (binary mode) disables both transformations entirely: f.read() returns raw bytes exactly as stored on disk, with no decoding attempted and no newline translation applied, and f.write(some_bytes) writes those exact bytes with no transformation on the way out. This is the required mode for any file that isn't genuinely text ā the rule to internalize as an unconditional default: if a file format isn't specifically defined as text in a known encoding, open it in binary mode, full stop.
with open("image.png", "rb") as f:
header = f.read(8)
print(header) # b'\x89PNG\r\n\x1a\n' -- the PNG file signature, as raw bytes
with open("copy.png", "wb") as f:
f.write(header) # 'wb' -- write raw bytes, no text encoding appliedExact bytes from disk ā no decoding, no newline translation
2struct: Interpreting Fixed-Layout Binary Data
Reading raw bytes gets you the data, but many binary formats ā custom file formats, network protocol headers, data exported from a C program ā encode structured values (integers, floats, in a specific byte order) as a fixed sequence of bytes, and you need to interpret that sequence back into meaningful Python values (an int, a float) rather than working with opaque bytes objects directly. The struct module handles exactly this translation, in both directions.
struct.pack(format, *values) converts Python values into their raw binary representation according to a format string: ">If" specifies big-endian byte order (>; < for little-endian, = for native), followed by one character per value describing its type and size ā I for a 4-byte unsigned integer, f for a 4-byte float, among many other format codes (h/H for short integers, q/Q for 8-byte integers, d for double-precision floats). struct.unpack(format, packed_bytes) reverses the process, taking raw bytes and a matching format string and returning a tuple of the decoded Python values.
Byte order (endianness) matters specifically because different systems and protocols disagree on it ā network protocols conventionally use big-endian ('network byte order'), while many desktop CPU architectures are natively little-endian ā and getting it wrong doesn't raise an error, it silently produces a completely different, wrong numeric value from the same underlying bytes. Any binary format specification (a file format's documentation, a network protocol's RFC) will specify its byte order explicitly, and matching it exactly in your struct format string is essential for correct interpretation.
# WRONG: text mode on a binary file
with open("image.png", "r") as f:
data = f.read() # UnicodeDecodeError -- PNG bytes aren't valid UTF-8
# Even if decoding "succeeds" on some binary data, newline translation
# can silently alter byte sequences that happen to look like \r\n(42, 3.14) ā raw bytes correctly interpreted as int and float
3A Practical Pattern: Reading a Binary File Header
Combining binary-mode reading with struct unpacking is the standard approach for reading a binary file's structured header before deciding how to process the rest of the file ā many binary formats begin with a fixed-size 'magic number' or signature (the PNG example's b'\x89PNG\r\n\x1a\n' is exactly this: a fixed 8-byte sequence every valid PNG file starts with, letting code verify the file is actually a PNG before attempting to process the rest of it as one) followed by structured metadata fields at fixed byte offsets.
A typical pattern: f.read(N) reads exactly the first N bytes (the header's known, fixed size), struct.unpack(format_string, header_bytes) interprets those bytes according to the format's documented layout, and the resulting Python values (perhaps an image's width, height, and color depth, packed as consecutive integers) drive the rest of the processing logic ā deciding how to interpret whatever variable-length data follows the fixed header.
This pattern ā binary mode for exact byte access, struct for interpreting fixed-layout structured data within those bytes ā is the foundation for working with essentially any binary file format or network protocol at the byte level in Python, and understanding it demystifies what's actually happening 'under the hood' in higher-level binary-format libraries (an image-processing library, a specific network protocol implementation) that wrap this exact pattern behind a more convenient, format-specific API.
import struct
# Pack: 1 unsigned int (4 bytes) + 1 float (4 bytes), big-endian
packed = struct.pack(">If", 42, 3.14)
print(packed) # b'\x00\x00\x00*@I\x0f\xdb' -- raw binary layout
# Unpack: reverse the process
value_int, value_float = struct.unpack(">If", packed)
print(value_int, value_float) # 42 3.140000104904175The foundation of binary format parsing
4Step-by-Step Breakdown
Opening a binary file in text mode doesn't just fail loudly ā it can silently corrupt the data through encoding/decoding that was never appropriate to apply in the first place.
open(path, 'rb') reads raw bytes -- no encoding/decoding, no newline translation. This is REQUIRED for non-text files.
Opening a binary file in TEXT mode can corrupt it -- Python tries to decode bytes as text, and can fail outright or silently alter byte sequences that look like newlines.
Checkpoint: Why does open('image.png', 'r') (text mode) risk corrupting or failing to read a PNG file?
- āText mode attempts to decode the raw bytes as text using an encoding (like UTF-8), and PNG's binary data is not valid text in any such encoding
- āText mode has a maximum file size that binary files typically exceed
struct.pack/unpack convert between Python values and fixed-layout binary data -- essential for reading custom binary formats and network protocols.
Checkpoint: What does the format string ">If" tell struct.pack()?
- āBig-endian byte order (>), followed by an unsigned int (I) and a float (f), defining the exact binary layout
- āIt specifies the file extension to use when writing the packed data
Binary files complete the file-format toolkit; Streaming Large Files closes this section with the memory-efficient patterns for processing any of these formats at scale.
Pack and Unpack Real Binary Data. Finish encode_and_decode(): struct.pack/unpack convert between Python values and raw bytes.
Level Up š
Advanced cheat sheets, SEO tricks, and interview prep for this topic.
Browser Support
Fully supported (via server-side Python execution).
Fully supported (via server-side Python execution).
Fully supported (via server-side Python execution).
Fully supported (via server-side Python execution).
Best Practices
Always open non-text files (images, audio, custom binary formats) in binary mode ("rb"/"wb")
Text mode's decoding and newline translation are specifically appropriate for genuine text data ā applying them to binary data risks a crash or, worse, silent data corruption.
Match struct format strings exactly to a binary format's documented byte order and field sizes
Getting endianness or field size wrong doesn't raise an error ā it silently produces a plausible-looking but completely incorrect value from the same underlying bytes.
Frequent Bugs
Opening a binary file (image, audio, compiled binary) in text mode ('r'/'w' instead of 'rb'/'wb'), causing either an immediate UnicodeDecodeError or, worse, silent data corruption from inappropriate newline translation.
Always use binary mode ("rb" for reading, "wb" for writing) for any file that is not genuine, encoding-defined text data.
Real-World Examples
Verifying a File Is a Valid PNG Before Processing It
An image-processing pipeline receives uploaded files and needs to verify each one is actually a valid PNG (not a mislabeled or corrupted file) before attempting further processing.
PNG_SIGNATURE = b"\x89PNG\r\n\x1a\n"
def is_valid_png(path: str) -> bool:
with open(path, "rb") as f:
header = f.read(8)
return header == PNG_SIGNATURE