8.1 io and File Handling — Specification
Reference material. Each interface listed with method signature, preconditions, postconditions, and invariants. Sentinel-error tables follow the interface tables. Concurrency table at the end.
This file is a distillation of the io, os, io/fs, and bufio package documentation as of Go 1.22, with implementation notes that the docs leave implicit. For prose explanations, see senior.md. For production patterns, see professional.md.
1. io.Reader
type Reader interface {
Read(p []byte) (n int, err error)
}
| Aspect | Specification |
| Return range | 0 <= n <= len(p) |
| Short reads | MAY return n < len(p) even when more data exists |
| Scratch use | MAY use all of p (including p[n:]) as scratch space |
| EOF with data | MAY return (n > 0, io.EOF) in a single call |
| EOF after data | MAY return data first, then (0, io.EOF) on the next call |
| Zero/nil return | SHOULD NOT return (0, nil) except when len(p) == 0 |
| Buffer retention | MUST NOT retain p after Read returns |
| Caller obligation | MUST process n > 0 bytes before considering err |
len(p) == 0 | Implementation-defined; usually returns (0, nil) |
Errors other than io.EOF are reader-defined. io.ErrUnexpectedEOF is not returned by Read directly — it is generated by helpers (io.ReadFull, io.ReadAtLeast) when a Reader ends prematurely.
2. io.Writer
type Writer interface {
Write(p []byte) (n int, err error)
}
| Aspect | Specification |
| Return range | 0 <= n <= len(p) |
| Short writes | MUST return non-nil err if n < len(p) |
| Slice mutation | MUST NOT modify p[i] for any i, even temporarily |
| Buffer retention | MUST NOT retain p after Write returns |
| Retry of suffix | Caller pattern: retry with p[n:] until empty or error |
len(p) == 0 | Implementation-defined; usually returns (0, nil) |
io.ErrShortWrite is the conventional error for "I tried to write all of p but only n < len(p) bytes made it." Callers see it from io.Copy-like helpers; primitives raise their own errors.
3. io.Closer
type Closer interface {
Close() error
}
| Aspect | Specification |
| First call | Releases the resource; may return an error |
| Second call | Behavior is undefined (most stdlib types return os.ErrClosed) |
| Error meaning on writers | A non-nil error means buffered/in-flight data may not have reached the destination |
| Error meaning on readers | Usually informational; data already returned remains valid |
A reader's Close releases its hold on the source; a writer's Close finalizes the destination. For composed writers (gzip, bufio), Close also writes any pending trailer or buffered data and is therefore a correctness concern, not just resource cleanup.
4. io.Seeker
type Seeker interface {
Seek(offset int64, whence int) (int64, error)
}
whence | Constant | Meaning |
| 0 | io.SeekStart | Relative to start of file |
| 1 | io.SeekCurrent | Relative to current offset |
| 2 | io.SeekEnd | Relative to end of file |
| Aspect | Specification |
| Return | New absolute offset, or error |
| Seek past end | Allowed on *os.File; subsequent Write creates a hole |
| Negative result | Error (os.PathError with EINVAL on POSIX) |
Seek(0, SeekCurrent) | Returns current position without moving |
5. io.ReaderAt
type ReaderAt interface {
ReadAt(p []byte, off int64) (n int, err error)
}
| Aspect | Specification |
| Position cursor | MUST NOT be affected; ReadAt is independent of Read/Seek |
| Short read with no error | MUST return non-nil err if n < len(p) |
| Concurrency | Multiple ReadAt calls on the same source MUST be safe in parallel |
| EOF | If off + len(p) reaches EOF, returns (n, io.EOF) with n ≤ len(p) |
Negative off | Implementation-defined; usually error |
Note the difference from Read: ReadAt MUST signal a short read with an error. This is stricter than Read's contract.
6. io.WriterAt
type WriterAt interface {
WriteAt(p []byte, off int64) (n int, err error)
}
| Aspect | Specification |
| Position cursor | MUST NOT be affected |
| Short write | MUST return non-nil err if n < len(p) |
| Concurrency | Safe in parallel for non-overlapping [off, off+len(p)) ranges |
| Overlapping writes | Behavior undefined for the overlapping bytes |
O_APPEND interaction | On Linux, O_APPEND overrides off; writes go to end |
The O_APPEND interaction is a Linux quirk to know: opening a file with O_APPEND makes WriteAt's offset argument a lie. The kernel ignores it.
7. io.ReaderFrom
type ReaderFrom interface {
ReadFrom(r Reader) (n int64, err error)
}
| Aspect | Specification |
| Behavior | Reads from r until EOF or error, writing into the receiver |
| Used by | io.Copy(dst, src) when dst implements it |
| Return | Total bytes successfully written; first error encountered (excluding io.EOF) |
| EOF semantics | A clean EOF from r is NOT returned as an error |
Implementations exist on *os.File (uses copy_file_range/splice when applicable), *bytes.Buffer, *bufio.Writer, *net.TCPConn, and io.Discard.
8. io.WriterTo
type WriterTo interface {
WriteTo(w Writer) (n int64, err error)
}
| Aspect | Specification |
| Behavior | Writes the source's content into w until exhausted |
| Used by | io.Copy(dst, src) when src implements it (preferred over ReaderFrom) |
| Return | Total bytes successfully written; first error encountered |
| EOF semantics | NOT returned as an error from WriteTo |
io.Copy checks WriterTo first, then ReaderFrom, then falls back to its 32 KiB internal-buffer loop.
9. io.ByteReader / io.ByteWriter
type ByteReader interface { ReadByte() (byte, error) }
type ByteWriter interface { WriteByte(c byte) error }
| Aspect | Specification |
ReadByte EOF | Returns (0, io.EOF) when no more bytes |
ReadByte short read | Cannot be short — either one byte or error |
WriteByte partial | Cannot be partial — either success or error |
encoding/binary.Uvarint and similar code prefer a ByteReader so they don't allocate buffers per byte. bufio.Reader and strings.Reader implement both.
10. io.RuneReader
type RuneReader interface {
ReadRune() (r rune, size int, err error)
}
| Aspect | Specification |
| Return | Decoded rune, byte length consumed, error |
| Invalid UTF-8 | Returns utf8.RuneError ('�') and size == 1 |
| Used by | regexp and similar text-processing packages |
11. io.StringWriter
type StringWriter interface {
WriteString(s string) (n int, err error)
}
Optional companion to Writer. io.WriteString(w, s) checks for it and falls back to w.Write([]byte(s)) if absent. Implementing it saves one allocation per call when the source is already a string.
12. io.ReadCloser, io.WriteCloser, etc.
The compositions below are pure interface combinations. They contain no methods of their own; the named methods are only those from the embedded interfaces.
| Composite | Embeds |
io.ReadCloser | Reader, Closer |
io.WriteCloser | Writer, Closer |
io.ReadWriteCloser | Reader, Writer, Closer |
io.ReadSeeker | Reader, Seeker |
io.WriteSeeker | Writer, Seeker |
io.ReadWriteSeeker | Reader, Writer, Seeker |
io.ReadSeekCloser | Reader, Seeker, Closer |
io.ReadWriter | Reader, Writer |
Naming convention: alphabetical order of method names.
13. Sentinel errors in io
| Error | Returned by | Meaning |
io.EOF | Reader.Read, ByteReader.ReadByte, RuneReader.ReadRune | Stream ended cleanly; no more data |
io.ErrClosedPipe | io.PipeReader.Read, io.PipeWriter.Write | The other end of the pipe was closed |
io.ErrNoProgress | bufio.Reader.Read, io.ReadFull | Reader returned (0, nil) more than 100 times |
io.ErrShortBuffer | io.ReadAtLeast | min > len(buf); cannot satisfy request |
io.ErrShortWrite | io.Copy, bufio.Writer.Write | A Write returned n < len(p) with err == nil |
io.ErrUnexpectedEOF | io.ReadFull, io.ReadAtLeast, binary.Read | EOF arrived before the required number of bytes |
Use errors.Is(err, io.EOF) rather than err == io.EOF. Some readers wrap the sentinel (e.g., decompressors may produce *gzip.Error{Err: io.ErrUnexpectedEOF}).
14. Sentinel errors in os and io/fs
| Error | Meaning | Notes |
fs.ErrInvalid | Invalid argument (e.g., empty path) | Underlying EINVAL |
fs.ErrPermission | Operation not permitted | Underlying EACCES/EPERM |
fs.ErrExist | File already exists | Returned with O_CREATE\|O_EXCL |
fs.ErrNotExist | File does not exist | Underlying ENOENT |
fs.ErrClosed | File already closed | Used after Close |
os.ErrDeadlineExceeded | Read/Write deadline expired | Network-only; not regular files |
os.IsNotExist, os.IsExist, os.IsPermission exist for legacy reasons. New code uses errors.Is(err, fs.ErrNotExist) etc.
15. io.Pipe semantics
func Pipe() (*PipeReader, *PipeWriter)
| Aspect | Specification |
| Buffer size | Zero — Write blocks until Read consumes |
| Read after writer Close | Returns io.EOF, or whatever was passed to CloseWithError |
| Write after reader Close | Returns io.ErrClosedPipe, or CloseWithError's value |
| Multiple writers | Each Write is delivered atomically; no interleaving guarantee on splits |
| Multiple readers | Each Read gets some bytes; other readers may starve |
CloseWithError(nil) | Equivalent to Close() (yields io.EOF on the read side) |
The "one writer, one reader" recommendation is operational, not contractual — multiple writers/readers don't corrupt memory, but ordering becomes nondeterministic.
16. bufio.Reader reference
| Method | Behavior | Allocates |
Read(p) | Reads from underlying source, possibly through buffer | No |
ReadByte() | One byte; refills buffer if empty | No |
UnreadByte() | Push the most-recent ReadByte back | No |
ReadRune() | One UTF-8 rune | No |
UnreadRune() | Push the most-recent ReadRune back | No |
ReadSlice(delim) | Slice into internal buffer; invalid after next read | No |
ReadBytes(delim) | Copy of bytes up to and including delim | Yes |
ReadString(delim) | String of bytes up to and including delim | Yes (string) |
ReadLine() | Slice into internal buffer; isPrefix == true if not finished | No |
Peek(n) | Slice into internal buffer; valid until next read | No |
Discard(n) | Skip n bytes; returns count actually skipped | No |
Reset(r) | Reset to wrap a different reader, reuse buffer | No |
Buffered() | Bytes currently in buffer | No |
Size() | Buffer capacity | No |
ReadSlice returning bufio.ErrBufferFull means the delimiter wasn't found in the buffer — the caller should keep accumulating, or grow the buffer.
17. bufio.Writer reference
| Method | Behavior | Allocates |
Write(p) | Buffers; flushes when full | No |
WriteByte(c) | Buffers single byte | No |
WriteRune(r) | Buffers UTF-8 encoding | No |
WriteString(s) | Buffers; uses WriteString on underlying if available | No |
Flush() | Writes buffered data to underlying writer | No |
Available() | Bytes free in buffer | No |
Buffered() | Bytes currently buffered | No |
Reset(w) | Reset to wrap a different writer, reuse buffer | No |
ReadFrom(r) | Copies from r until EOF, bypassing buffer for large reads | No |
Flush is mandatory before Close of the underlying writer. bufio.Writer has no Close method — it relies on the caller to flush and close the underlying stream in the right order.
18. bufio.Scanner reference
| Method | Behavior |
Scan() | Advance to next token; false at EOF or error |
Bytes() | Slice for the current token; invalid after next Scan |
Text() | String copy of Bytes() |
Err() | First non-EOF error from scanning, or nil |
Buffer(buf, max) | Set initial buffer and max token size |
Split(SplitFunc) | Set the split function |
| Split function | Token shape |
bufio.ScanBytes | One byte at a time |
bufio.ScanRunes | One UTF-8 rune at a time |
bufio.ScanWords | Whitespace-separated tokens |
bufio.ScanLines | Lines, with trailing \r\n or \n stripped |
Default MaxScanTokenSize is 65536. A token larger than this without calling Buffer results in bufio.ErrTooLong, and the token is lost (the scanner advances past it).
19. os.File method semantics
| Method | Pre/post |
Read(p) | Updates position cursor by n; respects O_APPEND only on Write |
Write(p) | With O_APPEND, ignores cursor; without, updates cursor |
ReadAt(p, off) | Independent of cursor; safe for concurrent calls |
WriteAt(p, off) | Independent of cursor; safe for non-overlapping concurrent calls. With O_APPEND on Linux, off is ignored |
Seek(off, whence) | Returns new absolute offset; allowed past EOF |
Truncate(size) | Resizes file; does NOT move cursor |
Sync() | fdatasync(2) on Linux; flushes data and required metadata |
Close() | First call releases FD; subsequent calls return os.ErrClosed |
Stat() | Returns FileInfo; on Linux, calls fstat(2) |
Fd() | Returns underlying FD as uintptr; FD owned by *os.File |
Name() | Returns the path passed to Open/Create; never updated by rename |
Chmod(mode) | fchmod(2) |
Chown(uid, gid) | fchown(2) |
20. os.OpenFile flags
| Flag | Meaning |
os.O_RDONLY | Open for reading only (mutually exclusive with the next two) |
os.O_WRONLY | Open for writing only |
os.O_RDWR | Open for reading and writing |
os.O_APPEND | Append on each Write; on Linux, makes WriteAt's offset moot |
os.O_CREATE | Create if not present; permission consulted then |
os.O_EXCL | With O_CREATE, fail if file exists |
os.O_SYNC | Each Write waits for disk (slow) |
os.O_TRUNC | Truncate to zero length on open |
os.Open is os.OpenFile(name, O_RDONLY, 0). os.Create is os.OpenFile(name, O_RDWR|O_CREATE|O_TRUNC, 0o666).
21. Concurrency safety
| Type | Safe for parallel reads | Safe for parallel writes | Mixed reads + writes |
*os.File (Read/Write) | No | No | No |
*os.File (ReadAt/WriteAt) | Yes | Yes (non-overlapping) | Yes (race for content, not memory) |
*os.File.Close w/ other ops | No | No | No |
bytes.Buffer | No | No | No |
bytes.Reader | Yes | n/a | n/a |
strings.Reader | Yes | n/a | n/a |
bufio.Reader | No | n/a | n/a |
bufio.Writer | n/a | No | n/a |
bufio.Scanner | No | n/a | n/a |
io.Pipe{Reader,Writer} | One reader + one writer | Same | One each |
*net.TCPConn | Yes (one reader) | Yes (one writer) | Yes (separate goroutines) |
io.MultiReader result | No (state in itself) | n/a | n/a |
io.MultiWriter result | n/a | Depends on each underlying writer | n/a |
22. io/fs interfaces
type FS interface {
Open(name string) (File, error)
}
type File interface {
Stat() (FileInfo, error)
Read([]byte) (int, error)
Close() error
}
| Optional interface | Method | Purpose |
fs.ReadDirFS | ReadDir(name) ([]DirEntry, error) | Directory listing |
fs.ReadFileFS | ReadFile(name) ([]byte, error) | Whole-file read |
fs.StatFS | Stat(name) (FileInfo, error) | Stat without open |
fs.GlobFS | Glob(pattern) ([]string, error) | Glob matching |
fs.SubFS | Sub(dir) (FS, error) | Subtree view |
fs.ValidPath(name) returns true for valid io/fs paths: forward slashes only, no leading slash, no . or .. components, no empty elements.
23. Path packages
| Package | Separator | Use |
path | always / | URL paths, io/fs paths, virtual paths |
path/filepath | OS-specific | Real filesystem paths via os |
filepath.Clean removes ./.. syntactically; it does NOT prevent traversal because .. after a symlink is filesystem-resolved differently. Use filepath.EvalSymlinks plus a containment check, or os.Root (Go 1.24+).
24. Default sizes and limits
| Constant | Value | Meaning |
bufio.MaxScanTokenSize | 65536 | Default Scanner token cap |
bufio.NewReaderSize default | 4096 | Default bufio.Reader buffer |
bufio.NewWriterSize default | 4096 | Default bufio.Writer buffer |
io.Copy internal buffer | 32768 | Used when no WriterTo/ReaderFrom |
os.File default mode (Create) | 0o666 (before umask) | Apply umask for actual mode |
25. What to read next
- senior.md — prose form of these contracts with examples.
- find-bug.md — bugs that result from violating items in the tables above.
- optimize.md — the performance implications of the default sizes.