Strings in Go — Optimize¶
Exercise 1 — Replace + concatenation with Builder 🟢¶
Problem: The following function builds a comma-separated list but is very slow for large inputs.
func joinComma(items []string) string {
result := ""
for i, item := range items {
if i > 0 {
result += ","
}
result += item
}
return result
}
Benchmark baseline: 10000 items → ~50ms, 5000 allocs/op
Optimized Solution
func joinComma(items []string) string {
// Option 1: strings.Join (simplest)
return strings.Join(items, ",")
// Option 2: strings.Builder with Grow
// var sb strings.Builder
// total := len(items) - 1 // separators
// for _, s := range items { total += len(s) }
// sb.Grow(total)
// for i, item := range items {
// if i > 0 { sb.WriteByte(',') }
// sb.WriteString(item)
// }
// return sb.String()
}
Exercise 2 — Avoid repeated ToLower in loop 🟢¶
Problem: Case-insensitive deduplication calls ToLower inside a hot loop.
func deduplicateCaseInsensitive(items []string) []string {
seen := make(map[string]bool)
var result []string
for _, item := range items {
key := strings.ToLower(item) // allocates every iteration
if !seen[key] {
seen[key] = true
result = append(result, item)
}
}
return result
}
Problem: strings.ToLower allocates a new string every call even when the item is already lowercase.
Optimized Solution
// Option 1: Accept the cost but avoid double work
// The current code is actually reasonable. ToLower is O(n) per string.
// Main optimization: avoid calling ToLower twice if you store lowercased keys
// Option 2: Use strings.EqualFold in a different approach (only if N is tiny)
// Option 3: For ASCII-only data, write a fast no-alloc toLower check
func isAlreadyLower(s string) bool {
for i := 0; i < len(s); i++ {
c := s[i]
if c >= 'A' && c <= 'Z' {
return false
}
}
return true
}
func deduplicateCaseInsensitive(items []string) []string {
seen := make(map[string]bool, len(items))
result := make([]string, 0, len(items))
for _, item := range items {
var key string
if isAlreadyLower(item) {
key = item // no allocation needed
} else {
key = strings.ToLower(item)
}
if !seen[key] {
seen[key] = true
result = append(result, item)
}
}
return result
}
Exercise 3 — Replace chained strings.Replace with NewReplacer 🟡¶
Problem: HTML escaping using chained calls scans the string 5 times.
func escapeHTML(s string) string {
s = strings.Replace(s, "&", "&", -1)
s = strings.Replace(s, "<", "<", -1)
s = strings.Replace(s, ">", ">", -1)
s = strings.Replace(s, "\"", """, -1)
s = strings.Replace(s, "'", "'", -1)
return s
}
Problem: 5 passes through the string, 5 intermediate allocations.
Optimized Solution
**Key points:** 1. `htmlReplacer` is declared as a package-level variable — it is built once at init time. 2. `NewReplacer` scans the string in a single pass using a trie internally. 3. Result: 1 allocation (the output string), 1 pass, ~5x faster for typical HTML. **Benchmark improvement:** - Before: 5 allocs/op - After: 1 alloc/op (or 0 if output is written to a Builder via `r.WriteString`)Exercise 4 — Avoid []byte ↔ string roundtrip 🟡¶
Problem: A log formatter unnecessarily converts between string and []byte.
func formatLogLine(level, message string) string {
b := []byte("[")
b = append(b, []byte(level)...)
b = append(b, []byte("] ")...)
b = append(b, []byte(message)...)
return string(b)
}
Problem: Every []byte(string) conversion allocates.
Optimized Solution
**Or even simpler for fixed format:** **Result:** 1 alloc/op (the final string) vs 4+ with the original approach.Exercise 5 — Eliminate allocation in hot read path 🔴¶
Problem: A high-throughput HTTP router converts request path bytes to string for lookup.
type Router struct {
routes map[string]http.HandlerFunc
}
func (r *Router) ServeHTTP(w http.ResponseWriter, req *http.Request) {
path := string(req.URL.Path) // allocation per request!
handler, ok := r.routes[path]
if ok {
handler(w, req)
}
}
Note: req.URL.Path is already a string in the standard library. This example simulates a scenario where you have []byte from a custom parser.
Problem (simulated with []byte): Converting []byte to string for every map lookup allocates.
Optimized Solution
// The compiler already optimizes map[string(b)] lookups in Go 1.6+
// for the pattern: m[string(byteSlice)]
type Router struct {
routes map[string]http.HandlerFunc
}
func (r *Router) lookup(pathBytes []byte) http.HandlerFunc {
// This does NOT allocate — compiler optimization for map key lookup
return r.routes[string(pathBytes)]
}
// For custom hot-path parsers where you control the []byte:
// Use unsafe zero-copy only if profiling confirms it is necessary
import "unsafe"
func lookupUnsafe(routes map[string]http.HandlerFunc, path []byte) http.HandlerFunc {
// Zero-copy: safe only because map lookup does not retain the string
return routes[unsafe.String(unsafe.SliceData(path), len(path))]
}
Exercise 6 — Fix substring memory leak 🔴¶
Problem: A session store extracts session tokens from large HTTP cookie headers.
type SessionStore struct {
sessions map[string]*Session
}
func (s *SessionStore) ParseCookie(cookieHeader string) string {
// cookieHeader might be 4KB of cookie data
i := strings.Index(cookieHeader, "session=")
if i < 0 {
return ""
}
i += len("session=")
j := strings.Index(cookieHeader[i:], ";")
if j < 0 {
return cookieHeader[i:]
}
return cookieHeader[i : i+j] // PROBLEM: keeps 4KB alive!
}
Problem: Each extracted token retains the full 4KB cookie header in memory.
Optimized Solution
func (s *SessionStore) ParseCookie(cookieHeader string) string {
i := strings.Index(cookieHeader, "session=")
if i < 0 {
return ""
}
i += len("session=")
j := strings.Index(cookieHeader[i:], ";")
var token string
if j < 0 {
token = cookieHeader[i:]
} else {
token = cookieHeader[i : i+j]
}
// strings.Clone creates an independent copy, releasing the 4KB backing array
return strings.Clone(token)
}
Exercise 7 — Reuse Builder across calls 🟡¶
Problem: A high-frequency log formatter creates a new Builder for every log entry.
type Logger struct{}
func (l *Logger) Format(fields map[string]string) string {
var sb strings.Builder // new builder each call
for k, v := range fields {
fmt.Fprintf(&sb, "%s=%s ", k, v)
}
return strings.TrimSpace(sb.String())
}
Problem: strings.Builder allocates its internal buffer on every call.
Optimized Solution
type Logger struct {
mu sync.Mutex
sb strings.Builder
}
func (l *Logger) Format(fields map[string]string) string {
l.mu.Lock()
defer l.mu.Unlock()
l.sb.Reset() // reset length, keep capacity
first := true
for k, v := range fields {
if !first {
l.sb.WriteByte(' ')
}
l.sb.WriteString(k)
l.sb.WriteByte('=')
l.sb.WriteString(v)
first = false
}
return l.sb.String()
}
var builderPool = sync.Pool{
New: func() interface{} { return new(strings.Builder) },
}
func Format(fields map[string]string) string {
sb := builderPool.Get().(*strings.Builder)
sb.Reset()
defer builderPool.Put(sb)
for k, v := range fields {
sb.WriteString(k)
sb.WriteByte('=')
sb.WriteString(v)
sb.WriteByte(' ')
}
return strings.TrimSpace(sb.String())
}
Exercise 8 — Use strings.IndexByte instead of strings.Index 🟡¶
Problem: A CSV parser searches for single-character delimiters using strings.Index.
func splitCSV(line string) []string {
var fields []string
for {
i := strings.Index(line, ",") // searches for string ","
if i < 0 {
fields = append(fields, line)
break
}
fields = append(fields, line[:i])
line = line[i+1:]
}
return fields
}
Optimized Solution
**Why faster:** `strings.IndexByte` can use SIMD instructions (via `internal/bytealg`) to scan 16 or 32 bytes at a time. `strings.Index(s, ",")` has more setup overhead for the single-character case. **Benchmark:** For long lines (1000+ bytes), `IndexByte` is typically 2-4x faster than `Index` for single-character separators. **Best practice for CSV:** Use `encoding/csv` for production CSV parsing.Exercise 9 — Pre-allocate output slice 🟢¶
Problem: A log processor splits each line and collects results without pre-allocating.
func processLines(data string) []string {
lines := strings.Split(data, "\n")
var results []string // starts nil, grows dynamically
for _, line := range lines {
trimmed := strings.TrimSpace(line)
if trimmed != "" {
results = append(results, trimmed)
}
}
return results
}
Optimized Solution
**Why better:** Pre-allocating with `len(lines)` as capacity avoids repeated `append` reallocations. Even though some lines may be blank (and skipped), the capacity is an upper bound. Trades a small over-allocation for zero reallocations. **Alternative:** Count non-blank lines first, then allocate exactly: (2-pass but exact allocation — worth it only if the output is very large and long-lived)Exercise 10 — Replace fmt.Sprintf with direct Builder writes 🟡¶
Problem: A hot path uses fmt.Sprintf for simple string formatting inside a tight loop.
func buildQueryParams(params map[string]string) string {
var parts []string
for k, v := range params {
parts = append(parts, fmt.Sprintf("%s=%s", k, v)) // Sprintf allocates
}
return strings.Join(parts, "&")
}
Optimized Solution
**Why better:** 1. Eliminates the intermediate `[]string` slice (1 alloc per param) 2. Eliminates `fmt.Sprintf` overhead (format parsing, reflection) for each param 3. Single final allocation for the result string **Benchmark improvement:** For 10 params, typically 10x fewer allocations. **When to keep Sprintf:** When you need number formatting, padding, or complex format verbs. For pure string concatenation, direct Builder writes are always faster.Exercise 11 — Lazy string construction 🔴¶
Problem: Detailed debug log messages are always built, even when debug logging is disabled.
func processItem(item Item) {
log.Printf("Processing item: id=%d, name=%s, tags=%v, meta=%+v",
item.ID, item.Name, item.Tags, item.Meta)
// ... actual processing
}
Optimized Solution
// Option 1: Guard with log level check
const debugEnabled = false // or from config
func processItem(item Item) {
if debugEnabled {
log.Printf("Processing item: id=%d, name=%s, tags=%v, meta=%+v",
item.ID, item.Name, item.Tags, item.Meta)
}
// ... actual processing
}
// Option 2: Lazy stringer interface
type ItemDebug struct{ item Item }
func (d ItemDebug) String() string {
return fmt.Sprintf("id=%d, name=%s, tags=%v", d.item.ID, d.item.Name, d.item.Tags)
}
func processItem(item Item) {
// String() is only called if the log level is active
logger.Debug("Processing item", "item", ItemDebug{item})
}
// Option 3: Use slog (Go 1.21+) with lazy evaluation
import "log/slog"
func processItem(item Item) {
slog.Debug("Processing item",
slog.Int("id", item.ID),
slog.String("name", item.Name),
)
// slog skips string formatting entirely if Debug level is disabled
}