Introduction

Distributed caching is a fundamental component of modern high-scale systems. In this post, we’ll explore advanced caching strategies, consistency models, and real-world implementation patterns that I’ve successfully deployed in production environments.

Cache Consistency Models

Strong Consistency vs. Eventual Consistency

When implementing distributed caches, one of the first architectural decisions is choosing between strong and eventual consistency. Let’s analyze the trade-offs:

Strong Consistency

  • Write-through caching
  • Synchronous updates
  • Higher latency
  • Better data integrity
  • Use case: Financial transactions

Eventual Consistency

  • Write-behind caching
  • Asynchronous updates
  • Lower latency
  • Potential stale reads
  • Use case: Content delivery

Advanced Caching Patterns

Cache-Aside (Lazy Loading) 

func GetUserData(id string) (*User, error) {
    // Try cache first
    if data, err := cache.Get(id); err == nil {
        return data, nil
    }

    // Cache miss - get from database
    data, err := db.GetUser(id)
    if err != nil {
        return nil, err
    }

    // Update cache asynchronously
    go cache.Set(id, data, expiration)
    return data, nil
}

Write-Through 

func UpdateUserData(id string, data *User) error {
    // Begin transaction
    tx := db.Begin()

    // Update database
    if err := tx.Update(id, data); err != nil {
        tx.Rollback()
        return err
    }

    // Update cache
    if err := cache.Set(id, data, expiration); err != nil {
        tx.Rollback()
        return err
    }

    return tx.Commit()
}

Cache Invalidation Strategies

Time-Based Invalidation

  • TTL (Time To Live)
  • Sliding expiration
  • Scheduled purge

Event-Based Invalidation 

type CacheInvalidator struct {
    subscribers map[string][]chan struct{}
    mu sync.RWMutex
}

func (c *CacheInvalidator) Invalidate(key string) {
    c.mu.RLock()
    defer c.mu.RUnlock()

    // Notify all subscribers
    for _, ch := range c.subscribers[key] {
        select {
        case ch <- struct{}{}:
        default:
            // Channel is blocked, skip
        }
    }
}

Production Considerations

Monitoring and Observability

  • Cache hit/miss ratios
  • Latency percentiles
  • Memory usage
  • Eviction rates

Hot Key Problem

Handling frequently accessed keys:

type ShardedCache struct {
    shards    []*redis.Client
    shardNum  int
}

func (c *ShardedCache) getShard(key string) *redis.Client {
    hash := fnv.New32a()
    hash.Write([]byte(key))
    shardIndex := hash.Sum32() % uint32(c.shardNum)
    return c.shards[shardIndex]
}

Circuit Breaking

Implementing fallback mechanisms:

type CacheWithCircuitBreaker struct {
    cache  Cache
    cb     *circuitbreaker.CircuitBreaker
}

func (c *CacheWithCircuitBreaker) Get(key string) (interface{}, error) {
    result, err := c.cb.Execute(func() (interface{}, error) {
        return c.cache.Get(key)
    })

    if err != nil {
        // Fallback to database
        return c.getFallbackData(key)
    }
    return result, nil
}

Performance Optimization

Compression

Implementing compression for large values:

func (c *Cache) SetCompressed(key string, value interface{}) error {
    // Serialize value
    data, err := json.Marshal(value)
    if err != nil {
        return err
    }

    // Compress data
    var buf bytes.Buffer
    w := lz4.NewWriter(&buf)
    if _, err := w.Write(data); err != nil {
        return err
    }
    w.Close()

    // Store compressed data
    return c.Set(key, buf.Bytes(), expiration)
}

Batch Operations

Optimizing multiple operations:

func (c *Cache) BatchGet(keys []string) (map[string]interface{}, error) {
    pipe := c.client.Pipeline()

    // Queue all get operations
    cmds := make(map[string]*redis.StringCmd)
    for _, key := range keys {
        cmds[key] = pipe.Get(key)
    }

    // Execute pipeline
    _, err := pipe.Exec()
    if err != nil && err != redis.Nil {
        return nil, err
    }

    // Collect results
    results := make(map[string]interface{})
    for key, cmd := range cmds {
        if val, err := cmd.Result(); err == nil {
            results[key] = val
        }
    }

    return results, nil
}

Conclusion

Effective distributed caching requires careful consideration of consistency models, invalidation strategies, and failure handling. The patterns and implementations discussed here have been battle-tested in high-scale production environments, serving millions of requests per second while maintaining sub-millisecond latencies.