Module 4: In-Memory Caching

Introduction to In-Memory Caching

In-memory caching is a technique that stores frequently accessed data in memory to improve application performance. By reducing the need to fetch data from slow storage systems or recompute complex calculations, caching can significantly improve response times.

In this module, we'll explore different caching strategies, implementation techniques, and best practices for effective cache management in Java applications.

Caching Fundamentals

Key Caching Concepts

• Cache Hit: When the requested data is found in the cache
• Cache Miss: When the requested data is not found in the cache and must be retrieved from the original source
• Cache Eviction: The process of removing entries from the cache, typically when the cache reaches capacity or entries expire
• Time-to-Live (TTL): The duration for which a cache entry is considered valid before it expires

Benefits of Caching

• Improved Performance: Faster data access compared to original sources (database, API, etc.)
• Reduced Load: Decreases the burden on backend systems
• Better Scalability: Handles more requests without proportional increase in resource usage
• Cost Efficiency: Reduces computational or I/O costs for repetitive operations

Implementing a Simple Cache in Java

Here's a basic implementation of an in-memory cache using Java's HashMap with time-based expiration:

public class SimpleCache<K, V> {
    private final Map<K, CacheEntry<V>> cache = new HashMap<>();
    private final long defaultTtlMillis;
    private final int maxSize;
    
    public SimpleCache(long defaultTtlMillis, int maxSize) {
        this.defaultTtlMillis = defaultTtlMillis;
        this.maxSize = maxSize;
    }
    
    public V get(K key) {
        CacheEntry<V> entry = cache.get(key);
        
        // Cache miss
        if (entry == null) {
            return null;
        }
        
        // Entry expired
        if (entry.isExpired()) {
            cache.remove(key);
            return null;
        }
        
        // Cache hit
        return entry.getValue();
    }
    
    public void put(K key, V value) {
        put(key, value, defaultTtlMillis);
    }
    
    public void put(K key, V value, long ttlMillis) {
        evictIfNeeded();
        cache.put(key, new CacheEntry<>(value, ttlMillis));
    }
    
    private void evictIfNeeded() {
        // Evict if cache is at capacity
        if (cache.size() >= maxSize) {
            // Simple strategy: remove oldest entry
            // More sophisticated: LRU, LFU, etc.
            K oldestKey = findOldestEntry();
            if (oldestKey != null) {
                cache.remove(oldestKey);
            }
        }
    }
    
    private K findOldestEntry() {
        // Implementation to find oldest entry
        // ...
    }
    
    private static class CacheEntry<V> {
        private final V value;
        private final long expiryTime;
        
        public CacheEntry(V value, long ttlMillis) {
            this.value = value;
            this.expiryTime = System.currentTimeMillis() + ttlMillis;
        }
        
        public boolean isExpired() {
            return System.currentTimeMillis() > expiryTime;
        }
        
        public V getValue() {
            return value;
        }
    }
}

Eviction Policies

When the cache reaches capacity, an eviction policy determines which items to remove:

• Least Recently Used (LRU): Removes the entry that hasn't been accessed for the longest time
• Least Frequently Used (LFU): Removes the entry with the lowest access count
• First In, First Out (FIFO): Removes the oldest entry first
• Time-Based Expiration: Removes entries based on a time-to-live value
• Random Replacement: Removes a random entry

The choice of policy depends on your application's specific access patterns and requirements.

Cache Size and TTL Considerations

Effects of Cache Size

• Larger Cache: Higher hit ratio but more memory consumption
• Smaller Cache: Lower memory usage but potentially more cache misses

Effects of TTL

• Longer TTL: Less frequent reloading of data but potentially stale data
• Shorter TTL: More up-to-date data but more frequent recomputation

The ideal settings balance memory usage, data freshness, and performance requirements.

Key Topics

Performance Considerations

While caching can significantly improve application performance, it's important to consider several factors:

• Memory Constraints: Caches consume memory, so properly sizing your cache is crucial
• Cache Coherency: Ensuring that cached data remains consistent with the source of truth
• Hit Ratio: Monitoring cache effectiveness through its hit ratio
• Cache Warming: Strategies for populating caches with data to avoid cold starts
• Distributed Caching: Understanding when to move beyond in-memory to distributed caching solutions

Effective caching strategies consider these factors to balance performance improvement against resource utilization and complexity.

Resources