Functional
- Insert data: The user of a distributed cache system must be able to insert an entry to the cache.
- Retrieve data: The user should be able to retrieve data corresponding to a specific key.
Non-functional requirements
- High performance: The primary reason for the cache is to enable fast retrieval of data. Therefore, both the
insert and retrieve operations must be fast.
- Scalability: The cache system should scale horizontally with no bottlenecks on an increasing number of requests.
- High availability: The unavailability of the cache will put an extra burden on the database servers, which can also go down at peak load intervals. We also require our system to survive occasional failures of components and network, as well as power outages.
- Consistency: Data stored on the cache servers should be consistent. For example, different cache clients retrieving the same data from different cache servers (primary or secondary) should be up to date.
- Affordability: Ideally, the caching system should be designed from commodity hardware instead of an expensive supporting component within the design of a system.
Design considerations
- Data structure
- hash table
- doubly linked list - for constant time operations
- Hash Functions
- Consistent hashing
- for cases of crashing and scaling
- Cache client
- full understanding of all cache servers
- TCP and UDP to efficiently communicate
- writing policies
- Write-through cache
- write to both cache and database
- sacrifice latency for consistency
- Write-back cache
- just create on the memory
- periodically dump to the disk
- small latency
- Write-around cache
- when write, straight to disk
- when read, access memory first, if missed, check the disk
- not favorable for reading recent data
- Eviction policies
- FIFO
- LRU - Least recently used
- LFU - Least frequently used
- Cache invalidation
- use TTL(time to live)
- Actively checking or passive checking upon access
- Scaling
- distributed cache
- higher availability
- avoid SPOF
- decoupling of sensitive datas
