The database queries are taking too much time. What’s the common resolution?, a cache in front of a database. The answer might not be a right one, but this is direct response you get many times.
Cache doesn’t just mean Redis. Redis has an early advantage of market adoption but that doesn’t mean Redis is only solution. Caching is a general concept and today we’re going to uncover if adding a cache really helps. If it does, then at which layer (yes, there could be multiple layers) a cache should be added.
Need For Caching
A same request can be made multiple times by a single user or multiple users depending on the application behaviour. Processing the request and generating response takes time as well as processing power. Caching helps remove this bottleneck.
The response once computed are stored in cache store, so user queries can be served from the cache instead of processing the request again and again. You can very well use any datastore as a caching solution such as a relational database (e.g. PostgreSQL, MySQL) and in-memory data stores (e.g. Redis, valkey, dragonflydb).
Read a detailed chapter ‘Caching Solutions and Strategies’ in my O’Reilly book System Design on AWS.
Today’s blog talks about different caching layers and how a generalized solution can be implemented across the teams. It references DoorDash internal implementation. Let’s start with the problem statement.
Problem Statement
Each team follows a different approach for caching implementation. Mostly Caffeine is used for local caching (instance level) and Redis Lettuce for distributed caching.
Each team is responsible to manage their data. The data is exposed to other teams via gRPC APIs.
Every team is spending time to manage the similar kind of infrastructure and posses multiple challenges. Let’s discuss challenges with current implementation.
Bottlenecks with Independent Implementations
Here are few challenges. :
The cache should not lag behind (by larger margin). Cache staleness should be handled appropriately.
Single point of failure on Redis. If Redis is unavailable, the whole caching infrastructure falls apart.
For introducing a new cache (or caching layer), new deployment and rollback strategies are needed. There is no control at runtime.
There is no standard schema to store the data in cache. Each team follows their own standards for caching keys.
Observability is big pain point. The metrics like cache hit, error rates are not consistent for every team to monitor the cache success.
Multi-layer caching support is not there. For teams, cache means Redis in current architecture.
Each team is spending the time to fix these challenges on their own so there is duplication of effort. The better idea could be a centralised solution, let’s discuss the solution in details.
Solution
The implementation abstraction can help teams to focus on the business logic instead of spending time on managing the caching infrastructure. The solution is to create a single interface for client’s interaction, depicted below.
single_interface_cache.get(cache_name, cache_key)The interface handles querying for the data across multi-layered cache and return once found, mentioned in the sequence below (subscribe for free to continue reading on the architecture details).
Let’s conclude on today’s discussion.
Conclusion
Every system design decision has trade-offs. It’s always recommended to do proper analysis if you think caching can solve a problem. If it does, which caching layer (or combination of layers) should be included as part of constructing request’s response.
You can watch complimentary video on YouTube for the analysis on DoorDash architecture -
Happy Building!