A cache is a short-term memory that is used to store recently accessed data from the original data store for faster retrieval, therefore, improving I/O operations.
Real World analogy
Let’s say you prepare dinner every day. You need ingredients for your recipe. When you cook, is it efficient to go to the store every time to buy the ingredients? Absolutely Not. It’s time-consuming to go to the store every time you want to cook.
It makes more sense to buy ingredients and store them in the refrigerator/pantry. This will save you time. In this case, your refrigerator/pantry acts like a cache.
Computer analogy
This is similar to computers. The CPU has a cache in the processor so that it doesn’t have to make data request every time from the RAM or Disk. Also, Accessing data from the RAM is faster than accessing data from the disk. Caching acts as a temporary local storage for data.
Caching hit and miss
When data is requested from a system, a copy of the data is either in the cache or not. If it’s present, this is referred to as cache hit, when it’s not, this is referred to as cache miss. A cache is said to perform well when it has more hits than misses.
When to use caching
You can’t store all your system information in a cache because caching hardware is limited in storage and more expensive than normal databases. Also, the search time will significantly increase when you store tons of data.
A cache, therefore, should contain the most relevant data for example data reads. This is because there are a lot more reads to handle than writes.
For instance, Twitter has an average of 300k requests on reads/second and only 6,000 writes/second. Caching tweets according to the user’s timeline greatly improves the system's performance and user experience.
Caching is mostly useful in the following scenarios:
Storing results of requests that are made many times to minimize data retrieval operations, especially on data that is immutable(does not change often ).
Storing results of complex and taxing computational problems, therefore, reducing the system latency.
Benefits of caching
Improved Application Performance - Memory is 50-200 times faster than disk(magnetic or SSD) and therefore reading from in-memory is extremely fast. The fast data access from the cache greatly improves the system's performance.
Reduce latency - Latency is a measure of delay. Modern applications such as Amazon may experience a high amount of traffic during some occasions like Black Friday and the Christmas season. Increased load on the databases results in higher latency to get data which makes the overall application slow. This may cost Amazon billions of dollars. By utilizing an in-memory cache, this issue can be avoided since it greatly improves system performance by reducing latency.
Increase Read Throughput - In addition to lower latency, caching greatly increases throughput. Throughput refers to how much data can be processed within a specific period of time. A single instance cache can serve hundreds of thousands of requests a second. This greatly improves system performance and scalability during spikes.
Reduce load on the database - By directing reads to the cache, it reduces the load on the database and protects it from slower performance or crashing at times of spikes.
Reduce Database Cost - A single cache instance can provide hundred of thousands of Input/Output operations per second. This can potentially replace the need for multiple database instances thus driving the database cost down.
Conclusion
Caching is a key component in improving the performance of a system. Each time you deliver resources closer to the user, you optimize your app performance.
