return to list of articles

Some quick lessons in caching

Notes from some recent experiences with caching and memcached.

Overview

I recently helped build an application using Ruby on Rails 4, the Russian Doll caching it provides, as well as a lot of memcached. I wanted to share some of my learnings.

Caching, your worst friend

If you have been a web developer for more a couple of months, you’re more than likely familiar with Phil Karlton’s quote “There are only two hard things in Computer Science: cache invalidation and naming things”. If you don’t really know what the former means, Phil was referring to the fact that funny things tend to happen when caching is in the picture because it’s really difficult to account for all possible scenarios that can pan out where it is involved.

It’s probably not uncommon that developer shops around the world experience the phenomenon of developer teams saying phrases like “it’s probably caching” or asking “have you cleared the cache first?” when debugging issues and bugs on production environments. I know that that’s the very first question I always ask before considering other possibilities, because it’s easy to dive into and get stuck in a codebase wasting hours only to have a problem magically fix itself because it was a caching issue.

Cache warming

In certain situations, it’s a good idea to warm your caches. What this essentially means is that you pre-calculate potentially expensive calculations so that when visitors come to your site they can be served a response a lot quicker from the cache instead of having to wait for the query to be calculated on their request. This can happen on a deploy, in cron-scheduled tasks, or however you may want to set them up.

If you have an application that has a large amount of different pages or API endpoints that change relatively frequently or expire quickly, it’s not a bad idea to have your cache warming run periodically so that you minimize the amount of requests that must be penalized with running expensive queries instead of having them warm and ready from your cache.

The upside is that you ensure that more of your site is snappy more often than not, but the downside comes with CPU and RAM resources being hogged for periods of time. I’m not sure what other downsides there are, message me if you strongly disagree with the idea of cache warming.

Beware of layering

In the recent application, we implemented an approach that is sort of but not really like an SOA approach. It was a non-transactional application with a fair amount of data which is loaded up once-off and decided to have one app handle the data and expose it via an API, and a separate app serve as the front-end facing app which was built using Ruby on Rails that consumed the API’s data.

This was beneficial in development as we could have a developer (there were 2 of us) work isolated on each of the applications. Also when certain design changes and idea testing was being done, we could quickly tweak and redeploy the API without having to worry about branching and conflicts on the front-end application.

As part of this, I ended up caching the requests in the front-end up in the controller, and also caching the response that the API provides. The approach was not all-too-different from the implmentation of Avdi’s RubyTapas screencast about caching domain objects.

This may sound over-zealous but the application was going to get a once-off spike of very heavy traffic, and the hardware resources were slightly limited. Additionally I slapped on some Russian-doll caching in the views of the front-end application. This, however, was actually more detrimental than beneficial as there is theoretically no performance increase of storing the controller variables in the cache and then wrapping view fragments in the cache of already partially cached objects (whereas having controller caching on each of the applications did show significant performance increases).

What this did do is make cache invalidation really hard and created more keys to invalidate to get the views to display the freshest data. Do not add extra caches as safety nets because they become liabilities. Instead cache the outermost space of whatever you need either in the controller or using Russian doll caching but not both. It’s worthwhile noting that if your cache storage is hosted on a separate host then each request incurs time costs, so having single lookups is much more beneficial.

Beware permanent keys

Because this application was not created with any mechanism to create or update data and was non-transactional in nature, there was no reason to create expiring cache keys. Instead all cache keys live in memcached for as long as memcached is running. There was no need, or so I thought.

Because the keys are forever, when something unexpected happens and the wrong thing is cached or old data is cached, you have to manually delete it from memcached (in Ruby on Rails via the Dalli gem) using something like this because it is easy to delete them:

Rails.cache.delete('name_of_key')

That’s fine if in your controllers you wrap your code in cache blocks with explicit key names like:

Rails.cache.fetch("#{cache_key}") { your_proc_here }

But now there is no way to delete your cache fragements that you have added in your views:

<% cache @object do %>
  # I am in here FOREVER unless I have a TTL
<% end %>

The cache key contains an MD5 hash so there’s no way that you can manually delete it. If you namespace your keys then you can delete matched keys, but beware because Memcached does not support that feature. In Memcached you can only delete a key by using the exact key name.

Expire those keys, dammit

In our case flushing the cache was not an option because re-caching everything takes a really long time and is extremely resource intensive, so that was a lesson learned the hard way. Add a TTL to your cache keys so that even your untouchable keys will be cleared eventually.

The law of the unexpected

As mentioned earlier, whenever caching is involved funny things happen. More accurately than that, things happen that you do not account for. In our case, after we uploaded data and started our cache warming task, we ran into a problematic scenario.

While the data was still being loaded up, certain pages of the front-end application were accessed. This caused keys to be cached and the pages ended up showing tables with messages saying there are no results. Even when the cache warming tasks finished, the same messages were displayed (because the keys do not expire).

To fix this, the cache keys needed to be deleted. Another issue was that the key for the back-end and and front-end app both needed to be deleted before reloading the page, which one may forget under time pressure. Even if both are cleared, if you are using Russian Doll caching, that won’t make a difference because that is cached without any results despite the JSON from your API now displaying the data.

My advice is if you ever do build an application of this sort with pre-calculated caches is to not be tempted to touch that website until EVERYTHING has finished running, add expiring keys, be explicit with your cache naming strategy, and using one key instead of two or more wherever you can.

To cache or not to cache?

Caching will always be hard and you will always learn new things about it, but not having caching as part of your strategy is unfeasible unless your application is small. I also don’t think there’s any other way to learn this than through hard lessons and experiencing it.

Feedback

Feel free to (call me an idiot if you must and) point out all the things I missed out or could have done better.


Get notified when Pawel releases new posts, guides, or projects