What is the meaning of all of this?
I was inspired to try my hand at the visual metaphor approach so I thought up a contrived story to try to explain background processing. The metaphor explains how Ruby’s Sidekiq works conceptually although the explanation will be useful for anyone trying to wrap their heads around background processing. The pics are bad because I can’t draw, alright?
Let’s kick this off using a super realistic scenario
Our imaginary acquaintance, Bob, uses Twitter frequently enough to understand just how important the platform is. He gets to opine about whatever he wants and he also gets to chime in to other people’s conversations with his own superior advice. If you’re a specimen of the male variety, you’ll know that the Twitter application expects you to opine strongly at least 20 times per week or your account will be deleted and you’ll bring shame upon your family. No pressure, Bob.
Bob believes that it’s important to broadcast his superior knowledge around topics related to newsworthy current events. In fact he thinks it’s absolutely crucial to his ego and outward projection of the fact that he’s in the top 0.1% of smartest thinkers. Ever. But for Bob it’s even more important to correct strangers discussing things with their online friends, things that don’t involve him at all. Whether he’s pointing out innocuous typos or doing gracious work like mansplaining space science to female NASA engineers, he always manages to hit that 20 tweet quota week after week. No sweat.
After his first year of operating in the Twitter mansplainsphere, Bob opened up a bricks-and-mortar store to offer his advice to his local community. Considering himself to be shy and reserved, he decides to call his business Bob’s Murmurs. On quiet days he’ll do his regular tweeting or purple-vegetable-picture-sending on dating apps. But whenever a local radio station mentions Bob’s business, the queues start to pile up.
Let’s see how we can help Bob become better at mansplaining more efficiently…
The problem with synchronous code
When a person gets to Bob’s counter in his store, the interaction can last anywhere between 30 seconds to upwards of 10 minutes. People asking for grammar corrections are usually in and out pretty quickly. For the more complicated pieces of advice, Bob first needs to think about his answers so people have to wait while queueing AND wait while he thinks up a response. People hate standing in queues. This high abandonment rate is hurting Bob’s business.
In programming terms, the queue is being handled using a single, synchronous process - Bob. Bob is the process. Synchronous means that he will only handle one request at a time and he can only move onto the next one once he’s finished with the current request.
Web applications run on a simple response-request lifecycle. Your browser sends a request for some information and the web app responds with some other information. Most requests are simple and can be handled within milliseconds - kind of like Bob’s typo corrections.
But sometimes web applications receive requests that require slightly more complex computation. Maybe the app needs to generate a PDF, compile a report, or fetch information from a sluggish 3rd party API. These requests are kind of like how Bob treats complex requests.
If a server cannot complete the query in time then it responds with an HTTP 504 status meaning that the request timed out. Bob’s equivalent would be when someone gets sick of standing around queueing and also waiting for him to come up with a decent response after being asked for advice. So they end up leaving to go live their magical lives. Life is magical and don’t let anyone tell you otherwise!
Introducing background processing
To help reduce his customer churn (he also knows pretty much everything about startup growth and customer service), Bob decides to refactor his business methods. This is where asynchronicity comes in.
Sidekiq is the most popular Ruby library for managing your background processes. It’s really fast, too. How it works is that you define a bunch of workers and those workers will handle processes for you in the background in a non-blocking manner.
Normally your application would execute the usual logic flow of a request and then return it in the response. Sidekiq allows you to queue up slow processes in a background task and return a response much quicker. The response might just be a message or an HTTP status code in the 200 range (Bob says this means that everything went smoothly). Once the slow process is finished running, you can handle that completed process.
Here’s an example: Bill requests a PDF and your website responds with a message notifying him that you’ll make the PDF available to him once it’s compiled. Bill will see the message and the job will be running in the background off of the main processing thread for your application. Once it’s done you can either email that PDF to Bill, make it accessible for him via a page on your dashboard, or whatever other method you choose. The point is that Bill did not sit there staring at a spinner gif for 45 seconds on a fibre connection waiting for a TPS report.
Bob goes async
Being a guru rockstar developer, Bob puts his application development knowledge into practice in the real world. Instead of having a single super-slow queue, Bob installs a ticket machine and hires a small bro army to help him grow his indie business.
Brett, Brad, and Broseph each get their own counters. Luckily for Bob all 4 of them lived in the same frat house so they all share similar knowledge and outlooks on life. Go Alpha Tau Omega! Go diversity, right Zuck?
Now when someone goes to Bob’s Murmurs they’ll interact with the ticket machine by submitting their query. They’ll then get a ticket from the ticket machine with a number. The store now also has comfortable chairs and Jack Johnson music for a bit of ambience. As soon as one of the bros is available, they call out the next number in the queue.
We can think of someone submitting their query as an HTTP request to a server. Instead of getting a gem of wisdom back immediately, the ticket machine responds with a numbered ticket and the person gets to relax while listening to some soothing tunes. That accounts for the response part of the request-response lifecycle. When one of the bros becomes available they’ll access the query (the ticket machine is linked to their computers at their counters) and google for some answers. Once they have an answer they can call the customer up, give them the answer/sage advice, and the person can go home a happy customer.
This works out better for Bob’s paying clientele because instead of standing around in a queue, they get to take a seat and waste time scrolling through Facebook. They’re comfortable. They’re less irate. And they don’t have to talk to a bro for longer than they absolutely need to.
In Sidekiq you’d define a worker class to handle a specific type of background request:
class ComplexBroWorker include Sidekiq::Worker def perform(name, count) # does some mansplaining end end
This class can then be invoked using a line like:
Brad has come into work with a hangover. He’s still able to dish out valuable advice but his mathematics abilities are not available for the day. He can’t even think about numbers without getting sick. When he’s assigned a math question by the ticket machine he knows that he won’t have the answer ready any time soon. The machine knows this too so it prints instructions on the ticket telling the customer to go home and that Brad will phone them with a detailed answer then following day. This buys him enough time to recover. When his brain can start processing numbers without making him sick he can ask the question on the Mathematics Stack Exchange and retain his reputation of being super smart. In Sidekiq terms, you achieve this using the following code:
BadMathBroWorker.perform_in(24.hours, 'fix_my_algebra', 5)
Business booms because 4 workers are able to handle way more customers, way quicker than before. In this scenario customers will only be served if they have a ticket. Each customer is considered a job to be done. Jobs reside in specific queues and are handled by workers once workers have the capacity to take on new processing work.
Splitting up the queues
Being the astute business mind that he is, Bob notices a weakness in his new system. The customers needing quick grammatical corrections are still being placed into the same queues as everyone else. This is very inefficient. Bob decides to implement a second queue.
He hacks the ticket machine and customers can now choose between several options before getting a ticket. They’ll either be assigned to the grammar queue or the default queue depending on the nature of their query. That poor customer that was going to wait for 20 minutes would now only have to wait 5 minutes or fewer!
Sidekiq lets you define however many queues you need inside of a configuration file. By defining the grammar queue first, it will be processed with higher priority than the default queue.
# This file is located at config/myapp_sidekiq.yml --- :concurrency: 1 :pidfile: tmp/pids/sidekiq.pid staging: :concurrency: 1 production: :concurrency: 4 :queues: - grammar - default
Then you simply need to specify your custom queue inside of your worker class:
class GrammarBroWorker include Sidekiq::Worker sidekiq_options queue: 'grammar' def perform(*important_args) # Fix some English, bro! end end
If Bob’s Murmurs were a web application, the ticket machine would be a controller (think MVC) that would decide which worker to dispatch a request to based on the request’s parameters. The only parameter here would be whether a customer pressed the grammar query button or the all other possible queries button on the machine.
You can start Sidekiq and specify your configuration file so that it’s aware of your options:
sidekiq -C config/myapp_sidekiq.yml
Corrections for the metaphor
There’s a limit to how efficiently your Sidekiq workers perform. The server’s memory is that limit. Obviously a machine running with 16gb of RAM will outperform a 512mb VPS. Bob can only hire as many bros as his business budget allows him to. For the metaphor above, think of free memory as your worker budget.
By default a Sidekiq process creates 25 different threads. Since Bob can only afford to pay himself and 3 employees, the equivalent for the ongoing analogy would be to set the concurrency to 4. Note that this number has already been added to the config file above (the store would be the production environment).
Concurrency refers to how many workers can work at any given time. We haven’t touched on threads yet. Think of a thread as a bro’s work ethic. If we change our thread limit to 2 then 2 bros would always have to be on an extended lunch break.
Also our bros are interchangeable. When Brett (or Bob or Brad or Broseph) stands behind one of the bulletproof counters at the complex queries department then he’s a ComplexBroWorker (using our above class names). But what if it’s Saturday and every single customer queueing in the store is hungover from Friday’s party? Maybe everyone is queueing for grammar corrections. Well then all 4 bros would take off their ComplexBroWorker hats and put on their GrammarBroWorker hats. Luckily Bob installed digital signage above the counters so the bros wouldn’t even need to move counters to begin serving people. Good thing too because those bicep curling weights are a chore to move around…
Hope this was useful
It’s difficult to map out software development concepts directly into silly little scenarios. Hopefully this post provides some of you newcomers with a simple metaphor to better understand background processing. If you enjoyed this then say wassup on twitter (I promise not to mansplain you) and sign up for my newsletter below.
If I’m totes wrong about stuff then pls hit me so I can make changes, I haven’t proofread this yet