Queueing Systems

In web applications, not every task needs to be processed immediately. When youupload a large video file, send a bulk email campaign, or generate a complexreport, these time-consuming operations are often handled in the background.This is where queueing systems like Sidekiq orSolid Queue comes into play.

Queueing theory helps us understand how these systems behave under differentconditions - from quiet periods to peak load times.

Let's understand the fundamentals of queueing theory.

Basic Terminology in Queueing Theory

1. Unit of Work: This is the individual item needing service - a job.

2. Server: This is one "unit of parallel processing capacity." In queuingtheory, this doesn't necessarily mean a physical server. It refers to theability to process one unit of work at a time. For JRuby or TruffleRuby, eachthread can be considered a separate "server" since they can execute inparallel. For CRuby/MRI, because of the GVL, the concept of a "server" isdifferent. We'll discuss it later.

3. Queue Discipline: This is the rule determining which unit of work isselected next from the queue. For Sidekiq and Solid Queue, it is FCFS(FirstCome First Serve). If there are multiple queues, which job is selecteddepends on the priority of the queue.

4. Service Time: The actual time it takes to process a unit of work (howlong a job takes to execute).

5. Latency/Wait Time: How long jobs spend waiting in the queue before beingprocessed.

6. Total Time: The sum of service time and wait time. It's the completeduration from when a job is enqueued until it finishes executing.

Little's law

Little's law is a theorem in queuing theory that states that the average numberof jobs in a system is equal to the average arrival rate of new jobs multipliedby the average time a job spends in the system.

L = W

L = Average number of jobs in the system
= Average arrival rate of new jobs
W = Average time a job spends in the system

For example, if jobs arrive at a rate of 10 per minute (), and each job takes30 seconds (W) to complete:

Average number of jobs in system = 10 jobs/minute * 0.5 minutes = 5 jobs

This helps us understand the current state of our system and that there are 5jobs in the system on average at any given moment.

L is also called offered traffic.

Note: Little's Law assumes the arrival rate is consistent over time.

Managing Utilization

Utilization measures how busy our processing capacity is.

Mathematically, it is the ratio of how much processing capacity we're using tothe processing capacity we have.

utilization = (average number of jobs in the system/capacity to handle jobs) * 100

In other words, it could be written as follows.

utilization = (offered_traffic / parallelism) * 100

For example, if we are using Sidekiq to manage our background jobs then in asingle-threaded case, parallelism is equal to the number of Sidekiq processes.

Let's look at a practical case with numbers:

• We have 30 jobs arriving every minute
• It takes 0.5 minutes to process a job
• We have 20 Sidekiq processes

In this case, the utilization will be:

utilization = ((30 jobs/minute * 0.5 minutes) / 20 processes) * 100 = 75%

High utilization is bad for performance

Let's assume we maintain 100% utilization in our system. It means that if, onaverage, we get 30 jobs per minute, then we have just enough capacity to handle30 jobs per minute.

One day, we started getting 45 jobs per minute. Since utilization is at 100%,there is no extra room to accommodate the additional load. This leads to higherlatency.

Hence, having a high utilization rate may result in low performance, as it canlead to higher latency for specific jobs.

The knee curve

Mathematically, it would seem that only when the utilization rate hits 100%should the latency spike up. However, in the real world, it has been found thatlatency begins to increase dramatically when utilization reaches around 70-75%.

If we draw a graph between utilization and performance then the graph would looksomething like this.

The point at which the curve bends sharply upwards is called "Knee" in theperformance curve. At this point, the exponential effects predicted by queueingtheory become pronounced, causing the queue latency to climb up quickly.

Running any system consistently above 70-75% utilization significantly increasesthe risk of spiking the latency, as jobs spend more and more time waiting.

This would directly impact the customer experience, as it could result in delaysin sending emails or making calls to Twilio to send SMS messages, etc.

Tracking this latency will be covered in the upcoming blogs. The tracking ofmetrics depends on the queueing backend used (Sidekiq or Solid Queue).

Concurrency and theoretical parallelism

In Sidekiq, a process is the primary unit of parallelism. However, concurrency(threads per process) significantly impacts a process's effective throughput.Because of GVL we need to take into account how long the job is waiting for I/O.

The more time a job spends waiting on external resources (like databases orAPIs) rather than executing Ruby code, the more other threads within the sameprocess can run Ruby code while the first thread waits.

We learned about Amdahl's law inPart 3of this series.

Where:

p is the portion that can be parallelized (the I/O percentage)

n is the number of threads (concurrency)

Speedup is equivalent to theoretical parallelism in this context. In queueingtheory, parallelism refers to how many units of work can be processedsimultaneously. When we calculate speedup using Amdahl's Law, we're essentiallydetermining how much faster a multi-threaded system can handle work compared toa single-threaded system.

Let's assume that a system has an I/O of 50% and a concurrency of 10. ThenSpeedup will be:

Speedup = 1 / ((1 - 0.5) + 0.5 / 10) = 1 / 0.55 = 1.82  2

This means one Sidekiq process with 10 threads will handle jobs twice as fast asSidekiq with a single process with a single thread.

Let's recap what we are saying here. We are assuming that the system has I/O of50%. System is using a single Sidekiq process with 10 threads(concurrency). Thenbecause of 10 threads the system has a speed gain of 2x compared to systemhaving just a single thread. In other words, just because we have 10 threadsrunning we are not going to gain 10X performance improvement. What those 10threads is getting us is what is called "theoretical parallelism".

Similarly, for other values of I/O and Concurrency, we can get the theoreticalparallelism.

Let's go over one more time. In the last example, what we are stating is that ifa system has 95% I/O and if the system has 64 threads running, then that willgive 16x performance improvement over the same system running on a singlethread.

Here is the graph for this data.

As shown in the graph, a Sidekiq process with 16 threads handling jobs that are75% I/O-bound achieves a theoretical parallelism of approximately 3. In otherwords, 3x performance improvement is there over a single-threaded system.

Calculating the number of processes required

At the beginning of this article, we discussed "Little's law" and we discussedthat L is also called "offered traffic," which depicts the "average number ofjobs in the system".

If "offered traffic" is 5, then it means we have 5 units of work arriving onaverage that requires processing simultaneously.

We just learned that if the utilization is greater than 75% then it can causeproblems as there is a risk of latency to spike.

For queues with low latency requirements(eg. urgent), we need to target alower utilization rate. Let's say we want utilization to be around 50% to be onthe safe side.

Now we know the utilization rate that we need to target and we know the "offeredtraffic". So now we can calculate the "parallelism".

utilization = offered_traffic / parallelism=> 0.50 = 5 / parallelism=> parallelism = 5 / 0.50 = 10

This means we need a theoretical parallelism of 10 to ensure that theutilization is 50% at max.

Let's assume the jobs in this queue have an average of 50% I/O. Based on theabove mentioned graph, we can see that if the concurrency is 10 then we getparallelism of 2. However, increasing the concurrency doesn't increase theparallelism. It means if we want 10 parallelism, then we can't just switch toconcurrency of 50. Even the concurrency of 50 (or 50 threads) will only yield aparallelism of 2.

So we have no choice but to add more processes. Since one process with 10concurrency is yielding a parallelism of 2 we need to add 5 processes to get 10parallelism.

To get the I/O wait percentage, we can make use of perfm.Hereis the documentation on how it can be done.

Total number of Sidekiq processes required = 10 / 2 = 5

Here we're talking about Sidekiq free version, where we'll only be able to run asingle process per dyno. If we're using Sidekiq Pro, we can run multipleprocesses per dyno via Sidekiq Swarm.

We can provision 5 dynos for the urgent queue. But we should always have a queuetime based autoscaler like Judoscale enabled to handle spikes.

Sources of Saturation

We discussed earlier that, in the context of queueing theory, the saturationpoint is typically reached at around 70-75% utilization. This is from the pointof view of further gains by adding more threads.

However saturation can occur in other parts of the system.

CPU

The servers running your Sidekiq processes have finite CPU and memory. While CPUusage is a metric we can track for Sidekiq, it's generally not the only one weneed to focus on for scaling decisions.

CPU utilization can be misleading. If our jobs spend most of their time doingI/O (like making API calls or database queries), in which case CPU usage will bevery low, even when our Sidekiq system is at capacity.

Memory

Memory utilization impacts performance very differently from CPU utilization.Memory utilization generally exhibits minimal changes in latency or throughputfrom 0% to 100% utilization. However, after 100% utilization, things will startto deteriorate significantly. The system will start using the swap memory, whichcan be very slow and thereby increase the job service times.

Redis

Another place where saturation can occur is in our datastore i.e Redis in caseof Sidekiq. We have to make sure that we provision a separate Redis instance forSidekiq and also make sure to set the eviction policy to noeviction. Thisensures that Redis will reject new data when the memory limit is reached,resulting in an explicit failure rather than silently dropping important jobs.

This was Part 6 of our blog series onscaling Rails applications. If any part of theblog is not clear to you then please write to us atLinkedIn,Twitter orBigBinary website.

Please note that Solid Queue currently doesn't support cron-style or recurringjobs. There is a PR openregarding this issue. We have only partially migrated to Solid Queue. Forrecurring jobs, we are still using Sidekiq. Once the PR is merged, we willmigrate completely to Solid Queue.

Migrating to Solid Queue from Sidekiq

Here is a step-by-step migration guide you can use to migrate your Railsapplication from Sidekiq to Solid Queue.

1. Installation

• Add gem "solid_queue" to your Rails application's Gemfile and runbundle install.
• Run bin/rails generate solid_queue:install which copies the config file andthe required migrations.
• Run the migrations using bin/rails db:migrate.

2. Configuration

The installation step should have created a config/solid_queue.yml file.Uncomment the file and modify it as per your needs. Here is how the file looksfor our application.

default: &default  dispatchers:    - polling_interval: 1      batch_size: 500  workers:    - queues: "auth"      threads: 3      processes: 1      polling_interval: 0.1    - queues: "urgent"      threads: 3      processes: 1      polling_interval: 0.1    - queues: "low"      threads: 3      processes: 1      polling_interval: 2    - queues: "*"      threads: 3      processes: 1      polling_interval: 1development:  <<: *defaultstaging:  <<: *defaultheroku:  <<: *defaulttest:  <<: *defaultproduction:  <<: *default

3. Starting Solid Queue

On your development machine, you can start Solid Queue by running the followingcommand.

bundle exec rake solid_queue:start

This will start Solid Queue's supervisor process and will start processing anyenqueued jobs. The supervisor process forksworkers and dispatchersaccording to the configuration provided in the config/solid_queue.yml file.The supervisor process also controls the heartbeats of workers and dispatchers,and sends signals to stop and start them when needed.

Since we use foreman, we added the abovecommand to our Procfile.

# Procfileweb:  bundle exec puma -C config/puma.rbworker: bundle exec sidekiq -C config/sidekiq.ymlsolidqueueworker: bundle exec rake solid_queue:startrelease: bundle exec rake db:migrate

4. Setting the Active Job queue adapter

You can set the Active Job queue adapter to :solid_queue by adding thefollowing line in your application.rb file.

# application.rbconfig.active_job.queue_adapter = :solid_queue

The above change sets the queue adapter at the application level for all thejobs. However, since we wanted to use Solid Queue for our regular jobs andcontinue using Sidekiq for cron jobs, we didn't make the above change inapplication.rb.

Instead, we created a new base class that inherited from ApplicationJob andset the queue adapter to :solid_queue inside that.

# sq_base_job.rbclass SqBaseJob < ApplicationJob  self.queue_adapter = :solid_queueend

Then we made all the classes implementing regular jobs inherit from this newclass SqBaseJob instead of ApplicationJob.

# send_email_job.rb- class SendEmailJob < ApplicationJob+ class SendEmailJob < SqBaseJob  # ...end

By making the above change, all our regular jobs got enqueued via Solid Queueinstead of Sidekiq.

But we realized later that emails were still being sent via Sidekiq. Ondebugging and looking into Rails internals, we found that ActionMailer usesActionMailer::MailDeliveryJob for enqueuing or sending emails.

ActionMailer::MailDeliveryJob inherits from ActiveJob::Base rather than theapplication's ApplicationJob. So even if we set the queue_adapter inapplication_job.rb, it won't work. ActionMailer::MailDeliveryJob fallbacksto using the adapter defined in application.rb or environment-specific(production.rb / staging.rb / development.rb) config files. But we can't do thatbecause we still want to use Sidekiq for cron jobs.

To use Solid Queue for mailers, we needed to override the queue_adapter formailers. We can do that in application_mailer.rb.

# application_mailer.rbclass ApplicationMailer < ActionMailer::Base  # ...  ActionMailer::MailDeliveryJob.queue_adapter = :solid_queueend

This change is only until we use both Sidekiq and Solid Queue. Once cron stylejobs feature lands in Solid Queue, we can remove this override and set thequeue_adapter directly in application.rb, which will enforce the settingglobally.

5. Code changes

For migrating from Sidekiq to Solid Queue, we had to make the following changesto the syntax for enqueuing a job.

• Replaced .perform_async with .perform_later.
• Replaced .perform_at with .set(...).perform_later(...).

- SendMailJob.perform_async+ SendMailJob.perform_later- SendMailJob.perform_at(1.minute.from_now)+ SendMailJob.set(wait: 1.minute).perform_later

At some places we were storing the Job ID on a record, for querying the job'sstatus or for cancelling the job. For such cases, we made the following change.

def disable_form_at_deadline- job_id = DisableFormJob.perform_at(deadline, self.id)- self.disable_job_id = job_id+ job_id = DisableFormJob.set(wait_until: deadline).perform_later(self.id)+ self.disable_job_id = job.job_idenddef cancel_form_deadline- Sidekiq::Status.cancel(self.disable_job_id)+ SolidQueue::Job.find_by(active_job_id: self.disable_job_id).destroy!  self.disable_job_id = nilend

6. Error handling and retries

Initially, we thought theon_thread_error configurationprovided by Solid Queue can be used for error handling. However, during thedevelopment phase, we noticed that it wasn't capturing errors. We raisedan issue with Solid Queueas we thought it was a bug.

Rosa Gutirrezrespondedon the issue and clarified the following.

on_thread_error wasn't intended for errors on the job itself, but rathererrors in the thread that's executing the job, but around the job itself. Forexample, if you had an Active Record's thread pool too small for your numberof threads and you got an error when trying to check out a new connection,on_thread_error would be called with that.

For errors in the job itself, you could try to hook into Active Job's itself.

Based on the above information, we modified our SqBaseJob base class to handlethe exceptions and report it to Honeybadger.

# sq_base_job.rbclass SqBaseJob < ApplicationJob  self.queue_adapter = :solid_queue  rescue_from(Exception) do |exception|    context = {      error_class: self.class.name,      args: self.arguments,      scheduled_at: self.scheduled_at,      job_id: self.job_id    }    Honeybadger.notify(exception, context:)    raise exception  endend

Remember we mentioned that ActionMailer doesn't inherit from ApplicationJob.So similarly, we would have to handle exceptions for Mailers separately.

# application_mailer.rbclass ApplicationMailer < ActionMailer::Base  # ...  ActionMailer::MailDeliveryJob.rescue_from(Exception) do |exception|    context = {      error_class: self.class.name,      args: self.arguments,      scheduled_at: self.scheduled_at,      job_id: self.job_id    }    Honeybadger.notify(exception, context:)    raise exception  endend

For retries, unlike Sidekiq, Solid Queue doesn't include any automatic retrymechanism, itrelies on Active Job for this.We wanted our application to retry sending emails in case of any errors. So weadded the retry logic in the ApplicationMailer.

# application_mailer.rbclass ApplicationMailer < ActionMailer::Base  # ...  ActionMailer::MailDeliveryJob.retry_on StandardError, attempts: 3end

Note that, although the queue adapter configuration can be removed fromapplication_mailer.rb once the entire application migrates to Solid Queue,error handling and retry override cannot be removed because of the wayActionMailer::MailDeliveryJob inherits from ActiveJob::Base rather thanapplication's ApplicationJob.

7. Testing

Once all the above changes were done, it was obvious that a lot of tests werefailing. Apart from fixing the usual failures related to the syntax changes,some of the tests were failing inconsistently. On debugging, we found that theaffected tests were all related to controllers, specifically tests inheritingfrom ActionDispatch::IntegrationTest.

We tried debugging and searched for solutions when we stumbled uponBen Sheldon'scomment on one of Good Job's issues.Ben points out that this is actuallyan issue in Rails where Railssometimes inconsistently overrides ActiveJob's queue_adapter setting withTestAdapter. A PR is already openfor the fix. Thankfully, Ben, in the same comment, also mentioned a workaroundfor it until the fix has been added to Rails.

We added the workaround in our test helper_methods.rb and called the method ineach of our controller tests which were failing.

# test/support/helper_methods.rbdef ensure_consistent_test_adapter_is_used  # This is a hack mentioned here: https://github.com/bensheldon/good_job/issues/846#issuecomment-1432375562  # The actual issue is in Rails for which a PR is pending merge  # https://github.com/rails/rails/pull/48585  (ActiveJob::Base.descendants + [ActiveJob::Base]).each(&:disable_test_adapter)end

# test/controllers/exports_controller_test.rbclass ExportsControllerTest < ActionDispatch::IntegrationTest  def setup    ensure_consistent_test_adapter_is_used    # ...  end  # ...end

8. Monitoring

Basecamp has releasedmission_control-jobs, whichcan be used to monitor background jobs. It is generic, so it can be used withany compatible ActiveJob adapter.

Add gem "mission_control-jobs" to your Gemfile and run bundle install.

Mount the mission control route in your routes.rb file.

# routes.rbRails.application.routes.draw do  # ...  mount MissionControl::Jobs::Engine, at: "/jobs"

By default, mission control would try to load the adapter specified in yourapplication.rb or individual environment-specific files. Currently, Sidekiqisn't compatible with mission control, so you will face an error while loadingthe dashboard at /jobs. The fix is to explicitly specify solid_queue to thelist of mission control adapters.

# application.rb# ...config.mission_control.jobs.adapters = [:solid_queue]

Now, visiting /jobs on your site should load a dashboard where you can monitoryour Solid Queue jobs.

But that isn't enough. There is no authentication. For development environments,it is fine, but the /jobs route would be exposed on production too. Bydefault, Mission Control's controllers will extend the host app'sApplicationController. If no authentication is enforced, /jobs will beavailable to everyone.

To implement some kind of authentication, we can specify a different controlleras the base class for Mission Control's controllers and add the authenticationthere.

# application.rb# ...MissionControl::Jobs.base_controller_class = "MissionControlController"

# app/controllers/mission_control_controller.rbclass MissionControlController < ApplicationController  before_action :authenticate!, if: :restricted_env?  private    def authenticate!      authenticate_or_request_with_http_basic do |username, password|        username == "solidqueue" && password == Rails.application.secrets.mission_control_password      end    end    def restricted_env?      Rails.env.staging? || Rails.env.production?    endend

Here, we have specified that MissionControlController would be our basecontroller for mission control related controllers. Then inMissionControlController we implemented basic authentication for staging andproduction environments.

Observations

We haven't had any complaints so far. Solid Queue offers simplicity, requires noadditional infrastructure and provides visibility for managing jobs since theyare stored in the database.

In the coming days, we will migrate all of our 22 Neeto products to Solid Queue.And once cron-style job support lands in Solid Queue, we will completely migratefrom Sidekiq.

Recently, 37signals open sourcedSolid Queue.

Solid Queue is database based queuing backend for Active Job. In contrastSidekiq and Resque are Redis-based queuing backends.

In her blog Rosa Gutirrez mentioned following lineswhich captured our attention.

In our case, one feature that PostgreSQL has had for quite some time and thatwas finally introduced in MySQL 8, has been crucial to our implementation:

SELECT ... FOR UPDATE SKIP LOCKED

This allows Solid Queues workers to fetch and lock jobs without locking otherworkers.

As per her, this feature had beenin PostgreSQLfor a while, and now this feature has landedin MySQLmaking it possible to build Solid Queue.

We had never heard of UPDATE SKIP LOCKED feature either in PostgreSQL or inMySQL. We were wondering what is this UPDATE SKIP LOCKED without which it wasnot possible to build Solid Queue. So we decided to look into it.

Processing jobs from a queue

Consider a case where we need to build a system where a bunch of jobs need to beprocessed in the background.

There are a bunch of workers waiting to grab a job and start processing themoment a job becomes available. The challenge is when multiple workers attemptto claim the same job simultaneously how do we ensure that only one of theworkers claims the job for processing. At any point of time, a worker shouldclaim only the "unclaimed" job, and an "unclaimed" job should be claimed by oneand only one worker.

Here is how one might go about it implementing it.

START TRANSACTIONSELECT * FROM JOBS WHERE processed='no' LIMIT 1;-- Process the jobCOMMIT;

With the above code, it's possible that two workers might claim the same job.

One way to resolve this issue is by marking a particular row as locked forupdate.

START TRANSACTION;SELECT * FROM JOBS WHERE processed='no' FOR UPDATE LIMIT 1;-- Process the jobCOMMIT

SELECT ... FOR UPDATE locks a particular row, and hence no one else can lockthat record.

As soon as a new job comes in, multiple workers will execute the above query andwill try to take a lock on that record. The database will ensure that only oneof the workers gets the lock.

The first worker will take the lock on the record using FOR UPDATE. When otherworkers come to that record and they see that there is a lock FOR UPDATE, theywill wait for the lock to be lifted. Yes, these workers will wait until the lockis released.

The lock will only be released when the transaction is committed. When thetransaction is committed and the lock is released, then other workers will gethold of the record only to find that the job has already been processed. As youcan see, this is a highly inefficient process.

That is where FOR UPDATE SKIP LOCKED comes in.

SKIP LOCKED skips locked rows

START TRANSACTION;SELECT * FROM jobs_table FOR UPDATE SKIP LOCKED LIMIT 1;-- Process the jobCOMMIT;

Imagine the same scenario here. A job comes in. Multiple workers compete toclaim the job. The database ensures that only one worker gets the lock. Howeverin this case the other workers will move on to the next record. They will notwait. That's what SKIP LOCKED does.

MySQL has detailed documentation onhow SKIP LOCKED worksIf you want to read in more detail.

Solid Queue uses FOR UPDATE SKIP LOCKED feature to ensure that a job isclaimed by only one worker.

How GoodJob manages job processing without SKIP LOCKED

GoodJob burst into the scenearound July, 2020.GoodJobs supports only PostgreSQL database because it uses advisory locks toguarantee that no two workers claim the same job.

PostgreSQL folks understand that the lock mechanism provided by the databasewould not satisfy all the variety of cases that might arise in an application.Advisory locks are a mechanism that allows applications to establish acommunication channel to coordinate actions between different sessions ortransactions. Unlike regular row-level locks enforced by the database system,advisory locks are implemented as a set of low-level functions that applicationscan use to acquire and release locks based on their requirements. We can readmore about ithere.

pg_advisory_lock functionwill lock the given resource. However, if another session already holds a lock onthe same resourc,e then this function will wait. This is similar to theFOR UPDATE case we saw above.

However pg_try_advisory_lock function will either obtain the lock immediatelyand return true, or return false if the lock cannot be acquired immediately. Asyou can see, the name has the word try. This function attempts to acquire a lock.If it can't get the lock, then it won't wait. Now this function can be utilizedto build a queuing system.

Any usage of an advisory lock means the application needs to coordinate action. Itgives more power to the application but it also means more work by theapplication. In contrast, FOR UPDATE SKIP RECORD is natively supported byPostgreSQL.

Based on the discussionshere andhere,it seems GoodJob is evaluating the possibility of migrating from advisory locksto using FOR UPDATE SKIP LOCKED for better performance. Going through theseissues was quite revealing and I got to learn a lot about things I was unawareof.

Delayed job implementation

DelayedJob has been there since2009, long before Sidekiq. It doesn't use SKIP LOCK. Instead it uses a rowlevel locking system byupdating a field in the job recordto indicate that the job is being processed. In short DelayedJob ensures that notwo workers take the same job at the application level without taking any helpin this direction from the database.

What about SQLite

So far, we discussed PostgreSQL and MySQL. What about SQLite? Does it supportSKIP LOCK. No, it doesn't support it, but it's ok. As per thedocumentation, it supportsonly writer at any instant in time.

High Concurrency

SQLite supports an unlimited number of simultaneous readers, but it will onlyallow one writer at any instant in time. For many situations, this is not aproblem. Writers queue up. Each application does its database work quickly andmoves on, and no lock lasts for more than a few dozen milliseconds. But thereare some applications that require more concurrency, and those applicationsmay need to seek a different solution.

NOWAIT

For completeness, let's discuss the NOWAIT feature. We saw earlier thatif we take a lock on a row using FOR UPDATE, then other workers will wait untilthe lock is released.

START TRANSACTION;SELECT * FROM JOBS WHERE processed='no' FOR UPDATE NOWAIT LIMIT 1;-- Process the jobCOMMIT

NOWAIT feature allows other transactions to not wait for the lock to bereleased. In this case if a transaction is not able to get a lock on the givenrow then it will raise an error and the application needs to handle the error.

In contrast, SKIP LOCKED will allow the transaction to move on to the next rowif a lock is already taken.

Redis backed queue vs database backed queue

Now that we looked at how FOR UPDATE SKIP LOCK helps build queuing systemusing database itself, let's see some pros and cons of each type of queuingsystem.

Simplicity and familiarity

Database-backed queues are often simpler to set up and manage, especially ifyour application is already using a relational database. There's no need for anadditional dependency like Redis.

No Additional Infrastructure

Since the job information is stored in the same database as your applicationdata, you don't need to set up and maintain a separate infrastructure like aRedis server.

Transactionality

Database-backed queues can leverage database transactions, ensuring that boththe job creation and any related database operations are committed or rolledback together. This can be important in scenarios where data consistency iscritical.

Modifiability

It is easier to modify the jobs stored in the database than Redis, but doing sorequires caution, and it's generally not recommended. In Redis, jobs are oftenstored as serialized data, and modifying them directly is not as straightforwardor common. Redis provides commands to interact with data, but modifying job datadirectly is not a standard practice and could result in data corruption.