Laravel Queues in Production: Horizon, Batches, Retries and Graceful Deploys

Most Laravel applications get a queue the day someone notices a request taking eight seconds to send an email. ShouldQueue, a database connection, php artisan queue:work in a screen session, done. That setup survives until the first traffic spike, the first provider outage, or the first job that quietly fails ten thousand times overnight.

This tutorial covers the parts that separate a queue that works on your laptop from one you can run in production: choosing a driver, sizing workers with Horizon, batching, controlling retries and backoff, preventing overlap and duplicate work, handling failures on purpose, and deploying without losing jobs mid-run. Examples target Laravel 11+ and work unchanged on Laravel 13.

Pick the right connection

• sync — no queue at all. Correct for local development of unrelated features and for tests, wrong for anything else.
• database — fine up to modest volume. Laravel 11+ uses row locking that handles concurrency safely, and it is one less service to run. It gets expensive at high throughput because every worker polls a table.
• redis — the default choice for real workloads, and the only driver Horizon supports. Use a Redis instance dedicated to queues, separate from your cache: flushing the cache should never destroy pending jobs.
• sqs — good when you want a managed broker and do not need Horizon's dashboard. Note the 15-minute maximum delay and the 256 KB message limit.

Two rules regardless of driver. First, never pass large payloads into a job; pass an ID and re-query. Serialized Eloquent models are automatically reduced to identifiers by the SerializesModels trait, but arrays of rows are not. Second, give every queue a name and separate the fast from the slow:

class SendWelcomeEmail implements ShouldQueue
{
    use Queueable;

    public $queue = 'mail';
}

A single default queue means a 200-job video-encoding backlog delays every password reset behind it.

Run workers with Horizon

Horizon is a first-party dashboard and supervisor for Redis queues. Install it and you get autoscaling workers, per-queue throughput and wait-time metrics, retry-with-one-click for failed jobs, and job tagging.

composer require laravel/horizon
php artisan horizon:install

A production configuration usually looks like this:

// config/horizon.php
'environments' => [
    'production' => [
        'supervisor-interactive' => [
            'connection' => 'redis',
            'queue' => ['mail', 'default'],
            'balance' => 'auto',
            'autoScalingStrategy' => 'time',
            'minProcesses' => 2,
            'maxProcesses' => 20,
            'tries' => 3,
            'timeout' => 60,
            'memory' => 192,
        ],
        'supervisor-heavy' => [
            'connection' => 'redis',
            'queue' => ['reports', 'media'],
            'balance' => 'simple',
            'minProcesses' => 1,
            'maxProcesses' => 4,
            'tries' => 2,
            'timeout' => 900,
            'memory' => 512,
        ],
    ],
],

Points that matter in that block:

• Separate supervisors for separate shapes of work. Short jobs want a short timeout and many processes; long jobs want a long timeout, more memory, and few processes so they cannot exhaust your database connections.
• balance => 'auto' with autoScalingStrategy => 'time' scales processes toward the queue with the longest estimated wait, which is usually what users feel. The size strategy scales by job count instead — better when jobs take roughly equal time.
• timeout must be lower than the job's retry_after in config/queue.php. If retry_after is 90 and timeout is 120, a job can be released back to the queue and picked up by a second worker while the first is still running it. That is how you get duplicate charges.
• memory should be well under the container limit. Workers are long-lived PHP processes; they restart when they cross this line, which is the cheapest leak mitigation there is.

Set a wait threshold and alert on it, so growing latency pages you before customers do:

'waits' => ['redis:mail' => 30, 'redis:reports' => 300],

Retries, backoff and idempotency

Default retry behaviour is rarely what you want. Configure it per job:

class SyncInvoiceToLedger implements ShouldQueue
{
    use Queueable;

    public $tries = 5;
    public $maxExceptions = 2;
    public $timeout = 30;

    public function backoff(): array
    {
        return [10, 60, 300, 900];
    }

    public function retryUntil(): \DateTimeInterface
    {
        return now()->addHours(6);
    }
}

backoff() returning an array gives increasing delays per attempt — essential when the failure is a rate-limited third-party API, because retrying immediately five times just burns your quota. retryUntil() gives a wall-clock deadline, which is the honest way to express "this sync is worthless after the billing run".

Retries only make sense if the job is safe to run twice. Make handlers idempotent: check state before acting, and use a unique constraint or a lock rather than trusting your own reasoning.

class ChargeSubscription implements ShouldQueue, ShouldBeUnique
{
    use Queueable;

    public $uniqueFor = 3600;

    public function uniqueId(): string
    {
        return 'charge:'.$this->subscription->id;
    }

    public function handle(PaymentGateway $gateway): void
    {
        if ($this->subscription->fresh()->charged_for_period()) {
            return;
        }

        $gateway->charge($this->subscription, idempotencyKey: $this->uniqueId());
    }
}

ShouldBeUnique prevents a second copy from being queued. WithoutOverlapping (a middleware) prevents a second copy from running concurrently, releasing it back instead:

public function middleware(): array
{
    return [
        (new WithoutOverlapping($this->account->id))->releaseAfter(30)->expireAfter(180),
    ];
}

Always set expireAfter — otherwise a worker killed mid-job leaves a lock that blocks that key forever.

To respect an external API's limits, add RateLimited and define the limiter in a service provider:

RateLimiter::for('ledger-api', fn () => Limit::perMinute(60));

Batches for work that has a finish line

When you dispatch 5,000 jobs and need to know when they are all done, use a batch. Batches need the job_batches table (php artisan queue:batches-table if you do not have it).

use Illuminate\Bus\Batch;
use Illuminate\Support\Facades\Bus;

$batch = Bus::batch(
    $accounts->map(fn ($account) => new RebuildAccountReport($account->id))
)
    ->name('Monthly reports '.now()->format('Y-m'))
    ->onQueue('reports')
    ->allowFailures()
    ->then(fn (Batch $batch) => ReportsReady::dispatch($batch->id))
    ->catch(fn (Batch $batch, \Throwable $e) => report($e))
    ->finally(fn (Batch $batch) => Log::info('Batch finished', [
        'id' => $batch->id,
        'failed' => $batch->failedJobs,
    ]))
    ->dispatch();

Inside a batched job, $this->batch()?->cancelled() lets each job bail out early once the batch is cancelled — check it at the top of handle() for long batches. Store $batch->id on your own model so a UI can poll progress ($batch->progress()), and prune old rows with queue:prune-batches in the scheduler.

Related structures worth knowing: Bus::chain([...]) runs jobs strictly in order and stops on the first failure; chained batches (Bus::chain([Bus::batch([...]), new FinaliseJob])) run a parallel stage, then a sequential one.

Fail on purpose

A job that throws for a reason that will never resolve should not retry. Delete it:

public function handle(): void
{
    $order = Order::find($this->orderId);

    if (! $order) {
        $this->delete(); // Deleted between dispatch and execution: nothing to do.
        return;
    }

    if ($order->isCancelled()) {
        $this->fail(new OrderCancelled($order->id)); // Record it, do not retry.
        return;
    }

    // ...
}

Give every job a failed() method that does something useful — mark the record, notify the owner, emit a metric:

public function failed(?\Throwable $e): void
{
    $this->subscription->update(['sync_state' => 'failed']);

    Log::error('Subscription sync failed', [
        'subscription' => $this->subscription->id,
        'error' => $e?->getMessage(),
    ]);
}

Then watch the failed table. queue:failed lists, queue:retry {id} or queue:retry all re-runs, queue:flush clears. The number worth alerting on is the rate of new failures, not the total; a dashboard that shows 12,000 failed jobs from last year teaches everyone to ignore it.

Deploy without dropping jobs

Workers boot your application once and keep it in memory, so a deployment that only swaps files leaves workers running old code. The sequence:

php artisan down --render="errors::503"   # optional, for schema-breaking releases
# deploy code, run migrations
php artisan config:cache && php artisan event:cache
php artisan queue:restart                 # or: php artisan horizon:terminate
php artisan up

queue:restart and horizon:terminate tell workers to finish the job in hand and exit gracefully; your process manager (Supervisor, systemd, or the platform's own) starts them again on the new code. On Laravel Cloud or a Forge server with Horizon installed, hook php artisan horizon:terminate into the deploy script and let the daemon do the rest — see our Laravel hosting and deployment engineering page for how we wire this up.

Two compatibility traps during rolling deploys: old workers may pick up payloads for jobs whose constructor signature you just changed, and new workers may pick up payloads serialized by old code. Add new constructor arguments as optional with defaults, and drain the queue before removing a job class entirely.

Instrument it

Numbers to keep on a dashboard, all available from Horizon's metrics or your APM:

• Wait time per queue — the user-facing figure. Alert here.
• Throughput and runtime per job class — spot the job that grew from 200 ms to 9 seconds after a feature launch.
• Failed jobs per hour — a rate, not a total.
• Worker memory and restarts — steady climbs mean a leak, usually an unbounded array or an ever-growing query log.

Also queue-tag jobs (public function tags(): array) with a customer or tenant identifier so Horizon lets you filter a specific incident.

A short checklist

• Redis connection dedicated to queues; separate named queues per work shape.
• timeout < retry_after, everywhere.
• Explicit tries, backoff() and retryUntil() on anything touching a third-party API.
• Idempotent handlers, with ShouldBeUnique or WithoutOverlapping where duplicates would hurt.
• failed() on every job; alert on failure rate.
• horizon:terminate in the deploy script; queue:prune-batches and queue:prune-failed in the scheduler.
• Load-test the queue, not just the web tier. Workers are where capacity problems hide.

Queues are where a healthy Laravel application quietly starts costing money — in duplicate side effects, in silent failures, in workers scaled by guesswork. If you would like a second pair of eyes on yours, we do this as part of Laravel performance optimization and ongoing support and maintenance, or just get in touch with a description of your setup.