← Back to Articles
#laravel#queues#redis#backend

Queue Job Batching & Idempotency in Laravel

"Production-grade background processing in Laravel: job batching, atomic locks, and idempotent webhook handling."

•By huud•
0 views
~2 min read
Queue Job Batching & Idempotency in Laravel

Queue Job Batching & Idempotency in Laravel

Background queues form the backbone of high-throughput web applications. Two critical design pillars for production reliability are Job Batching and Job Idempotency.

1. Job Batching (Bus::batch)

Job batching allows dispatching a group of jobs with completion tracking, cancellation, and failure callbacks.

Database Setup

Ensure the batch table exists:

bash 
php artisan queue:batches-table
php artisan migrate

Job Implementation

Add the Batchable trait to the job:

php 
namespace App\Jobs;

use Illuminate\Bus\Batchable;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class ProcessUserReport implements ShouldQueue
{
    use Batchable, Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public function __construct(public int $userId) {}

    public function handle(): void
    {
        if ($this->batch()?->cancelled()) {
            return;
        }

        // Process report logic...
    }
}

Dispatching the Batch

php 
use App\Jobs\ProcessUserReport;
use Illuminate\Support\Facades\Bus;
use Throwable;

$jobs = User::pluck('id')->map(fn ($id) => new ProcessUserReport($id));

$batch = Bus::batch($jobs)
    ->then(function () {
        Log::info('All user reports generated successfully.');
    })
    ->catch(function (Throwable $e) {
        Log::error('Batch processing failed: ' . $e->getMessage());
    })
    ->name('Export User Reports')
    ->dispatch();

2. Idempotency: Handling Retries and Webhook Duplication

Because distributed message queues operate on at-least-once delivery, jobs and payment webhooks can be delivered more than once. An idempotent job ensures that multiple executions produce the same state as a single execution.

Atomic Lock Pattern for Duplicate Prevention

php 
namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Support\Facades\Cache;
use Illuminate\Support\Facades\DB;

class ProcessPaymentWebhook implements ShouldQueue
{
    use Queueable;

    public function __construct(
        public string $eventReference,
        public array $payload
    ) {}

    public function handle(): void
    {
        $lock = Cache::lock('webhook:' . $this->eventReference, 60);

        if (! $lock->get()) {
            // Lock active: already being processed
            return;
        }

        try {
            DB::transaction(function () {
                $payment = Payment::where('reference', $this->eventReference)->first();

                if ($payment && $payment->status === 'paid') {
                    return; // Already processed
                }

                $payment->update([
                    'status' => 'paid',
                    'paid_at' => now(),
                ]);
            });
        } finally {
            $lock->release();
        }
    }
}

Summary

  1. Job Batching organizes bulk operations and lifecycle hooks.
  2. Check $this->batch()?->cancelled() in batchable jobs to prune work early upon failures.
  3. Design all workers and webhooks to be idempotent using Atomic Locks and state checks before committing database transactions.

About the Author

huud

huud

@huud

About →

Systems architect and software engineer building high-performance distributed platforms.