CentercodePorygon
DittoPorygonAPI

Background Jobs

Event-driven background processing with Inngest

FastReliable

Overview

Use Inngest for background processing, long-running tasks, and scheduled jobs. Send events with dispatch(), handle them in registered functions.

What it is

Inngest provides event-driven serverless functions with automatic retries, step functions, and observability.

Why we use it

Non-blocking request handling, automatic retries on failure, step-based workflows, and built-in monitoring.

When to use

Operations longer than HTTP timeout (10s+), AI tasks, file processing, email sending, or any work that should not block the request.

Key Features

  • Type-safe event definitions and handlers
  • Automatic retries with configurable attempts
  • Step functions for atomic, resumable workflows
  • Concurrency limits and rate limiting

Quick Start

Sending Events

Trigger a background job from any part of your app.

// 1. Send an event from anywhere
import { dispatch } from '@/lib/jobs';

await dispatch({
  name: 'context/file.uploaded',
  data: {
    fileId: 'file_123',
    contextId: 'ctx_456',
    uploadedById: 'user_789',
  },
});

// 2. The function handles it in the background
// (defined in src/lib/jobs/functions/)

Patterns

Defining Events

Type-safe event definitions in types.ts.

// src/lib/jobs/types.ts - Define event types
export interface JobEvents {
  'system/health.check': {
    data: { timestamp: string };
  };
  'context/file.uploaded': {
    data: {
      fileId: string;
      contextId: string;
      uploadedById: string;
    };
  };
  'ai/generation.requested': {
    data: {
      prompt: string;
      userId: string;
      outputId: string;
    };
  };
}

// Type helpers for consumers
export type JobEventName = keyof JobEvents;
export type JobEventData<T extends JobEventName> = JobEvents[T]['data'];

Creating Functions

Event handlers with step-based execution.

// src/lib/jobs/functions/process-file.ts
import { defineJob } from '@/lib/jobs';
import { logger } from '@/lib/logger';

export const processContextFile = defineJob(
  {
    id: 'process-context-file',
    name: 'Process Context File',
    retries: 3,
    concurrency: { limit: 5 },
  },
  { event: 'context/file.uploaded' },
  async ({ event, step }) => {
    const { fileId, contextId, uploadedById } = event.data;

    // Step 1: Fetch file metadata
    const file = await step.run('fetch-file', async () => {
      logger.info({ fileId }, 'Fetching file metadata');
      return await getFileById(fileId);
    });

    // Step 2: Parse with AI
    const parsed = await step.run('parse-document', async () => {
      const { parseDocumentWithAI } = await import('@/lib/ai/files');
      return await parseDocumentWithAI(file.content, file.name);
    });

    // Step 3: Store result
    await step.run('store-result', async () => {
      await updateContextFile(fileId, {
        summary: parsed.summary,
        keyPoints: parsed.keyPoints,
      });
    });

    return { success: true, fileId };
  }
);

Use step.run() to make each operation resumable and retryable.

Function Registration

Registering functions in the API route.

// src/app/api/inngest/route.ts
import { jobs } from '@/lib/jobs';
import { healthCheck, processContextFile } from '@/lib/jobs/functions';

export const { GET, POST, PUT } = jobs.createHandler([
  healthCheck,
  processContextFile,
  // Add new functions here
]);

RLS Context (Critical)

Jobs have no user session - RLS bypass is required.

// CRITICAL: Jobs run without user context
// Without withRLSBypass, queries return empty results!

import { defineJob, withRLSBypass } from '@/lib/jobs';
import { prisma } from '@/lib/db';
import { logger } from '@/lib/logger';

export const processMessageJob = defineJob(
  { id: 'process-message', retries: 3 },
  { event: 'messaging/send.requested' },
  async ({ event, step }) => {
    const { messageId } = event.data;

    // REQUIRED: Wrap entire handler in withRLSBypass
    return await withRLSBypass('inngest:process-message', async () => {
      const message = await step.run('fetch-message', async () => {
        return await prisma.message.findUnique({
          where: { id: messageId },
        });
      });

      if (!message) {
        logger.error({ messageId }, 'Message not found');
        return { success: false };
      }

      // Process message...
      return { success: true };
    });
  }
);

Why bypass is safe in jobs:

  • Jobs have no user session (system-level operation)
  • Input is validated before job dispatch
  • Tagged context creates audit trail

Watch Out

Running long tasks in request handlers

Don't

// Blocking the request thread
export async function POST(request: Request) {
  const data = await request.json();

  // This takes 30+ seconds!
  await parseDocumentWithAI(data.content);

  return NextResponse.json({ ok: true });
}

Do

// Offload to background job
import { dispatch } from '@/lib/jobs';

export async function POST(request: Request) {
  const data = await request.json();

  // Send to background, return immediately
  await dispatch({
    name: 'context/file.uploaded',
    data: { fileId: data.fileId },
  });

  return NextResponse.json({ ok: true, status: 'processing' });
}
  • Missing withRLSBypass() wrapper (queries return empty)
  • Functions that aren't idempotent (unsafe retries)
  • Not configuring appropriate retry counts
  • Missing logging in background functions

Related

AI Integration

AI tasks in background

File Storage

File processing jobs

Email

Async email sending

External Documentation

Inngest Documentation