Operations

Operations

Externa depends on a long-running queue worker and Laravel’s scheduler for file zips/uploads, AI attachments, activity-log retention, and AI sync sources. Without them, async features stall and temporary storage grows.

Queue worker (required)

Default in .env.example for the full stack: QUEUE_CONNECTION=redis with Horizon. Cloners without Redis should use QUEUE_CONNECTION=database (see Minimal vs full stack and Configuration).

Jobs that expect a worker include:

  • Multi-file / folder zip preparation
  • Large / bulk file duplicates
  • Collection import jobs started from AI tools
  • Outbound webhook delivery (DeliverOutboundWebhookJob) — see Outbound webhooks

With Redis + Horizon, those jobs are consumed by php artisan horizon instead of queue:listen. Queue health lives on the dashboard Health tab (native cards — no primary “Open Horizon” CTA); failed/retry deep-dive: /horizon (super-admin).

Local

Minimal (composer run dev) starts:

php artisan queue:listen --tries=1 --timeout=0

Full stack with Herd Redis + Herd Reverb (recommended):

php84 artisan horizon
php84 artisan pulse:work

Herd Reverb is already on :8080 — match REVERB_* / VITE_REVERB_* and skip reverb:start. Standalone all-in-one: composer run dev:full (includes reverb:start; conflicts if Herd already binds :8080). See Redis prerequisite and Minimal vs full stack.

Production

Prefer Horizon when using Redis:

php artisan horizon

Also run a Reverb process (or managed WebSocket) and pulse:work when Pulse redis ingest is enabled. Or a plain worker:

php artisan queue:work --tries=3 --timeout=300

Use your process manager (Supervisor, systemd, Forge, Cloud Run sidecars, etc.). Restart workers after each deploy so they pick up new code.

Do not use sync in production

QUEUE_CONNECTION=sync runs jobs inline and will block HTTP requests on large zips/imports. Keep a real queue driver and a worker.

Scheduler

Defined in routes/console.php:

CommandFrequencyPurpose
activitylog:cleandailyPurge old Spatie activity log rows per package config
files:cleanup-uploadshourlyRemove expired partial/chunked uploads
files:cleanup-zipshourlyRemove prepared zips past FILES_ZIP_TTL_MINUTES
ai:cleanup-attachmentsdailyDelete expired AI chat attachments + files
ai:run-sync-sourceseveryMinuteRun due AI collection sync sources

The scheduler only fires when something invokes Laravel’s schedule runner every minute:

* * * * * cd /path/to/externa-core && php artisan schedule:run >> /dev/null 2>&1

On platforms with a native scheduler (Laravel Cloud, Forge, etc.), enable the equivalent cron / schedule feature.

Manual runs

php artisan activitylog:clean
php artisan files:cleanup-uploads
php artisan files:cleanup-zips
php artisan ai:cleanup-attachments
php artisan ai:run-sync-sources
php artisan schedule:run   # runs due events once

Permissions sync

When PermissionEnum changes (or after deploy of permission-related code):

php artisan permissions:sync
# remove DB permissions no longer in the enum:
php artisan permissions:sync --prune

Also available from the admin UI via POST /settings/permissions/sync (can-edit-permissions). Sync upserts Spatie permission rows and regenerates resources/js/enums/permission-enum.ts.

After first install / empty DB, prefer php artisan db:seed (permissions → roles → super admin). See Configuration.

Health check

Registered in bootstrap/app.php:

GET /up

Use this for load balancers and uptime monitors. It is Laravel’s framework health endpoint (not an authenticated app route).

Jobs monitor (admin UI)

Settings → Jobs (/settings/jobs):

PermissionCapability
can-show-jobsView pending sample + failed jobs list
can-manage-jobsRetry / delete failed jobs

Reads Laravel jobs and failed_jobs (database queue). Domain status UIs (AI import job UUID, zip preparation) stay in their existing screens — this page is queue ops only, not a workflow builder.

Routes: jobs.index, jobs.retry, jobs.destroy in routes/admin.php. Controller: JobMonitorController.

Performance settings

Settings → Performance (/settings/performance):

PermissionCapability
can-manage-project-settingsView status, flush targeted caches

Operators see nested ops status (cache store, Redis reachable, queue connection, app env/debug), bootstrap cache read-only (config/routes/events/packages cached yes/no), public API cache parameters (TTL 120s safety net, epoch counter), and observability quick links when permitted (Jobs, Pulse, Horizon). Four targeted flush actions let you invalidate specific subsystems without a full artisan optimize:clear:

  1. Public API cache — bumps the global epoch counter so all collection responses miss immediately (next hit rebuilds with fresh permission matrix).
  2. Permission matrices — forgets cached collection/file permission matrices for every role (Forces lazy rebuild on next role check).
  3. Spatie permissions — flushes Spatie's permission registrar cache (when PermissionEnum changes or roles/permissions are mutated).
  4. Dashboard metrics — forgets the 13 dashboard metric keys (fresh metrics on next dashboard load).

Bootstrap cache status is read-only — no artisan optimize buttons from the UI; prefer php artisan optimize / php artisan optimize:clear in your deploy script or CI (or do not cache at all in local).

Observability links appear when the user has the relevant permission:

  • Jobscan-show-jobs/settings/jobs
  • PulseviewPulse gate (super-admin) → /pulse
  • HorizonviewHorizon gate (super-admin) → /horizon

Routes: performance.edit, performance.flushPublicApi, performance.flushPermissionMatrices, performance.flushSpatiePermissions, performance.flushDashboardMetrics in routes/admin.php. Controller: PerformanceSettingsController.

Observability

Laravel Pail

Dev script includes:

php artisan pail --timeout=0

Tail application logs in real time during local development. In production, rely on your log stack (LOG_CHANNEL, centralized logging).

Activity log

Spatie Laravel Activitylog records auth and domain mutations (configurable via config/activitylog.php, env ACTIVITYLOG_ENABLED).

  • UI: /activity-logs (can-show-activity-logs)
  • Cleanup: activitylog:clean (daily schedule)
  • AI: QueryActivityLogs tool for users with show permission

Notifications

Unread counts are shared on every Inertia page (notifications.unread_count). With Reverb + Echo, the sidebar bell listens on the private user channel; the 60s unread poll runs only when BROADCAST_CONNECTION is log/null. JSON helpers:

  • GET /notifications
  • GET /notifications/unread-count
  • POST /notifications/read

(all auth + verified). Details: Reverb & Echo · Activity & notifications.

Operational checklist

  1. Worker process up and processing the default queue.
  2. Cron / platform scheduler running schedule:run every minute.
  3. storage:link present if serving public/assets URLs.
  4. permissions:sync after permission enum deploys.
  5. /up monitored.
  6. Disk space watched under storage/app (uploads, zips, AI attachments, logs).
  7. Run pending migrations before relying on revisions / M2M junction shape (collection_item_revisions, junction object migration).
  8. Prefer LIGHTHOUSE_QUERY_CACHE_MODE=opcache (default) so GraphQL parse cache does not store AST objects in the database cache driver.
  9. Walk the Security checklist before production.
  10. After deploy, smoke the Public API / GraphQL with externa-bruno (duplicate Local, set staging base_url + key) — see Public CMS API.
Previous
Outbound webhooks