Operations
Operations
Externa depends on a long-running queue worker and Laravel’s scheduler for file zips/uploads, AI attachments, activity-log retention, and AI sync sources. Without them, async features stall and temporary storage grows.
Queue worker (required)
Default in .env.example for the full stack: QUEUE_CONNECTION=redis with Horizon. Cloners without Redis should use QUEUE_CONNECTION=database (see Minimal vs full stack and Configuration).
Jobs that expect a worker include:
- Multi-file / folder zip preparation
- Large / bulk file duplicates
- Collection import jobs started from AI tools
- Outbound webhook delivery (
DeliverOutboundWebhookJob) — see Outbound webhooks
With Redis + Horizon, those jobs are consumed by php artisan horizon instead of queue:listen. Queue health lives on the dashboard Health tab (native cards — no primary “Open Horizon” CTA); failed/retry deep-dive: /horizon (super-admin).
Local
Minimal (composer run dev) starts:
php artisan queue:listen --tries=1 --timeout=0
Full stack with Herd Redis + Herd Reverb (recommended):
php84 artisan horizon
php84 artisan pulse:work
Herd Reverb is already on :8080 — match REVERB_* / VITE_REVERB_* and skip reverb:start. Standalone all-in-one: composer run dev:full (includes reverb:start; conflicts if Herd already binds :8080). See Redis prerequisite and Minimal vs full stack.
Production
Prefer Horizon when using Redis:
php artisan horizon
Also run a Reverb process (or managed WebSocket) and pulse:work when Pulse redis ingest is enabled. Or a plain worker:
php artisan queue:work --tries=3 --timeout=300
Use your process manager (Supervisor, systemd, Forge, Cloud Run sidecars, etc.). Restart workers after each deploy so they pick up new code.
Do not use sync in production
QUEUE_CONNECTION=sync runs jobs inline and will block HTTP requests on large zips/imports. Keep a real queue driver and a worker.
Scheduler
Defined in routes/console.php:
| Command | Frequency | Purpose |
|---|---|---|
activitylog:clean | daily | Purge old Spatie activity log rows per package config |
files:cleanup-uploads | hourly | Remove expired partial/chunked uploads |
files:cleanup-zips | hourly | Remove prepared zips past FILES_ZIP_TTL_MINUTES |
ai:cleanup-attachments | daily | Delete expired AI chat attachments + files |
ai:run-sync-sources | everyMinute | Run due AI collection sync sources |
The scheduler only fires when something invokes Laravel’s schedule runner every minute:
* * * * * cd /path/to/externa-core && php artisan schedule:run >> /dev/null 2>&1
On platforms with a native scheduler (Laravel Cloud, Forge, etc.), enable the equivalent cron / schedule feature.
Manual runs
php artisan activitylog:clean
php artisan files:cleanup-uploads
php artisan files:cleanup-zips
php artisan ai:cleanup-attachments
php artisan ai:run-sync-sources
php artisan schedule:run # runs due events once
Permissions sync
When PermissionEnum changes (or after deploy of permission-related code):
php artisan permissions:sync
# remove DB permissions no longer in the enum:
php artisan permissions:sync --prune
Also available from the admin UI via POST /settings/permissions/sync (can-edit-permissions). Sync upserts Spatie permission rows and regenerates resources/js/enums/permission-enum.ts.
After first install / empty DB, prefer php artisan db:seed (permissions → roles → super admin). See Configuration.
Health check
Registered in bootstrap/app.php:
GET /up
Use this for load balancers and uptime monitors. It is Laravel’s framework health endpoint (not an authenticated app route).
Jobs monitor (admin UI)
Settings → Jobs (/settings/jobs):
| Permission | Capability |
|---|---|
can-show-jobs | View pending sample + failed jobs list |
can-manage-jobs | Retry / delete failed jobs |
Reads Laravel jobs and failed_jobs (database queue). Domain status UIs (AI import job UUID, zip preparation) stay in their existing screens — this page is queue ops only, not a workflow builder.
Routes: jobs.index, jobs.retry, jobs.destroy in routes/admin.php. Controller: JobMonitorController.
Performance settings
Settings → Performance (/settings/performance):
| Permission | Capability |
|---|---|
can-manage-project-settings | View status, flush targeted caches |
Operators see nested ops status (cache store, Redis reachable, queue connection, app env/debug), bootstrap cache read-only (config/routes/events/packages cached yes/no), public API cache parameters (TTL 120s safety net, epoch counter), and observability quick links when permitted (Jobs, Pulse, Horizon). Four targeted flush actions let you invalidate specific subsystems without a full artisan optimize:clear:
- Public API cache — bumps the global epoch counter so all collection responses miss immediately (next hit rebuilds with fresh permission matrix).
- Permission matrices — forgets cached collection/file permission matrices for every role (Forces lazy rebuild on next role check).
- Spatie permissions — flushes Spatie's permission registrar cache (when
PermissionEnumchanges or roles/permissions are mutated). - Dashboard metrics — forgets the 13 dashboard metric keys (fresh metrics on next dashboard load).
Bootstrap cache status is read-only — no artisan optimize buttons from the UI; prefer php artisan optimize / php artisan optimize:clear in your deploy script or CI (or do not cache at all in local).
Observability links appear when the user has the relevant permission:
- Jobs —
can-show-jobs→/settings/jobs - Pulse —
viewPulsegate (super-admin) →/pulse - Horizon —
viewHorizongate (super-admin) →/horizon
Routes: performance.edit, performance.flushPublicApi, performance.flushPermissionMatrices, performance.flushSpatiePermissions, performance.flushDashboardMetrics in routes/admin.php. Controller: PerformanceSettingsController.
Observability
Laravel Pail
Dev script includes:
php artisan pail --timeout=0
Tail application logs in real time during local development. In production, rely on your log stack (LOG_CHANNEL, centralized logging).
Activity log
Spatie Laravel Activitylog records auth and domain mutations (configurable via config/activitylog.php, env ACTIVITYLOG_ENABLED).
- UI:
/activity-logs(can-show-activity-logs) - Cleanup:
activitylog:clean(daily schedule) - AI:
QueryActivityLogstool for users with show permission
Notifications
Unread counts are shared on every Inertia page (notifications.unread_count). With Reverb + Echo, the sidebar bell listens on the private user channel; the 60s unread poll runs only when BROADCAST_CONNECTION is log/null. JSON helpers:
GET /notificationsGET /notifications/unread-countPOST /notifications/read
(all auth + verified). Details: Reverb & Echo · Activity & notifications.
Operational checklist
- Worker process up and processing the
defaultqueue. - Cron / platform scheduler running
schedule:runevery minute. storage:linkpresent if serving public/assets URLs.permissions:syncafter permission enum deploys./upmonitored.- Disk space watched under
storage/app(uploads, zips, AI attachments, logs). - Run pending migrations before relying on revisions / M2M junction shape (
collection_item_revisions, junction object migration). - Prefer
LIGHTHOUSE_QUERY_CACHE_MODE=opcache(default) so GraphQL parse cache does not store AST objects in the database cache driver. - Walk the Security checklist before production.
- After deploy, smoke the Public API / GraphQL with externa-bruno (duplicate Local, set staging
base_url+ key) — see Public CMS API.