Skip to main content
Canary deployment lets us run a separate app instance on a newer (or experimental) build and route a specific subset of platforms to it, while all other traffic continues hitting the primary app. This gives us real-world validation with live data before a full rollout.

​
Architecture Overview

Browser / External Webhook
         │
         ▼
   Primary App (prod)
   ┌─────────────────────────────┐
   │  preHandler hook            │
   │  canaryRoutingMiddleware    │
   │    ├─ platformId ∈ canary?  │──── No ──▶ handle normally
   │    └─ Yes                   │
   │         ▼                   │
   │  HTTP proxy to canary app   │──────────▶ Canary App
   └─────────────────────────────┘               │
                                                  ▼
                                         Canary Worker
                                         (polls canaryWorkerJobs queue)
All traffic enters the primary app. The canaryRoutingMiddleware runs as a Fastify preHandler hook on every request. If the resolved platformId is in the canary list, the request is proxied in-process to the canary app and the response is returned directly to the caller — the primary app never processes it further.

​
Canary membership: DB-backed with Redis cache

Canary platform membership is stored in the platform_plan.canary boolean column. On each lookup, the list of canary platform IDs is fetched from Redis (canary-platform-ids key). On cache miss the list is read from the database and cached. When the canary flag is changed via the API, the cache is invalidated immediately. This removes the need to keep AP_CANARY_PLATFORM_IDS in sync across services and allows runtime changes without a redeploy.

​
Components

​
Canary App

A second instance of the server API running a different image tag. No special env vars are needed for canary membership — it is driven entirely by the DB. Configure it with:
Env varValue
AP_FRONTEND_URLcanary app’s public URL
The canary app runs as a full app instance. Global queue consumers (runsMetadata, system-job-queue) run on both the primary and canary app — this is safe because BullMQ ensures each job is processed by exactly one consumer.

​
Canary Workers

Dedicated workers that connect to the canary app instead of the primary app:
Env varValue
AP_FRONTEND_URLcanary app’s URL
AP_IS_CANARY_WORKERtrue
Workers use AP_FRONTEND_URL for their Socket.IO RPC channel and for posting engine results back to the app. Canary workers are registered with the canary app, so the RPC path is fully isolated.

​
Primary App Config

The primary app only needs to know where to proxy canary requests:
Env varDescription
AP_CANARY_APP_URLInternal URL of the canary app

​
WebSocket / Real-time Updates

Socket.IO is configured with a Redis adapter (@socket.io/redis-adapter). Events emitted on any app instance (primary or canary) are broadcast through Redis pub/sub to all connected instances. This means:
  • Users connected to the primary app receive real-time flow run updates even when the execution happened on the canary app.
  • No WebSocket proxying is required.

​
Queue Isolation

QueuePrimary AppCanary App
workerJobs✅ Consumed by primary workersNot consumed
runsMetadata✅ Consumed✅ Consumed
system-job-queue✅ Consumed✅ Consumed
canaryWorkerJobsNot consumed✅ Consumed by canary workers
Jobs for canary platforms are enqueued to canaryWorkerJobs, which only canary workers poll — fully isolated from the primary worker fleet.

​
Deploying a New Canary Build

The Continuous Delivery — Canary workflow (continuous-delivery-canary.yml) runs automatically every day at 9 AM UTC, building from the latest main. The workflow:
  1. Builds and pushes a new image tagged <version>.<sha>.canary
  2. Checks if any new migrations are breaking — fails the workflow if they are (no override)
  3. Deploys the canary app (config/app-canary.yml) and canary workers
Required GitHub secrets: AP_API_KEY (the primary app’s AP_API_KEY value).

​
Rolling Back a Canary Deployment

Trigger the Continuous Delivery — Rollback Canary workflow (continuous-delivery-rollback-canary.yml) with the image tag you want to roll back to. The workflow:
  1. Extracts the migration manifest from the target image
  2. Rolls back DB migrations not present in that manifest
  3. Redeploys the canary app and workers to the target image
InputDescription
rollback_to_image_tagImage tag to roll back to (e.g. 0.51.0.abc1234.canary)
forceForce rollback even if breaking migrations exist. Default: false.

​
Promoting canary build to production

Canary is a validation environment, not a promotion path. Full promotion happens via the Sunday scheduled cloud workflow, which deploys release-candidate to production. If a canary build has been validated and the corresponding commit has been tagged as release-candidate, it will automatically reach production on the next Sunday.

​
Managing Platform Routing

​
Enable canary routing for a platform

curl -X POST https://cloud.activepieces.com/v1/admin/platforms/canary \
  -H "api-key: <AP_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"platformId": "<platformId>", "canary": true}'

​
Disable canary routing for a platform

curl -X POST https://cloud.activepieces.com/v1/admin/platforms/canary \
  -H "api-key: <AP_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"platformId": "<platformId>", "canary": false}'
Both calls update platform_plan.canary in the database and invalidate the Redis cache (canary-platform-ids). The change takes effect on the next request — no restart required.