Background Jobs

A large part of Sim runs on a schedule rather than in response to a user request: scheduled workflows, every polling trigger, connector syncs, the outbox, data drains, and retention. All of it is driven by HTTP endpoints that something external must call on a timer.

Both deployments ship a scheduler and enable it by default: Kubernetes as CronJobs, Docker Compose as a cron service. Both authenticate with CRON_SECRET, and both use the same schedules.

Authentication

Every endpoint is protected by CRON_SECRET and expects it as a bearer token:

curl -f -s -S --max-time 60 \
  -H "Authorization: Bearer $CRON_SECRET" \
  https://sim.yourdomain.com/api/schedules/execute

Generate it like the other secrets:

openssl rand -hex 32

Point cron at an internal address where possible (the in-cluster Service, or localhost on a single node). These endpoints should not be reachable from the internet; if they are, CRON_SECRET is the only thing protecting them.

The jobs

Kubernetes

Enabled by default. Nothing to do beyond setting CRON_SECRET.

cronjobs:
  enabled: true

Each job runs a small curlimages/curl pod that calls the app's in-cluster Service (not the ingress), with concurrencyPolicy: Forbid so a slow run never overlaps itself. The Service load-balances that single call to one replica, so a job never fans out across pods. The chart keeps three completed and one failed Job per CronJob, but ttlSecondsAfterFinished defaults to 600, so a finished Job is deleted ten minutes after it ends — inspect a failure promptly or collect the logs. It sets no backoffLimit, so the Kubernetes default of six applies and the chart exposes no key to change it. cronjobs.activeDeadlineSeconds (default 300) bounds the whole Job — retries and the controller's back-off included — so whichever limit is reached first ends the Job — six retries after the initial failure, up to seven attempts, or five minutes of attempts plus back-off. The same deadline cuts off a single hung request. One attempt is already several HTTP calls: the container retries three times, ten seconds apart, with curl --retry 2 inside each.

Disable individual jobs you do not need — billing reconciliation is the obvious one on a self-hosted install:

cronjobs:
  jobs:
    reconcileBillingSeats:
      enabled: false

Do not disable schedule execution, outbox processing, or subscription renewal unless you know you do not use the features behind them.

Check they are running:

kubectl get cronjobs -n simstudio
kubectl get jobs -n simstudio --sort-by=.metadata.creationTimestamp | tail
kubectl logs -n simstudio job/<job-name>

A CronJob whose LAST SCHEDULE is stale, or whose jobs are failing, means the corresponding feature is dead. Alert on it — see Observability.

Docker Compose

The cron service runs the same jobs on the same schedules, so nothing to configure beyond CRON_SECRET:

CRON_SECRET=$(openssl rand -hex 32)

Without it the cron service logs exactly what to set — including a freshly generated value — and exits, leaving the rest of the stack running. Schedules live in docker/crontab and mirror helm/sim/values.yaml cronjobs.jobs one-for-one.

The service runs supercronic rather than the app image: it logs each job's output to the container log, forwards SIGTERM so docker compose stop is graceful, and will not start an iteration while the previous one is still running.

docker compose -f docker-compose.prod.yml logs -f cron

A healthy log line looks like:

level=info msg=starting iteration=0 job.schedule="*/1 * * * *"
level=info msg="job succeeded" iteration=0

To drop a job you do not need, comment out its line in docker/crontab and restart the service.

Verifying

Create a workflow with a Schedule trigger set to every minute, deploy it, and watch the Logs view. An execution should appear within ~2 minutes. If nothing appears:

1. Check the scheduler's own logs — docker compose -f docker-compose.prod.yml logs cron, or kubectl get cronjobs -n simstudio for a recent LAST SCHEDULE.
2. Confirm the app and the scheduler share the same CRON_SECRET. A mismatch shows up as 401 in the scheduler log.
3. A 202 means the endpoint accepted the run; it does not confirm a schedule was due, so check the Logs view.

Concurrency

Scheduled execution volume is bounded per app instance by:

Raise the schedule limit only alongside memory headroom: concurrent executions run in the app process, so throughput is bounded by the pod's memory before it is bounded by this number.