Migrate task queue from in-memory to BullMQ + Redis
Publish Image / publish (push) Successful in 36s

Replace the volatile in-memory Map/Set queue with BullMQ backed by Redis
for persistence across restarts, automatic retry with exponential backoff,
and non-blocking workspace cleanup.

- Two queues: workspace-create (concurrency-limited) and workspace-cleanup
  (with polling instead of sleep-based stop→delete)
- Active workspaces tracked in Redis hash for dedup across restarts
- Stale sweep every 10 minutes catches orphaned workspaces
- Graceful shutdown on SIGTERM/SIGINT
- Replace node-cron with BullMQ repeatable jobs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Daniel Samson
2026-04-12 15:40:12 +01:00
co-authored by Claude Opus 4.6
parent a6ff7b7fb5
commit 13e96e9356
16 changed files with 710 additions and 180 deletions
+69
View File
@@ -0,0 +1,69 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: redis-data
namespace: sdlc-orchestrator
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis
namespace: sdlc-orchestrator
spec:
replicas: 1
selector:
matchLabels:
app: redis
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: redis:7-alpine
args: ["--appendonly", "yes"]
ports:
- containerPort: 6379
volumeMounts:
- name: data
mountPath: /data
resources:
requests:
cpu: 50m
memory: 64Mi
limits:
cpu: 200m
memory: 256Mi
livenessProbe:
exec:
command: ["redis-cli", "ping"]
initialDelaySeconds: 5
periodSeconds: 30
readinessProbe:
exec:
command: ["redis-cli", "ping"]
initialDelaySeconds: 3
periodSeconds: 10
volumes:
- name: data
persistentVolumeClaim:
claimName: redis-data
---
apiVersion: v1
kind: Service
metadata:
name: redis
namespace: sdlc-orchestrator
spec:
selector:
app: redis
ports:
- port: 6379
targetPort: 6379