Multi-Agent Task Supervision and Process Isolation in Node.js

Node.js Multi-Agent Task Supervision: Process Isolation and Heartbeats

SitePoint TeamPublished inProgramming·JavaScript·AI·
September 25, 2026

The AI briefing for Developers

Stay up to date with AI tools, model releases, and developer workflows that matter.

Weekly. Free. One click to leave.

SitePoint Premium
Stay Relevant and Grow Your Career in Tech

• Premium Results
• Publish articles on SitePoint
• Daily curated jobs
• Learning Paths
• Discounts to dev tools

7 Day Free Trial. Cancel Anytime.

As AI-powered background agents become standard fixtures in production Node.js systems, the gap between “spawn and forget” prototypes and supervision that detects failures and kills stuck workers within a bounded timeout keeps growing. A Node.js multi-agent supervisor daemon that combines process isolation, IPC heartbeat monitoring, timeout-driven termination, and concurrency token buckets prevents long-running, unpredictable agent workloads from destabilizing the host process. This guide walks through a complete, production-grade TypeScript architecture for doing exactly that.

Prerequisites: All code in this guide requires Node.js ≥ 18 LTS and TypeScript ≥ 5.0. Your package.json must include “type”: “module” for ESM imports with the Node16 module setting. All file paths at runtime must reference compiled .js output in dist/, not .ts

Table of Contents

Why Node.js Needs Supervisor Patterns for AI Agents

The Problem with Unsupervised Background Work

AI agents are fundamentally different from typical request-response handlers. They run for seconds to minutes, and their memory footprint is hard to predict since it grows with accumulated context (often 100 MB to 1 GB per agent conversation, depending on context window size). They frequently make external API calls with variable latency. An agent stuck in an infinite reasoning loop, leaking memory through unbounded conversation history, or throwing unhandled promise rejections will not politely clean up after itself. Node.js runs on a single-threaded event loop, so one misbehaving agent can starve all other work of CPU time. Spawning agents with no oversight turns a single failure into a cascading system-wide problem.

What a Supervisor Daemon Actually Does

The supervisor pattern, borrowed conceptually from Erlang/OTP’s supervision trees, dedicates a process to managing the lifecycle of worker processes. This implementation does not replicate OTP’s restart strategy specifications (one-for-one, one-for-all, or rest-for-one). In Node.js terms, the supervisor spawns agents as worker threads or child processes, monitors their health through heartbeat messages, enforces execution time limits, restarts failed agents according to policy, and rate-limits concurrent execution through a token bucket. The architecture built throughout this guide composes four discrete modules: a heartbeat protocol, a worker manager, a token bucket, and an orchestrating supervisor class that runs as a single long-running process.

Node.js runs on a single-threaded event loop, so one misbehaving agent can starve all other work of CPU time.

Architecture Overview

Component Map

The system comprises five cooperating components. The supervisor process runs on the main thread and owns all coordination logic. Worker threads, created via node:worker_threads, handle lightweight, in-process agent tasks where shared memory access is beneficial. Child processes, spawned through node:child_process, provide full V8 isolate separation for untrusted or memory-intensive agents. Between the supervisor and every worker or child, an IPC heartbeat channel carries structured liveness messages at regular intervals. At the entry point of the spawn path, a concurrency token bucket governs the maximum number of parallel agents and shapes burst behavior.

When to Use Worker Threads vs. Child Processes

The decision between worker_threads and child_process is not aesthetic. It hinges on concrete trade-offs across six dimensions:

For agents that need access to shared state or where startup latency matters, worker threads are appropriate. For untrusted code, agents with unpredictable memory profiles, or workloads where a crash must never touch the supervisor, child processes are the correct choice.

Code Example #1: Project scaffolding and type definitions

{"compilerOptions":{"target":"ES2022","module":"Node16","moduleResolution":"Node16","strict":true,"outDir":"./dist","rootDir":"./src","declaration":true},"include":["src/**/*.ts"]}

{"type":"module","scripts":{"build":"tsc","start":"node dist/agent-worker-supervisor.js"},"devDependencies":{"typescript":"^5.0.0","@types/node":"^20.0.0"}}

exportinterfaceAgentTask{id:string;type:string;priority:number;payload: Record<string,unknown>;maxDurationMs:number;isolationStrategy:'thread'|'process';}exportinterfaceWorkerState{workerId:string;taskId:string;status:'running'|'terminating'|'terminated';startedAt:number;pid?:number;}exportinterfaceHeartbeatMessage{type:'heartbeat';workerId:string;timestamp:number;payload?:unknown;}exportinterfaceShutdownMessage{type:'shutdown';}exportinterfaceTaskResultMessage{type:'result';taskId:string;data:unknown;}exporttypeWorkerMessage= HeartbeatMessage | ShutdownMessage | TaskResultMessage;exportinterfaceWorkerHandle{workerId:string;terminate:()=>Promise<void>;state: WorkerState;}

Implementing the Heartbeat Protocol

Designing the IPC Heartbeat Message Schema

The HeartbeatMessage type defines a structured contract: { type: 'heartbeat'; workerId: string; timestamp: number; payload?: unknown }. The type discriminator enables multiplexing multiple message kinds over a single IPC channel. The timestamp field, set by the worker using Date.now(), lets the supervisor calculate drift and staleness without relying solely on its own clock for interval measurement. The optional payload field supports extensibility: workers can attach memory usage snapshots, progress percentages, or diagnostic data without changing the protocol. This structured approach beats raw ping/pong patterns because every message is self-describing, loggable, and parseable by monitoring tools.

importtype{ HeartbeatMessage }from'./types.js';exportconstHEARTBEAT_INTERVAL_MS=2_000;exportconstHEARTBEAT_TIMEOUT_MS=10_000;exportfunctioncreateHeartbeat(workerId:string, payload?:unknown): HeartbeatMessage {return{type:'heartbeat',workerId,timestamp: Date.now(),payload,};}exportfunctionisHeartbeatStale(lastSeenTimestamp:number,timeoutMs:number=HEARTBEAT_TIMEOUT_MS):boolean{return Date.now()- lastSeenTimestamp > timeoutMs;}exportfunctionsendHeartbeatWorkerThread(parentPort:import('node:worker_threads').MessagePort,workerId:string,payload?:unknown):void{parentPort.postMessage(createHeartbeat(workerId, payload));}exportfunctionsendHeartbeatChildProcess(workerId:string,payload?:unknown):void{if(typeof process.send ==='function'){process.send(createHeartbeat(workerId, payload));}}

Supervisor-Side Heartbeat Monitor

The supervisor maintains a Map<string, number> mapping each workerId to its last-seen timestamp. A setInterval sweep runs every HEARTBEAT_INTERVAL_MS and calls isHeartbeatStale() for each entry. When it detects a stale worker, the supervisor immediately terminates the worker in two phases: first a soft shutdown signal (postMessage({ type: 'shutdown' }) for threads, SIGTERM for child processes), then forced termination via worker.terminate() or child.kill('SIGKILL') if the worker fails to exit within the grace period. The staleSince map ensures each stale worker triggers termination only once, preventing redundant attempts.

Code Example #3: HeartbeatMonitor class

import{HEARTBEAT_INTERVAL_MS,HEARTBEAT_TIMEOUT_MS, isHeartbeatStale }from'./heartbeat.js';exporttypeStaleCallback=(workerId:string, staleDurationMs:number)=>void;exportclassHeartbeatMonitor{private lastSeen: Map<string,number>=newMap();private staleSince: Map<string,number>=newMap();private sweepTimer: ReturnType<typeof setInterval>|null=null;privatereadonly timeoutMs:number;privatereadonly intervalMs:number;constructor(timeoutMs:number=HEARTBEAT_TIMEOUT_MS,intervalMs:number=HEARTBEAT_INTERVAL_MS){this.timeoutMs = timeoutMs;this.intervalMs = intervalMs;}register(workerId:string):void{this.lastSeen.set(workerId, Date.now());}receive(workerId:string):void{this.lastSeen.set(workerId, Date.now());this.staleSince.delete(workerId);}unregister(workerId:string):void{this.lastSeen.delete(workerId);this.staleSince.delete(workerId);}startSweep(onStale: StaleCallback):void{if(this.sweepTimer)return;this.sweepTimer =setInterval(()=>{const now = Date.now();for(const[workerId, timestamp]ofthis.lastSeen){if(isHeartbeatStale(timestamp,this.timeoutMs)){if(!this.staleSince.has(workerId)){this.staleSince.set(workerId, now);onStale(workerId, now - timestamp);}}}},this.intervalMs);}stop():void{if(this.sweepTimer){clearInterval(this.sweepTimer);this.sweepTimer =null;}this.lastSeen.clear();this.staleSince.clear();}}

Worker-Side Heartbeat Emitter

Worker threads send heartbeats through parentPort.postMessage(), while child processes use process.send(). Both transports carry identical HeartbeatMessage payloads. Graceful teardown requires clearing the heartbeat interval when the worker receives a 'shutdown' message or a SIGTERM signal. Failing to clear intervals in worker threads can prevent the thread from exiting, as active timers keep the event loop alive.

Process Isolation and Worker Lifecycle Management

Spawning and Managing Worker Threads

The Worker constructor from node:worker_threads accepts an options object where workerData passes the task payload, re transfer of ArrayBuffer instances. Set reout it, a worker thread can grow its heap until the OS kills the entire process. Event handlers for ‘error’, ‘exit’, and ‘messageerror’ provide the supervisor’s observation points

Spawning and Managing Child Processes

fork() spawns a new Node.js process with a built-in IPC channel. Passing execArgv: ['--max-old-space-size=256'] enforces heap limits at the V8 level. Construct a filtered env object that strips secrets so the agent cannot access credentials the agent should not have. Setting stdio to ['pipe', 'pipe', 'pipe', 'ipc'] enables structured logging capture while preserving the IPC channel.

When constructing sanitizedEnv, retain essential OS variables like PATH. Stripping PATH entirely will cause child processes to fail when spawning any subprocesses or using OS utilities. Only strip variables that contain secrets or are explicitly unnecessary.

Code Example #4: WorkerManager class

import{ Worker }from'node:worker_threads';import{ fork,typeChildProcess}from'node:child_process';import{ randomUUID }from'node:crypto';import{ HeartbeatMonitor }from'./heartbeat-monitor.js';importtype{ AgentTask, WorkerHandle, WorkerState, WorkerMessage }from'./types.js';constGRACE_PERIOD_MS=5_000;functionisValidWorkerMessage(msg:unknown): msg is WorkerMessage {if(typeof msg !=='object'|| msg ===null||!('type'in msg))returnfalse;const{ type }= msg as{ type:unknown};return type ==='heartbeat'|| type ==='shutdown'|| type ==='result';}exportclassWorkerManager{private handles: Map<string, WorkerHandle>=newMap();private workers: Map<string, Worker | ChildProcess>=newMap();private exitCallbacks: Map<string,(code:number|null)=>void>=newMap();constructor(private heartbeatMonitor: HeartbeatMonitor){}setExitCallback(workerId:string,cb:(code:number|null)=>void):void{this.exitCallbacks.set(workerId, cb);}spawnWorkerThread(task: AgentTask, scriptPath:string): WorkerHandle {const workerId =randomUUID();const worker =newWorker(scriptPath,{workerData:{ task, workerId },resourceLimits:{maxOldGenerationSizeMb:256,maxYoungGenerationSizeMb:64,},});const state: WorkerState ={workerId, taskId: task.id, status:'running', startedAt: Date.now(),};this.heartbeatMonitor.register(workerId);worker.on('message',(msg:unknown)=>{if(!isValidWorkerMessage(msg))return;if(msg.type ==='heartbeat')this.heartbeatMonitor.receive(workerId);});worker.on('error',(err)=>{console.error(`[worker:${workerId}] error:`, err.message);});worker.on('messageerror',(err)=>{console.error(`[worker:${workerId}] messageerror:`, err.message);});worker.on('exit',(code)=>{state.status ='terminated';this.heartbeatMonitor.unregister(workerId);this.handles.delete(workerId);this.workers.delete(workerId);const cb =this.exitCallbacks.get(workerId);this.exitCallbacks.delete(workerId);cb?.(code);});let terminating =false;const handle: WorkerHandle ={workerId,state,terminate:async()=>{if(terminating)return;terminating =true;state.status ='terminating';worker.postMessage({ type:'shutdown'});awaitnewPromise<void>((resolve)=>{const forceTimer =setTimeout(()=>{worker.terminate().then(resolve).catch(resolve);},GRACE_PERIOD_MS);worker.once('exit',()=>{clearTimeout(forceTimer);resolve();});});},};this.handles.set(workerId, handle);this.workers.set(workerId, worker);return handle;}spawnChildProcess(task: AgentTask, scriptPath:string): WorkerHandle {const workerId =randomUUID();const sanitizedEnv: Record<string,string>={NODE_ENV: process.env.NODE_ENV??'production',PATH: process.env.PATH??'/usr/bin:/bin',};const child =fork(scriptPath,[],{execArgv:['--max-old-space-size=256'],env: sanitizedEnv,stdio:['pipe','pipe','pipe','ipc'],});const state: WorkerState ={workerId, taskId: task.id, status:'running', startedAt: Date.now(), pid: child.pid,};this.heartbeatMonitor.register(workerId);const sent = child.send({ task, workerId });if(!sent){child.kill('SIGKILL');this.heartbeatMonitor.unregister(workerId);thrownewError(`[WorkerManager] IPC send failed for workerId=${workerId}; child killed`);}child.on('message',(msg:unknown)=>{if(!isValidWorkerMessage(msg))return;if(msg.type ==='heartbeat')this.heartbeatMonitor.receive(workerId);});child.on('exit',(code)=>{state.status ='terminated';this.heartbeatMonitor.unregister(workerId);this.handles.delete(workerId);this.workers.delete(workerId);const cb =this.exitCallbacks.get(workerId);this.exitCallbacks.delete(workerId);cb?.(code);});let terminating =false;const handle: WorkerHandle ={workerId,state,terminate:async()=>{if(terminating)return;terminating =true;state.status ='terminating';child.kill('SIGTERM');awaitnewPromise<void>((resolve)=>{const forceTimer =setTimeout(()=>{child.kill('SIGKILL');resolve();},GRACE_PERIOD_MS);child.once('exit',()=>{clearTimeout(forceTimer);resolve();});});},};this.handles.set(workerId, handle);this.workers.set(workerId, child);return handle;}getHandle(workerId:string): WorkerHandle |undefined{returnthis.handles.get(workerId);}getAllHandles(): WorkerHandle[]{returnArray.from(this.handles.values());}}

Timeout-Driven Termination

Every task carries a configurable maxDurationMs. The supervisor sets a setTimeout at spawn time. When it fires, the supervisor first sends a soft shutdown signal: postMessage({ type: ‘shutdown’ }) for threads, SIGTERM for child processes. If the worker does not exit within the grace period, forced termination follows. This two-phase approach gives agents the chance to persist partial results or release external re

On Windows, child.kill('SIGTERM') maps to immediate process termination equivalent to SIGKILL. The grace period for soft shutdown will be bypassed on Windows child processes.

Handling Zombie Processes

If the supervisor itself crashes and restarts, previously spawned child processes may remain running as orphans. Track each child’s PID in a file so the supervisor can clean up orphans on restart: write the PID to a known directory on spawn and delete it on exit. On startup, the supervisor reads any lingering PID files, checks whether those PIDs are still alive via process.kill(pid, 0), and issues SIGKILL to any orphans before beginning normal operation. On Windows, process.kill(pid, 0) throws for dead PIDs rather than returning false; wrap the call in a try/catch for cross-platform compatibility. PID file implementation is left as an exercise; focus on writing the PID immediately after fork() returns and deleting it in the 'exit' handler.

Concurrency Control with Token Buckets

Why Simple Semaphores Aren’t Enough

A basic semaphore caps concurrency but says nothing about burst behavior. When agents call rate-limited external APIs, imagine all maxConcurrency tokens are available and requests arrive simultaneously: every token gets consumed at once, triggering API throttling or account-level blocks. A token bucket adds rate shaping: tokens refill at a fixed rate, and each spawn consumes one token. If the bucket is empty, the spawn request queues until a token becomes available. This provides both a concurrency ceiling and burst control in a single mechanism. The burst shaping is a function of the refill timer: tokens accumulate at a controlled rate independent of the acquire/release lifecycle, which is what differentiates this from a plain semaphore.

Code Example #5: TokenBucket class

exportclassBucketShutdownErrorextendsError{constructor(){super('TokenBucket stopped during acquire');this.name ='BucketShutdownError';}}exportclassTokenBucket{private tokens:number;privatereadonly maxTokens:number;privatereadonly refillRatePerSecond:number;private pendingResolvers:Array<{resolve:()=>void;reject:(e: Error)=>void}>=[];private refillTimer: ReturnType<typeof setInterval>|null=null;constructor(maxTokens:number, refillRatePerSecond:number){if(maxTokens <=0|| refillRatePerSecond <=0){thrownewRangeError(`maxTokens and refillRatePerSecond must be positive numbers, got maxTokens=${maxTokens}, refillRatePerSecond=${refillRatePerSecond}`);}this.maxTokens = maxTokens;this.tokens = maxTokens;this.refillRatePerSecond = refillRatePerSecond;this.startRefill();}privatestartRefill():void{const intervalMs =1000/this.refillRatePerSecond;this.refillTimer =setInterval(()=>{this.drainOrRefill();}, intervalMs);}privatedrainOrRefill():void{if(this.pendingResolvers.length >0){const waiter =this.pendingResolvers.shift()!;waiter.resolve();}elseif(this.tokens <this.maxTokens){this.tokens++;}}asyncacquire():Promise<void>{if(this.tokens >0){this.tokens--;return;}returnnewPromise<void>((resolve, reject)=>{this.pendingResolvers.push({ resolve, reject });});}release():void{const waiter =this.pendingResolvers.shift();if(waiter){waiter.resolve();}else{this.tokens = Math.min(this.tokens +1,this.maxTokens);}}stop():void{if(this.refillTimer){clearInterval(this.refillTimer);this.refillTimer =null;}const err =newBucketShutdownError();for(const{ reject }ofthis.pendingResolvers){reject(err);}this.pendingResolvers =[];}}

Integrating the Token Bucket into the Supervisor

Every call to spawnWorkerThread() or spawnChildProcess() must call await tokenBucket.acquire() before spawning. On worker exit, whether successful or failed, the supervisor calls tokenBucket.release(). Different task priorities can be served by separate bucket instances: a high-priority bucket with more tokens and a faster refill rate, and a low-priority bucket with tighter constraints. The priority sorting in submitTask operates on a single queue; integrating priority-aware routing with separate bucket instances is an extension point beyond the scope of this example.

Putting It All Together: The Supervisor Daemon

The AgentSupervisor Class

The supervisor composes HeartbeatMonitor, WorkerManager, and TokenBucket into a single orchestrator. It exposes four public methods: start() begins the task-processing loop, submitTask(task) enqueues work, shutdown() drains active workers gracefully, and getStatus() returns a snapshot of queue depth and active worker states. An EventEmitter interface surfaces lifecycle events: task:started, task:completed, task:failed, worker:timeout, and worker:restarted.

Code Example #6: Complete agent-worker-supervisor.ts

import{ EventEmitter }from'node:events';import{ HeartbeatMonitor }from'./heartbeat-monitor.js';import{ WorkerManager }from'./worker-manager.js';import{ TokenBucket }from'./token-bucket.js';importtype{ AgentTask, WorkerHandle }from'./types.js';interfaceSupervisorConfig{maxConcurrency:number;refillRatePerSecond:number;heartbeatTimeoutMs:number;heartbeatIntervalMs:number;workerScriptPath:string;childScriptPath:string;}exportclassAgentSupervisorextendsEventEmitter{private queue: AgentTask[]=[];private active: Map<string,{ handle: WorkerHandle; task: AgentTask; timer: ReturnType<typeof setTimeout>}>=newMap();private heartbeatMonitor: HeartbeatMonitor;private workerManager: WorkerManager;private tokenBucket: TokenBucket;private running =false;private draining =false;private processingQueue =false;private config: SupervisorConfig;constructor(config: SupervisorConfig){super();this.config = config;this.heartbeatMonitor =newHeartbeatMonitor(config.heartbeatTimeoutMs, config.heartbeatIntervalMs);this.workerManager =newWorkerManager(this.heartbeatMonitor);this.tokenBucket =newTokenBucket(config.maxConcurrency, config.refillRatePerSecond);}submitTask(task: AgentTask):void{if(this.draining)thrownewError('Supervisor is shutting down');this.queue.push(task);this.queue.sort((a, b)=> b.priority - a.priority);this.processQueue().catch(err =>this.emit('error', err));}start():void{this.running =true;this.heartbeatMonitor.startSweep((workerId, staleDurationMs)=>{if(!this.active.has(workerId))return;this.emit('worker:timeout',{ workerId, staleDurationMs });const entry =this.active.get(workerId);if(entry){entry.handle.terminate().then(()=>{this.emit('task:failed',{ taskId: entry.task.id, reason:'heartbeat_timeout'});this.cleanupWorker(workerId);}).catch(err =>this.emit('error', err));}});this.processQueue().catch(err =>this.emit('error', err));}privateasyncprocessQueue():Promise<void>{if(this.processingQueue)return;this.processingQueue =true;try{while(this.running &&this.queue.length >0){const task =this.queue.shift()!;try{awaitthis.tokenBucket.acquire();}catch{break;}if(!this.running){this.tokenBucket.release();break;}const scriptPath = task.isolationStrategy ==='thread'?this.config.workerScriptPath:this.config.childScriptPath;const handle = task.isolationStrategy ==='thread'?this.workerManager.spawnWorkerThread(task, scriptPath):this.workerManager.spawnChildProcess(task, scriptPath);const timer =setTimeout(()=>{if(!this.active.has(handle.workerId))return;this.emit('worker:timeout',{ workerId: handle.workerId, reason:'max_duration'});handle.terminate().then(()=>{this.emit('task:failed',{ taskId: task.id, reason:'timeout'});this.cleanupWorker(handle.workerId);}).catch(err =>this.emit('error', err));}, task.maxDurationMs);constonExit=(exitCode:number|null)=>{if(exitCode !==0&& exitCode !==null){this.emit('task:failed',{ taskId: task.id, reason:'nonzero_exit', exitCode });}else{this.emit('task:completed',{ taskId: task.id });}this.cleanupWorker(handle.workerId);};this.workerManager.setExitCallback(handle.workerId, onExit);this.active.set(handle.workerId,{ handle, task, timer });this.emit('task:started',{ taskId: task.id, workerId: handle.workerId });}}finally{this.processingQueue =false;}}privatecleanupWorker(workerId:string):void{const entry =this.active.get(workerId);if(!entry)return;clearTimeout(entry.timer);this.active.delete(workerId);this.tokenBucket.release();if(this.running){setImmediate(()=>this.processQueue().catch(err =>this.emit('error', err)));}}getStatus():{ queueDepth:number; activeWorkers:number; workers:string[]}{return{queueDepth:this.queue.length,activeWorkers:this.active.size,workers:Array.from(this.active.keys()),};}asyncshutdown():Promise<void>{this.draining =true;this.running =false;const terminations =Array.from(this.active.values()).map(async({ handle, timer })=>{clearTimeout(timer);await handle.terminate();});awaitPromise.allSettled(terminations);this.heartbeatMonitor.stop();this.tokenBucket.stop();this.active.clear();}}

Example Agent Worker Script

Warning: Never call process.exit() inside a worker thread. Doing so terminates the entire host process, not just the thread. Use parentPort?.close() to signal the thread should exit, and allow the event loop to drain naturally once timers are cleared.

Code Example #7: example-agent.worker.ts

import{ isMainThread, parentPort, workerData }from'node:worker_threads';import{ createHeartbeat,HEARTBEAT_INTERVAL_MS}from'./heartbeat.js';if(isMainThread){thrownewError('This module must be run as a worker thread, not directly.');}const{ task, workerId }= workerData as{ task:{ id:string; payload: Record<string,unknown>}; workerId:string};const heartbeatTimer =setInterval(()=>{parentPort?.postMessage(createHeartbeat(workerId,{ memoryUsage: process.memoryUsage().rss }));},HEARTBEAT_INTERVAL_MS);const taskTimer =setTimeout(()=>{parentPort?.postMessage({type:'result',taskId: task.id,data:{ response:'Agent completed successfully'},});clearInterval(heartbeatTimer);parentPort?.close();},3_000);parentPort?.on('message',(msg:{ type:string})=>{if(msg.type ==='shutdown'){clearInterval(heartbeatTimer);clearTimeout(taskTimer);parentPort?.close();}});

Build and Run

Before running the supervisor, compile the TypeScript

npm run buildnode dist/agent-worker-supervisor.js

In your supervisor configuration, workerScriptPath and childScriptPath must point to the compiled .js files inside dist/, not the .ts

const config: SupervisorConfig ={maxConcurrency:4,refillRatePerSecond:2,heartbeatTimeoutMs:10_000,heartbeatIntervalMs:2_000,workerScriptPath:'./dist/example-agent.worker.js',childScriptPath:'./dist/example-agent.worker.js',};const supervisor =newAgentSupervisor(config);supervisor.start();supervisor.submitTask({id:'task-1',type:'llm-query',priority:1,payload:{ prompt:'Hello, agent'},maxDurationMs:30_000,isolationStrategy:'thread',});

Testing the Supervisor

Instantiate the supervisor with environment-driven configuration: MAX_CONCURRENCY, HEARTBEAT_TIMEOUT_MS, and WORKER_SCRIPT_PATH. To simulate agent failures, create test workers that enter infinite loops (triggering heartbeat timeout and forced termination), allocate arrays until hitting maxOldGenerationSizeMb (triggering V8 OOM within the worker thread), or throw unhandled rejections (triggering the 'error' event). The supervisor logs should show the sequence: heartbeat stale detection, shutdown signal, grace period expiry, forced kill, and token release.

Production Hardening Checklist

Logging and Observability

Attach a correlation ID (the workerId) to every log line emitted by or about a worker. Structured JSON logging, using libraries like pino (install, enables downstream aggregation. Expose Prometheus-style metrics: gauges for active worker count and task queue depth, counters for heartbeat failures and forced terminations

Graceful Shutdown Under SIGINT/SIGTERM

Register handlers for SIGINT and SIGTERM on the supervisor process that call supervisor.shutdown(). The shutdown method prevents new task acceptance by setting a draining flag, then awaits termination of all active workers with a configurable grace period before the process exits.

Security Considerations

rerocesses provide V8-level memory constraints. For production deployments on Linux, OS-level cgroup limits add a second layer of defense; consult your container runtime or systemd documentation for configuration

Never pass raw API keys or database credentials through workerData or child process environment variables; use scoped, short-lived tokens instead.

Validate all incoming IPC messages against the expected type discriminators. A minimal guard before processing any message:

functionisValidWorkerMessage(msg:unknown): msg is WorkerMessage {if(typeof msg !=='object'|| msg ===null||!('type'in msg))returnfalse;const{ type }= msg as{ type:unknown};return type ==='heartbeat'|| type ==='shutdown'|| type ==='result';}

Note that postMessage uses the structured clone algorithm, which does not preserve __proto__ properties. The prototype pollution risk is primarily relevant when IPC messages are deserializedlone. Apply type guards regardless as a defense-in-depth measure

Production Hardening Checklist (shareable):

• Set resourceLimits.maxOldGenerationSizeMb on all worker threads
• Pass --max-old-space-size via execArgv for all child processes
• Strip unnecessary env vars before forking (retain PATH and OS essentials)
• Validate IPC message shapes with type guards before processing
• Implement PID file tracking for orphan detection on supervisor restart
• Attach correlation IDs to all log output per worker
• Expose health and concurrency metrics via a /metrics endpoint
• Register SIGINT/SIGTERM handlers that trigger graceful drain
• Set per-task maxDurationMs with two-phase termination (soft then hard)
• Test with simulated OOM, infinite loops, and unhandled rejections regularly

Key Takeaways

Isolation, liveness monitoring, and concurrency control each address a distinct failure mode. Worker threads and child processes contain crashes and memory leaks. IPC heartbeats with timeout-driven termination catch stuck agents. Token bucket concurrency control prevents burst-spawning from overwhelming external APIs. Together, these form a supervision layer that keeps individual agent failures from compromising system stability.

This pattern fits when the number of agent types and the deployment topology stay within a single Node.js process group. You have outgrown it when you need cross-host coordination, persistent retry across restarts, or cross-service orchestration; at that point, external systems like BullMQ, Temporal, or Kubernetes Jobs are more suitable. The supervisor daemon built here serves as the foundation: extend it with persistent task queues backed by Redis, configurable restart policies per task type, and dead-letter handling for repeatedly failing agents.

Sharing our passion for building incredible internet things.