Stop Running Sync Jobs Blind: A Practical Idempotent Workflow for Agent Teams

When multiple agents or processes run sync jobs, one hidden problem appears quickly: the same job runs twice with slightly different side effects.

That is how teams get inconsistent indexes, half-written files, and "works on my machine" bugs.

Concurrency without locks introduces inconsistent state, duplicate side effects, and non-reproducible failures.

Step 1: Assign a run identity

Every sync run should have a unique run ID and explicit start time.

function newRunId(): string {
  return 'sync-' + Date.now() + '-' + Math.random().toString(16).slice(2, 8);
}

This gives you traceability in logs and status files.

Step 2: Enforce a lock before mutating data

No lock means no guarantees.

#!/usr/bin/env bash
set -euo pipefail

LOCK_FILE="/tmp/sync.lock"
exec 9>"$LOCK_FILE"
flock -n 9 || { echo "Another sync is running"; exit 1; }

echo "Lock acquired"

A lock is a simple guardrail, but it prevents a lot of chaos.

Step 3: Make each step idempotent

If a step runs twice, the second run should produce the same end state.

• upsert records instead of append-only duplicates
• write to temp files then atomic rename
• skip unchanged inputs by checksum

Step 4: Write machine-readable status after each run

import { writeFileSync } from 'node:fs';

type SyncStatus = {
  runId: string;
  ok: boolean;
  startedAt: string;
  finishedAt: string;
  itemsProcessed: number;
  error?: string;
};

function saveStatus(status: SyncStatus) {
  writeFileSync('data/sync-status.json', JSON.stringify(status, null, 2));
}

This gives operators and agents one source of truth.

Step 5: Recover predictably after failures

If step 3 fails, rerun should:

1. detect partial output
2. clean invalid temp state
3. continue safely

Do not rely on manual cleanup as your default recovery strategy.