Skip to main content

Part of topic: Next.js Architecture, Zero-Trust & Auditable Infrastructure, Full-Stack Systems Design

NestFi: Financial Correctness Under Failure with Distributed Locks and Double-Entry Ledgers

NestFi: Financial Correctness Under Failure with Distributed Locks and Double-Entry Ledgers

March 22, 2025 3 min read NestJS, TypeScript, PostgreSQL, Redis, M-Pesa, Fintech, Distributed Systems

NestFi: Financial Correctness Under Failure with Distributed Locks and Double-Entry Ledgers is a Developer Journal article by Ancel Ajanga on https://ancel.co.ke. A production-grade finance platform designed for financial correctness under failure: fail-closed Redis distributed locks, double-entry ledger invariants, and resilient M-Pesa B2C callback processing. It focuses on: In financial applications, network blips, replayed payment webhooks, and concurrent withdrawal requests can easily cause double-spending or…

A production-grade finance platform designed for financial correctness under failure: fail-closed Redis distributed locks, double-entry ledger invariants, and resilient M-Pesa B2C callback processing.

Who this is for

Fintech engineers, backend developers, and systems programmers who need to guarantee financial invariants and prevent double-spending across asynchronous payment providers.

Problem

In financial applications, network blips, replayed payment webhooks, and concurrent withdrawal requests can easily cause double-spending or corrupted account balances if mutations are not strictly synchronized.

Business outcome

A robust financial transaction coordinator ensuring debits equal credits at all times, with fail-closed distributed locking on user wallets and idempotent callback deduplication.

Metrics

  • Strict zero-sum double-entry ledger balance guarantee
  • Fail-closed distributed locking on wallet mutations via Redlock algorithm
  • Idempotency key enforcement on all outward payment dispatches

Financial applications are not standard CRUD systems. In a social app, an occasional lost post or duplicate notification is annoying; in a financial platform, a race condition that permits an account to withdraw $500 twice before the balance updates is a catastrophic failure.

I built NestFi to tackle the hard problems of distributed financial engineering: **How do you keep ledger accuracy, auditability, and user trust when payment providers are asynchronous, callbacks are delayed or replayed, caches fail, and infrastructure experiences partial outages?**

The Problem: The Illusion of Atomic Balance Updates

In naive implementations, a withdrawal endpoint looks like this: 1. `SELECT balance FROM wallets WHERE user_id = :id` 2. Check if `balance >= withdrawal_amount`. 3. Dispatch payout request to M-Pesa B2C or bank API. 4. `UPDATE wallets SET balance = balance - withdrawal_amount`.

This code contains a fatal **Time-of-Check to Time-of-Use (TOCTOU) race condition**. If a user issues two concurrent withdrawal requests 5 milliseconds apart, both queries read the original balance before either write occurs. Both requests succeed, draining double the authorized funds.

Even worse: what if step 3 succeeds (the money leaves via M-Pesa), but step 4 fails because the database crashes? The money is gone, but the wallet balance remains untouched.

Architectural Approach: Double-Entry Ledgers and Fail-Closed Locks

NestFi solves these failure modes with three foundational patterns:

### 1. Fail-Closed Redis Distributed Locks Before any wallet mutation occurs, the service must acquire an explicit distributed lock on the wallet resource ID using **Redlock algorithms**: - The lock carries a tight TTL (5,000ms) to prevent permanent deadlocks if a server process terminates unexpectedly. - If Redis is unreachable or the lock cannot be acquired within 500ms, the transaction fails immediately (**Fail-Closed**). In financial systems, denying an operation is infinitely safer than proceeding without concurrency guarantees.

ypescript
async function executeWalletMutation<T>(
  walletId: string,
  operation: (tx: PrismaClient) => Promise<T>
): Promise<T> {
  const lockKey = `lock:wallet:${walletId}`;
  const lock = await redlock.acquire([lockKey], 5000);

  try {
    return await prisma.$transaction(async (tx) => {
      return await operation(tx);
    }, { isolationLevel: Prisma.TransactionIsolationLevel.Serializable });
  } finally {
    await lock.release();
  }
}

### 2. Double-Entry Accounting Invariants NestFi enforces double-entry ledgering: - Money never exists as a raw scalar column. - Every financial movement creates matching debit and credit ledger rows. - A database check constraint enforces that total debits must equal total credits across every journal transaction: `SUM(debits) - SUM(credits) = 0`. - Wallet balances are computed snapshots backed by immutable, append-only ledger transaction histories.

### 3. Outward M-Pesa B2C Payout Idempotency When initiating outward payouts (Business-to-Customer) via Safaricom's Daraja B2C gateway: - Every payout request generates a UUID v4 **Idempotency Key** stored alongside the transaction record in a `PENDING_PAYOUT` status. - If network timeouts occur, the retry worker uses the identical idempotency key. - When Safaricom's final callback arrives, the transaction transitions to `COMPLETED` or rolls back cleanly to `FAILED`, restoring the user's funds without manual intervention.

Real-Time UX with Socket.io

Users waiting for financial transfers experience acute anxiety. NestFi bridges the backend state machine to the frontend via **Socket.IO**: - When a payout transitions from `PENDING` to `CONFIRMED`, a targeted event emits to the user's private session room. - The balance updates in real-time on the client with a celebratory confirmation sound, eliminating the need for frantic browser refreshes.

Key Engineering Takeaways

1. **Fail-Closed is the Only Option for Money**: When components fail or locks cannot be acquired, always deny the transaction. Availability is secondary to consistency in financial architecture. 2. **Never Trust Scalar Balances**: Always anchor financial balances to immutable double-entry ledger rows. If questions arise during an audit, you can reconstruct every cent from day one. 3. **Isolate Provider Timeouts with Strict State Machines**: External payment gateways *will* timeout and *will* replay callbacks. Designing state machines with explicit pending, confirmed, and compensated failure states prevents lost funds.

Explore the NestFi Case Study, visit the [NestFi Project Page](/projects/nestfi), or [get in touch](/contact) to discuss resilient backend architecture.

Trade-offs

Adopted a fail-closed architecture where locks default to denying transactions upon Redis connectivity loss, prioritizing data correctness over temporary availability.

Challenges faced

Coordinating state across PostgreSQL relational transactions, Redis lock managers, and external M-Pesa B2C payout callbacks without leaving wallets in an inconsistent pending state.