#architecture#simplicity#boring-tech#monolith#refactoring

In Praise of Boring Tech: De-Microservicing, Postgres, and the Art of Safely Deleting Code

"Over the past decade, the software engineering industry succumbed to an expensive collective delusion: that every early-stage product and mid-market enterprise"

•By huud•
0 views
~7 min read
In Praise of Boring Tech: De-Microservicing, Postgres, and the Art of Safely Deleting Code

Over the past decade, the software engineering industry succumbed to an expensive collective delusion: that every early-stage product and mid-market enterprise required the distributed infrastructure of Netflix or Google. Teams reflexively partitioned cohesive business logic across dozens of microservices, wiring them together with Kafka, Kubernetes clusters, and polyglot persistence tiers.

The result was rarely agility. More often, it produced an explosion of accidental complexity: distributed transactions resolved through fragile eventual consistency, labyrinthine cross-service distributed tracing, cascading RPC failures, and cloud infrastructure bills scaling faster than top-line revenue.

Engineering maturity is knowing when not to distribute. By leaning on battle-tested, "boring" technologies—principally PostgreSQL, SQLite, and modular monoliths—engineering organizations reclaim development velocity, lower operational expenditure, and build systems that fail predictably and recover instantly.


The Distributed Systems Tax

Distributed systems do not eliminate complexity; they swap compile-time safety and local function calls for runtime network failures.

Consider availability under synchronous coupling. If a critical transaction synchronously depends on nn independent services, each with an availability of AiA_i, composite system availability AsystemA_{\text{system}} is bounded by their product:

Asystem=∏i=1nAiA_{\text{system}} = \prod_{i=1}^{n} A_i

If a checkout workflow crosses 10 services, each maintaining three nines (A=0.999A = 0.999), your effective baseline drops substantially:

Asystem=0.99910≈0.990  ⟹  ≈1% downtimeA_{\text{system}} = 0.999^{10} \approx 0.990 \implies \approx 1\% \text{ downtime}

One percent downtime translates to roughly 3.65 days of system outage per year. Mitigating this through circuit breakers, retry budgets with exponential backoff, and fallback queues introduces immense operational surface area.

Latency compounds identically. Total end-to-end latency LtotalL_{\text{total}} transitions from microsecond-level in-memory calls to the cumulative overhead of network hops, serialization, and transport connection pools:

Ltotal=∑i=1k(τnet(i)+τserdes(i)+τexec(i))L_{\text{total}} = \sum_{i=1}^{k} \left( \tau_{\text{net}}^{(i)} + \tau_{\text{serdes}}^{(i)} + \tau_{\text{exec}}^{(i)} \right)

Across internal VPC networks and cloud availability zones, p99p99 tail latencies compound geometrically. A single degraded downstream dependency drags the entire user transaction down with it.


The Operational Core: PostgreSQL and SQLite

Before introducing a dedicated search engine, cache tier, or distributed queue, exhaust the primitives built into PostgreSQL. Modern Postgres is a convergent operational data engine:

  1. Document Storage: JSONB combined with Generalized Inverted Indexes (GIN) satisfies semi-structured document access patterns, eliminating the operational overhead of a separate document database.
  2. Transactional Queues: SELECT ... FOR UPDATE SKIP LOCKED allows standard relational tables to operate as reliable, ACID-compliant work queues. Workers consume jobs concurrently without lock contention or third-party message brokers.
  3. Search & Embeddings: Built-in extensions like pg_trgm handle fuzzy text search, while pgvector delivers production-grade vector similarity search alongside transactional business records.

Pair Postgres with a connection pooler like PgBouncer or pgcat, and a single vertical instance on modern NVMe storage easily sustains 20,000 to 50,000 write transactions per second—orders of magnitude beyond the peak demand of most companies.

When embedded, low-latency persistence is required, SQLite running in Write-Ahead Logging (WAL) mode provides concurrent reads, microsecond queries, and deterministic local state. Replicated using tools like Litestream, SQLite is more than capable of powering production-grade edge services with zero network overhead.


De-Microservicing: The In-Process Consolidation Blueprint

Consolidating microservices into a modular monolith does not mean building an unprincipled "big ball of mud." It means establishing physical code boundaries enforced by the compiler rather than serialized HTTP or gRPC payloads.

sql
Distributed Anti-Pattern:
[Gateway] ---> (HTTP) ---> [Order Service] ---> (gRPC) ---> [Inventory Service]
                                |                                 |
                            [DB: Orders]                    [DB: Inventory]

Modular Monolith:
+------------------------------------------------------------------------+
|                          Host Process (Node / Go)                      |
|  +---------------------------+          +---------------------------+  |
|  |     Order Module          |          |     Inventory Module      |  |
|  |   - In-memory Domain API  | <------> |   - In-memory Domain API  |  |
|  +-------------+-------------+          +-------------+-------------+  |
|                |                                      |                |
|                +------------------+-------------------+                |
|                                   |                                    |
|                      [PostgreSQL (Unified ACID Engine)]                |
+------------------------------------------------------------------------+

Enforcing Boundaries Without Network Hops

In a modular monolith, modules communicate through explicit domain interfaces. Direct table access across domain boundaries is forbidden. Language-level visibility controls—such as Go internal packages or TypeScript project references—guard these boundaries.

Crucially, in-process execution permits wrapping multi-module updates in a single database transaction, eradicating the need for distributed sagas or two-phase commits:

typescript
// packages/modules/orders/src/order.service.ts
import { InventoryAPI } from '@app/modules/inventory';
import { DatabaseClient, Transaction } from '@app/platform/database';

export class OrderService {
  constructor(
    private readonly inventory: InventoryAPI,
    private readonly db: DatabaseClient
  ) {}

  async checkout(orderId: string, items: Array<{ sku: string; qty: number }>): Promise<void> {
    await this.db.transaction(async (trx: Transaction) => {
      // In-process invocation sharing the active transactional context
      const reservation = await this.inventory.reserveStock(trx, { orderId, items });
      if (!reservation.success) {
        throw new Error(`Inventory reservation failed: ${reservation.reason}`);
      }

      // Atomically persisted in the same physical commit
      await trx.table('orders').insert({
        id: orderId,
        status: 'CONFIRMED',
        items: JSON.stringify(items),
        created_at: new Date(),
      });
    });
  }
}

By keeping calls in-memory, you retain compile-time type safety and the ability to execute transactional rollbacks within a single database connection.


Safe Deprecation and Code Deletion Without Downtime

Deleting code safely is an operational discipline. Ripping out legacy endpoints or schemas prematurely causes null-pointer exceptions and deployment rollbacks.

1. The Expand/Contract Migration Pattern

Never rename, drop, or alter database columns and external APIs in a single commit. Decouple storage changes from application logic using the four-phase expand/contract lifecycle:

sql
Phase 1 (Expand):    Deploy schema change. Add new column/field as nullable or with defaults.
Phase 2 (Dual-Write): Application writes to both Old and New fields; reads from Old.
                     Backfill existing historical records from Old to New.
Phase 3 (Contract):  Switch application reads to New field. Cease writing to Old field.
Phase 4 (Drop):      Drop the Old column, remove legacy fields from data models.

2. Async Shadow Execution

Before swapping out a legacy component for a rewritten engine, validate behavioral equivalence under real-world production load using async shadow traffic:

typescript
// packages/shared/src/experimentation/shadow-harness.ts
export async function executeWithShadow<T>(
  canonicalFn: () => Promise<T>,
  shadowFn: () => Promise<T>,
  comparator: (a: T, b: T) => boolean,
  metricKey: string
): Promise<T> {
  const primaryResult = await canonicalFn();

  // Non-blocking execution guarantees shadow failures cannot impact client paths
  setImmediate(async () => {
    try {
      const shadowResult = await shadowFn();
      const matches = comparator(primaryResult, shadowResult);

      if (!matches) {
        console.warn(`[Shadow Drift] Key: ${metricKey}`);
      }
    } catch (err) {
      console.error(`[Shadow Failure] Key: ${metricKey}`, err);
    }
  });

  return primaryResult;
}

3. Metric-Driven Dead Code Verification

Never guess if an old code path is active. Insert explicit tombstones into execution paths marked for deletion:

typescript
// packages/shared/src/monitoring/tombstone.ts
export function tombstone(featureName: string): void {
  // Emit a zero-cost counter metric to Prometheus or StatsD
  metrics.increment('code.tombstone.triggered', { feature: featureName });

  if (process.env.NODE_ENV === 'development') {
    throw new Error(`Tombstone reached: ${featureName} is marked for deprecation.`);
  }
}

If the tombstone counter records zero hits across a full business reporting cycle (typically 30 to 90 days), the legacy pathway can be excised in an isolated, deletion-only pull request.


The Senior Engineer's Heuristic

Architectural simplicity is an aggressive, proactive practice. It means resisting the temptation to implement novel, bleeding-edge frameworks in favor of patterns battle-tested across decades of industry operation.

bash
# The most valuable command in systems engineering:
git rm -rf ./services/unnecessary-microservice/

Before introducing another message broker, provisioning a distributed key-value cluster, or splitting code across another network boundary, ask: Can PostgreSQL do this? Can an in-process interface solve this? Can this run reliably on a single, well-provisioned server?

Most of the time, the boring answer is the correct one. Your systems will remain comprehensible, your operational overhead will drop, and your team will focus engineering effort where it creates genuine leverage: shipping resilient features to production.

About the Author

huud

huud

@huud

About →

Systems architect and software engineer building high-performance distributed platforms.