Engineering Unit Economics: Sizing, Build vs. Buy, and Technical Debt ROI

Infrastructure cost governance is an architectural discipline, not a procurement exercise. When cloud spend scales non-linearly with revenue, the root cause is rarely an unoptimized discount tier. The actual drivers are structural: unconstrained compute allocations, misaligned memory working sets, unaccounted SaaS operational overhead, and unquantified technical debt.

Pragmatic infrastructure engineering demands quantifiable financial models. Compute, memory, IOPS, network egress, and engineering hours must be managed as interchangeable capital inputs with measurable yields.

1. Cloud Unit Economics and Workload Cost Attribution

Aggregate cloud bills obscure systemic inefficiencies by grouping compute, storage, and networking into high-level line items. Effective cost governance requires workload-level attribution that ties infrastructure consumption directly to business transactions.

Define the base unit cost $C_u$ for a distributed service as:

$$C_u = \frac{C_{\text{compute}} + C_{\text{storage}} + C_{\text{network}} + C_{\text{third\_party}}}{N_{\text{units}}}$$

Where $N_{\text{units}}$ represents the core domain metric—such as settled transactions, active tenants, or generated reports.

// Granular cost attribution middleware for distributed microservices
export interface WorkloadMetrics {
  tenantId: string;
  route: string;
  durationMs: number;
  dbReadIops: number;
  egressBytes: number;
}

export function computeTransactionCost(metrics: WorkloadMetrics): number {
  const VCPU_HOUR_COST = 0.04048; // Base rate for c6g.large (2 vCPU) normalized to single vCPU hour
  const IOPS_RATE = 0.0000002;    // Managed relational database I/O unit price
  const EGRESS_GB_RATE = 0.09;     // Standard inter-region internet egress

  const computeCost = (metrics.durationMs / 3600000) * VCPU_HOUR_COST;
  const storageIoCost = metrics.dbReadIops * IOPS_RATE;
  const networkCost = (metrics.egressBytes / (1024 ** 3)) * EGRESS_GB_RATE;

  return computeCost + storageIoCost + networkCost;
}

Capturing metrics at the application boundary allows teams to track marginal costs per route in continuous integration, flagging algorithmic regressions before they reach production.

2. Compute and Query Sizing: Eliminating Systematic Waste

Most cloud footprints operate with $40\%$ to $60\%$ overprovisioned capacity. Teams frequently size infrastructure for unmitigated traffic spikes instead of decoupling resource tiers and tuning query access patterns.

Compute Sizing: Decoupling Resource Requests and Limits

In container environments, setting resource requests equal to resource limits creates cluster node fragmentation and forces schedulers to over-provision underlying VM instances:

• CPU Requests: Set to P95 steady-state utilization. CPU is compressible; burst throttling introduces bounded latency increases but avoids pod evictions.
• Memory Requests and Limits: Pin both to P99 peak usage. Memory is non-compressible; under-provisioned memory invokes the Linux kernel Out-Of-Memory (OOMKilled) killer, causing hard process termination.

{
  "apiVersion": "apps/v1",
  "kind": "Deployment",
  "metadata": { "name": "order-pipeline" },
  "spec": {
    "replicas": 12,
    "selector": {
      "matchLabels": { "app": "order-pipeline" }
    },
    "template": {
      "metadata": {
        "labels": { "app": "order-pipeline" }
      },
      "spec": {
        "containers": [
          {
            "name": "worker",
            "image": "order-pipeline:v3.4.1",
            "resources": {
              "requests": {
                "cpu": "250m",
                "memory": "512Mi"
              },
              "limits": {
                "cpu": "1200m",
                "memory": "512Mi"
              }
            }
          }
        ]
      }
    }
  }
}

Query Sizing and Database Working Sets

Database infrastructure costs are dominated by buffer pool eviction rates and persistent storage IOPS. Unindexed sequential scans load cold data pages into memory, displacing frequently accessed working sets and forcing physical disk reads.

Query execution time and operational cost follow this memory distribution:

$$T_{\text{query}} = N_{\text{pages}} \cdot \left( P_{\text{hit}} \cdot T_{\text{cache}} + (1 - P_{\text{hit}}) \cdot T_{\text{disk}} \right)$$

Where $P_{\text{hit}}$ is the buffer cache hit ratio. Inefficient queries drop $P_{\text{hit}}$, driving up required IOPS provisioning and memory footprints.

-- Inefficient: Sequential table scan forces disk reads and displaces hot cache pages
SELECT transaction_id, user_id, amount, status 
FROM ledger_entries 
WHERE account_id = 'acc_98312' AND status = 'settled' 
ORDER BY created_at DESC 
LIMIT 20;

-- Optimized: Composite index enables index-only scan, eliminating heap lookups
CREATE INDEX CONCURRENTLY idx_ledger_acc_status_created 
ON ledger_entries (account_id, status, created_at DESC) 
INCLUDE (transaction_id, user_id, amount);

Covering indexes eliminate heap fetches, lowering query IOPS by over $95\%$ and permitting instance downsizing.

3. Concrete Build vs. Buy Decision Framework

Engineering teams often bias toward building custom systems to maintain control, while finance teams lean toward SaaS to cap payroll. Both approaches introduce severe financial leakage when evaluated without full lifecycle accounting.

Model build versus buy decisions across a 36-month horizon ($T = 36$) using Total Cost of Ownership ($TCO$):

$$TCO_{\text{build}} = C_{\text{dev\_initial}} + \sum_{t=1}^{T} \left( C_{\text{infra}}(t) + C_{\text{maintenance}}(t) + C_{\text{opportunity}}(t) \right)$$

$$TCO_{\text{buy}} = C_{\text{integration}} + \sum_{t=1}^{T} \left( C_{\text{license}}(t) + C_{\text{vendor\_ops}}(t) + C_{\text{lockin\_risk}}(t) \right)$$

Key variables:

• $C_{\text{maintenance}}(t)$: Baseline engineering upkeep, typically $20\%$ to $25\%$ of initial build cost annually.
• $C_{\text{opportunity}}(t)$: Net present value of deferred core product features.
• $C_{\text{vendor\_ops}}(t)$: Engineering overhead for API upgrades, authentication maintenance, and vendor compliance.
• $C_{\text{lockin\_risk}}(t)$: Expected value of contract renewal price hikes and egress fees.

Strategic Sourcing Matrix

+--------------------------------+--------------------------------+
| High Strategic Differentiation | High Strategic Differentiation |
| Low Operational Complexity      | High Operational Complexity     |
| -> BUILD IN-HOUSE              | -> HYBRID / PARTNER PLATFORM   |
+--------------------------------+--------------------------------+
| Low Strategic Differentiation    | Low Strategic Differentiation    |
| Low Operational Complexity      | High Operational Complexity     |
| -> BUY COMMODITY SAAS           | -> BUY MANAGED ENTERPRISE SAAS |
+--------------------------------+--------------------------------+

Repatriating SaaS workloads in-house is financially viable only when the annual vendor markup exceeds fully loaded team operational costs:

$$C_{\text{license}} > C_{\text{infra\_bare}} + \frac{N_{\text{engineers}} \cdot \text{Salary}_{\text{loaded}}}{K_{\text{efficiency\_factor}}}$$

4. Calculating Hard ROI: Technical Debt vs. Feature Development

Refactoring proposals fail executive review when framed around code aesthetics. Technical debt must be presented with standard capital allocation metrics: infrastructure recovery, conversion impact, and reclaimed developer bandwidth.

Evaluate technical debt remediation using this ROI model:

$$\text{ROI}_{\text{debt}} = \frac{\Delta C_{\text{infra}} + \Delta R_{\text{perf}} + \Delta V_{\text{eng}} - C_{\text{remediation}}}{C_{\text{remediation}}}$$

Where:

• $\Delta C_{\text{infra}}$: Annualized cloud infrastructure reduction.
• $\Delta R_{\text{perf}}$: Revenue protected or generated via latency improvements: $$\Delta R_{\text{perf}} = R_{\text{baseline}} \cdot (\Delta L \cdot k_{\text{conversion}})$$
• $\Delta V_{\text{eng}}$: Engineering capacity reclaimed from reduced incident triage: $$\Delta V_{\text{eng}} = H_{\text{incident\_saved}} \cdot R_{\text{hourly}} + \Delta S_{\text{velocity}} \cdot V_{\text{story\_value}}$$
• $C_{\text{remediation}}$: Total engineering implementation cost ($N_{\text{engineers}} \times \text{Duration} \times \text{Loaded Weekly Cost}$).

Case Study: High-Throughput Ledger Ingestion Service

A core ingestion service suffered from connection pool starvation, memory leaks, and frequent production outages.

• Remediation Cost: 2 senior engineers for 5 weeks = $$50,000$.
• Infrastructure Savings: Downscaling from 16 memory-optimized instances to 4 compute-optimized instances = $$2,200/\text{month} = $26,400/\text{year}$.
• Incident Mitigation: Saving 20 triage hours per month at $$150/\text{hour} = $36,000/\text{year}$.
• Performance Impact: Reducing P99 latency by 180ms improved checkout conversion, yielding $$45,000/\text{year}$.

$$\text{ROI}_{\text{Year 1}} = \frac{26,400 + 36,000 + 45,000 - 50,000}{50,000} = \frac{57,400}{50,000} = 114.8\%$$

A verified $114.8\%$ first-year return establishes debt remediation as a high-yield investment rather than an operational tax.

5. Architectural Trade-Offs and Strategic Execution

Infrastructure optimization involves explicit architectural trade-offs:

1. Caching vs. Read Replicas: Adding a Redis caching tier can cut database replica costs by up to $60\%$, but it adds state invalidation logic and cache-aside failure modes.
2. Single-AZ vs. Multi-AZ Egress: Concentrating services in one Availability Zone eliminates cross-AZ network costs ($$0.01/\text{GB}$), but requires verified fast-failover topology to maintain disaster recovery SLAs.
3. Storage Lifecycle Policies: Moving unindexed object storage to cold or archival tiers reduces storage fees by up to $90\%$, but incurs retrieval delays and lifecycle transition API costs.

Execution Checklist

• Instrument per-transaction cost attribution on high-volume routes to isolate unit economics.
• Size Kubernetes CPU requests to P95 steady-state while pinning memory requests directly to P99 limits.
• Implement covering indexes to preserve database buffer pool cache hit ratios before vertically scaling instances.
• Apply 36-month TCO models to assess maintenance overhead prior to initiating in-house builds.
• Frame refactoring roadmaps around hard economic returns across infrastructure footprint, latency yield, and engineering velocity.