Overviews

System Design Foundations

System design turns product and operational requirements into a coherent technical system. It combines architecture, distributed systems, data, performance, reliability, security, capacity, and cost into explicit tradeoffs.

Explore the System Design layer Jump to capacity planning Review core metrics

A repeatable approach

From ambiguous problem to defensible design

Good system design is not a technology quiz. It is a sequence of decisions grounded in workload, constraints, failure modes, and evidence.

01Clarify the problem
Define users, use cases, boundaries, and what success means.
02Quantify the workload
Estimate traffic, concurrency, data volume, growth, and peak behavior.
03Set quality targets
Agree on latency, availability, consistency, durability, security, and cost.
04Sketch the architecture
Define components, data flows, ownership, APIs, and trust boundaries.
05Deep-dive critical paths
Analyze storage, caching, queues, contention, bottlenecks, and failure handling.
06Validate tradeoffs
Test assumptions against capacity, operations, migration, and business constraints.

Core pillars

Every design should address the whole system

Requirements and constraints

Start with users, workloads, latency targets, availability, consistency, security, compliance, cost, and delivery constraints.

Functional Requirement NFR Acceptance Criteria

Architecture and boundaries

Choose responsibilities, service boundaries, communication patterns, data ownership, and deployment topology before selecting individual technologies.

Software Architecture C4 Model API

Data and state

Model entities, access patterns, transactions, retention, replication, partitioning, and failure recovery around the workload.

Data Model Relational Database Transaction

Scale and performance

Estimate demand, identify bottlenecks, budget latency, and choose scaling mechanisms with explicit safety margins.

Scalability Latency Bottleneck

Reliability and operations

Design for failure, observability, recovery, deployment, incident response, and ongoing capacity management.

Reliability System Observability Runbook

Security and risk

Apply threat modeling, least privilege, data protection, abuse prevention, and operational controls throughout the design.

Threat Modeling Authentication Authorization

Capacity planning

Translate expected demand into safe operating capacity

Capacity planning estimates the resources needed to serve normal demand, peaks, growth, and failures without unacceptable latency, errors, or cost. It is a living model, not a one-time spreadsheet.

1. Define workload units
Requests, jobs, messages, active users, stored objects, bytes transferred, or another measurable unit.
2. Establish baseline and peak
Separate average load from peak load, bursts, seasonal patterns, and failure-mode traffic.
3. Forecast growth
Use historical trends, product plans, adoption scenarios, and explicit confidence ranges.
4. Model resource demand
Translate workload into CPU, memory, storage, network, connection, and dependency demand.
5. Add safety margins
Reserve headroom for variance, deployments, failover, maintenance, and imperfect forecasts.
6. Define scaling triggers
Specify when to scale, what can scale independently, and the operational limits of automation.
7. Validate continuously
Use load tests, production telemetry, capacity reviews, and post-incident evidence to update the model.

General metrics

Measure demand, service quality, saturation, and cost together

No single metric describes a system. A useful design combines user-facing indicators with resource and dependency signals.

Traffic and demand

Requests per second (RPS/QPS)
Concurrent users or sessions
Jobs or messages per second
Read/write ratio
Payload and object size
Daily and monthly active users

Latency and responsiveness

p50, p95, p99 latency
Time to first byte
Queue wait time
End-to-end response time
Timeout rate
Cold-start latency

Throughput and saturation

Completed operations per second
CPU utilization
Memory working set
Disk IOPS and throughput
Network bandwidth
Connection-pool utilization
Queue depth

Reliability and quality

Availability
Error rate
Success rate
Retry rate
Dropped or rejected work
Recovery time objective (RTO)
Recovery point objective (RPO)

Data and storage

Data growth rate
Storage utilization
Cache hit ratio
Replication lag
Transaction rate
Index size and selectivity
Retention volume

Economics and efficiency

Cost per request or transaction
Compute cost per active user
Storage cost per retained unit
Resource utilization efficiency
Reserved versus on-demand capacity
Cost of redundancy and failover

Design review checklist

Questions a complete design should answer

What are the primary use cases and excluded scope?
What are normal, peak, and failure-mode workloads?
Which quality attributes are explicit and measurable?
Where does state live, and who owns it?
What are the consistency and transaction boundaries?

What fails, how is failure detected, and how does recovery work?
Where are the bottlenecks and scaling limits?
How is the system observed and operated?
What security, privacy, and compliance controls apply?
What are the migration, cost, and organizational tradeoffs?