Nancy Aggarwal

35 practical thoughts on system design Core Principles 1. Every system is a trade-off -> you never get speed, cost, and simplicity all at once. 2. Latency compounds -> every millisecond added across layers adds up to user pain. 3. Scalability ≠ performance -> one handles growth, the other handles speed. 4. Read vs. write paths -> scaling each requires completely different strategies. 5. Design for change, not perfection -> requirements will shift. Databases & Storage 6. Indexes are levers -> high-selectivity columns are worth indexing, low ones often aren’t. 7. Replication helps reads, partitioning helps writes -> don’t confuse the two. 8. Dual writes are a lie -> without coordination, you will see drift. 9. Event stores > queues (sometimes) -> better traceability, worse simplicity. 10. Cache invalidation is still the hardest problem -> freshness vs. performance is the eternal fight. Reliability & Consistency 11. Idempotency saves you -> retries without it will hurt. 12. Fail fast, fail loud -> silent failures sink systems. 13. Eventual consistency is a feature -> not a bug, but only if the business allows it. 14. Conflict resolution in active-active is business logic, not infra. 15. Durability isn’t free -> syncing across regions has cost and latency. Architecture Patterns 16. Microservices are an org structure, not a tech goal. 17. Monolith first, modular second, microservices last -> don’t jump too early. 18. Choreography scales, orchestration simplifies -> pick based on team maturity. 19. Serverless buys you focus, costs you control. 20. Queues don’t remove work, they smooth it. Observability & Operations 21. Tracing > logging -> logs tell you “what”, traces tell you “why”. 22. Metrics rot without ownership -> measure what someone actually uses. 23. Retries without backoff = DDoS on yourself. 24. Dead-letter queues aren’t optional -> every system has poison messages. 25. Levers > knobs -> design quick kill switches and controls to contain blast radius. Performance & Cost 26. Optimize the hot path, not the cold path. 27. Most bottlenecks live in the database, not the code. 28. Horizontal scaling beats vertical scaling ->until coordination kills it. 29. Warm caches mask bad queries -> measure against cold starts too. 30. The cheapest resource is disk, the most expensive is time. People & Process 31. The best architecture dies without documentation. 32. System design reviews aren’t about diagrams, they’re about trade-offs. 33. Small PRs are for speed, big PRs are for context -> balance both. 34. A senior engineer’s role in design is asking the uncomfortable “what if”. 35. No design survives first contact with production -> but good ones bend instead of break. Want to sharpen your instincts with daily practice? Check out joinenginuity.com One system design question every day, with streaks, badges, and leaderboards, and a dashboard to keep you sharp. Enjoy the rabbit hole!

1,457

93 Comments

The worst way to teach/interview/learn system design is to design the entire company (ex: Quora, Airbnb, PayPal, Paytm, Hotstar) 🤦‍♂️ When you pick such a big system, you will create hypothetical Microservices and will not have time to go into engineering depth. A good system design discussion is when a problem statement is focused. You get a chance to discuss the component in-depth and touch upon the engineering aspects like bottlenecks, locking, and transactions. Yahi to asli maza hai! Tackling engineering challenges at scale. This is why every single system I discuss in my course has a limited scope so that I can cover it in-depth from all directions. ps: arpitbhayani.me/course

1,452

14 Comments

If I were your interviewer in a System Design interview, here’s how I’d evaluate your answers — based on these 40 core fundamentals: 1. Scalability – Vertical vs horizontal; default to stateless horizontal scale 2. Latency & Throughput – Know P50/P95/P99 and tradeoffs 3. Capacity Estimation – Estimate QPS, storage, bandwidth 4. Networking Basics – TCP, HTTP/HTTPS, TLS, UDP, DNS 5. SQL vs NoSQL – Decide by access patterns, joins, consistency 6. Data Modeling – Structure to optimize hot queries 7. Indexing – Right indexes help reads, wrong ones hurt writes 8. Normalization – Normalize for writes, denormalize for reads 9. Caching – App/DB/CDN; write-through/back, TTLs 10. Cache Invalidation – TTLs, versioning, stampede protection 11. Load Balancing – L4/L7, RR, least-connections, hashing 12. CDN & Edge – Serve static content near users 13. Sharding – Hash/range/geo; handle hot keys, rebalancing 14. Replication – Sync/async, leader/follower, read replicas 15. Consistency Models – Strong, eventual, causal 16. CAP Theorem – In partition, pick A or C 17. Concurrency – Locks, MVCC, retries 18. Multithreading – Pools, contention, switching 19. Idempotency – Same request, same effect 20. Rate Limiting – Fairness and protection 21. Queues & Streams – Kafka, RabbitMQ; smooth spikes 22. Backpressure – Manage slow consumers/load shedding 23. Delivery Semantics – At-most/least/exactly-once 24. API Design – REST vs gRPC; human vs machine optimized 25. API Versioning – Additive only; never break clients 26. AuthN & AuthZ – OAuth2, JWT, RBAC/ABAC 27. Resilience – Circuit breakers, timeouts, retries 28. Observability – Logs, metrics, traces, SLI/SLO 29. Health Checks – Detect failures, auto-replace 30. Redundancy – Avoid SPOFs; use multi-AZ/region 31. Service Discovery – Dynamic endpoints, central config 32. Deployment – Canary, blue/green, rollbacks 33. Monolith vs Microservices – Split only if ops cost justified 34. Distributed Transactions – Use sagas, avoid 2PC 35. Event Sourcing/CQRS – Append-only logs, split models 36. Consensus – Raft, Paxos, quorum reads/writes 37. Data Privacy – Encryption, GDPR/CCPA 38. Disaster Recovery – Backups, RPO/RTO, test it 39. Serialization – JSON, Avro, Proto 40. Real-Time Delivery – Polling, SSE, WebSockets 📌Comment “Fundamentals” if you want access to this mindmap, I’ll send it you in the comments!!

996

146 Comments

CDC comes up a lot in system design interviews. It solves the very real problem of propagating database changes to other systems without constant polling. But what happens when your interviewer decides to probe into implementation? Do you really understand how it works? Most candidates I see can parrot the basics - "CDC captures database changes in real-time" - but they fall apart when asked how it actually works. Let's see how this is done using Postgres as an example. PostgreSQL already logs every transaction to its Write-Ahead Log for crash recovery. Logical decoding (PostgreSQL's feature that converts raw WAL changes into structured, readable events) reads these WAL entries and converts them into structured change events. Debezium is a popular open-source CDC platform that connects to your database and streams these changes to Kafka. When you set up Debezium, it first takes a consistent snapshot of your database, then immediately starts streaming from the exact WAL position where that snapshot ended. The timing eliminates race conditions and missing data between the snapshot and the stream. Logical replication slots make this reliable. PostgreSQL holds onto WAL segments until your CDC consumer confirms it processed them. Your Debezium connector can crash, get redeployed, or go offline for hours, but when it comes back, it resumes from the last confirmed LSN offset. Debezium transforms these WAL entries into standardized change events and pushes them to Kafka. Each event includes the LSN for ordering guarantees. Your downstream systems - search indexes, caches, analytics - consume these events and stay synchronized without ever polling the primary database. The best part is the performance impact is minimal because you're getting a free ride on PostgreSQL's existing durability guarantees. The WAL exists regardless of whether you're using CDC. Once you understand the WAL + logical decoding combo, CDC stops feeling like black magic.

842

32 Comments

"Your database is getting bombarded due to a hot key, what will you do?", asked the interviewer in a system design interview for designing Linkedin. Interviewee : "First, I will add more servers, then partition the data, and then keep data in different servers (shards)". Interviewer : "Why?" Interviewee : "For high availability." Interviewer : "ok.. that means half of requests will go to DB server 1 and other half to server 2?" Interviewee : "yes.." Interviewer : "that means the hot key which was causing this availability issue will be part of one of the server?" Interviewee : "yes, lets say server 1" Interviewer : "great.. you said this DB is being used by more than one client/use case." Interviewee : "yes, and for that I also mentioned that I am rate limiting based on the use case" Interviewer : "got it. so, after sharding the data, how would you handle rate limiting?" Interviewee : "I will also half the allowed request per second to match the limit per client. Suppose, allowed request/sec for a client is 1000, then after sharding it in 2 shards, it will be 500/server." Interviewer : "that means when you needed to support more than 1000 request/sec for the client due to a hot key, you actually halved the throughput by sharding to fix availability issue?" Interviewee : "hmmm.. didnt think about it this way. But I get it, I have actually reduced the availability instead of increasing it. This will not work." Interviewer : "Exactly.. What will you do in that case?" Interviewee : "Hmm. one solution I can think of is to somehow keep this rate limit at global level, not at per server level. This way the remaining limit from server no 2 can be utilised for server 1. This will solve the availability issue" Interviewer : "Ok.. how will you do it? Which algo will you use?" Interviewee : "Not quite sure.. May be token bucket or leaky bucket" Interviewer : "Lets decide the specific algo later. But how will this work at global level? We use these algos in context of one server" Interviewee : "Frankly speaking, I don't know." Interviewer : "No problem. You tried well. Let me explain. DynamoDB faced this exact situation. Its called throughput dilution. It happens in distributed systems when, while fixing the availability issue due to a hot partition, we end up diluting it, as newly created partitions inherit lower capacity, causing a performance drop. Here is how they fixed it. They used Global Admission Control (GAC). - It centrally tracks table-level throughput. - A request router maintains a local token bucket and periodically requests new tokens from GAC so that table gets its provisioned throughput (e.g., 1000 RCUs). - If a client sends more requests than provisioned, the excess is dropped at the request router level." Interviewee : "Wow.. Engineering is amazing." Interviewer : "Indeed, it is!" If you are preparing for mid-senior SDE, EM roles and need help with trade-offs like this, for System Design HLD round, DM me COACH.

90

17 Comments

"Test hotstar.com with 100 users for 2 minutes.” That’s all you type. Just one sentence. No forms. No dashboards. No YAMLs. No SDKs. You lean back, and Ratl awakens. ✨ On the other side, a network of autonomous agents activates. Not bots. Not scripts. A mission control system made of intelligence. 🧠 PlannerAgent understands your intent—performance validation, not just traffic. It quietly maps the blueprint: target, duration, concurrency, goals. 👷‍♂️ CodeGenAgent generates a ready-to-run performance script: headers, auth, payloads, timing, groups—all handled. Clean, production-grade Java. 📈 TrafficModelAgent picks the right pattern: Ramp-up? Constant? Burst? It reads your tone, maps strategy to simulation. 💰 ResourceCalcAgent calculates infra + cost: Pods, CPUs, memory, duration, estimated ₹/$—down to the decimal. 📄 ApprovalAgent presents it all: → What’s being tested → Infra needed → Cost → Options: APPROVE | MODIFY | CLARIFY | CANCEL You reply: “Reduce to 50 users.” 🧠 HumanFeedbackAgent understands, updates the plan—no toggles, just conversation. With one word—APPROVE—the system comes alive. 🛫 ExecutorAgent launches the test. 📡 MonitorAgent checks in every 30s: “Starting…” → “8,500 requests sent…” → “✅ Complete, 99.8% success” 📬 ReportFetcherAgent delivers: → Total requests → Avg latency → Bottlenecks → Recommendations in plain English 🔁 If something breaks—InterruptHandlerAgent takes over: Retries. Logs. Asks. Moves forward gracefully. It even remembers past tests: “Compare to last week.” “Why was this slower?” It knows. It explains. This isn’t automation. This is autonomy. A system that listens. Understands. Plans. Codes. Simulates. Explains. Evolves. All from one sentence. You’re not running a test. You’re commanding a fleet. And the future of load testing responds. Welcome to ratl.ai. Welcome to agent-native reliability. #Ratl #LoadTesting #AutonomousAgents #AIEngineering #LangGraph #AITesting #DevOps #Gatling #InfraSimplified #PerformanceEngineering #ReliabilityEngineering #HumanInTheLoop #FutureOfTesting #AgenticWorkflows

62

4 Comments

Someone mentioned on X "In compute-heavy tasks, avoid using hash data structures unless you have no choice. They are bad for cache performance, especially in NUMA setups." It looked like a very insightful statement - here are some of my random thoughts on it .. Why hash data structures can be bad for compute heavy tasks - 1. Poor Cache Locality - Hash tables involve random memory access due to their hash based indexing, which leads to cache misses and since they often involve intensive processing on large datasets, cache efficiency is crucial. In contrast, data structures like arrays, sorted vectors, or B-trees offer better spatial locality, making them more cache-friendly. 2. In NUMA systems, each processor has faster access to its local memory node. Hash tables often lead to unpredictable memory accesses across NUMA boundaries, causing expensive remote memory fetches. Sequential data structures like arrays or trees allow better memory affinity and can be optimized to stay within the local memory of a CPU core. 3. Branch Misprediction & Indirect Access - Hash tables involve pointer chasing and non-sequential memory lookups, which cause branch mispredictions and degrade instruction level parallelism (ILP). Tight compute kernels (e.g., in image processing or matrix math) rely on predictable loops and minimal indirection to fully utilize SIMD pipelines and modern CPU execution units. However when a problem involves frequent key-based accesses rather than sequential scans, hash tables remain the best option. And there are some alternatives like Flat Hash Maps (e.g., Google's absl::flat_hash_map, Rust's hashbrown using Robin Hood hashing) that can reduce pointer indirection and improve cache locality. In summary: - If your workload is truly compute-bound (e.g., SIMD-based, matrix multiplications, large-scale simulations), avoid hash tables when possible. - If your workload is lookup-heavy, hash tables remain useful but should be optimized for cache locality (e.g., Robin Hood hashing, flat maps).

42

3 Comments

Lambda now charges you for init time, so it's useful to count sandboxes which are proactively initialized but never receive a request. Here's what happens after a 10k request burst. Hundreds of sandbox shutdowns, along with 22 sandboxes which were spun up but never received a request. The faster your cold starts are, the cheaper these will be!

40

9 Comments