Arpit Bhayani

35 practical thoughts on system design Core Principles 1. Every system is a trade-off -> you never get speed, cost, and simplicity all at once. 2. Latency compounds -> every millisecond added across layers adds up to user pain. 3. Scalability ≠ performance -> one handles growth, the other handles speed. 4. Read vs. write paths -> scaling each requires completely different strategies. 5. Design for change, not perfection -> requirements will shift. Databases & Storage 6. Indexes are levers -> high-selectivity columns are worth indexing, low ones often aren’t. 7. Replication helps reads, partitioning helps writes -> don’t confuse the two. 8. Dual writes are a lie -> without coordination, you will see drift. 9. Event stores > queues (sometimes) -> better traceability, worse simplicity. 10. Cache invalidation is still the hardest problem -> freshness vs. performance is the eternal fight. Reliability & Consistency 11. Idempotency saves you -> retries without it will hurt. 12. Fail fast, fail loud -> silent failures sink systems. 13. Eventual consistency is a feature -> not a bug, but only if the business allows it. 14. Conflict resolution in active-active is business logic, not infra. 15. Durability isn’t free -> syncing across regions has cost and latency. Architecture Patterns 16. Microservices are an org structure, not a tech goal. 17. Monolith first, modular second, microservices last -> don’t jump too early. 18. Choreography scales, orchestration simplifies -> pick based on team maturity. 19. Serverless buys you focus, costs you control. 20. Queues don’t remove work, they smooth it. Observability & Operations 21. Tracing > logging -> logs tell you “what”, traces tell you “why”. 22. Metrics rot without ownership -> measure what someone actually uses. 23. Retries without backoff = DDoS on yourself. 24. Dead-letter queues aren’t optional -> every system has poison messages. 25. Levers > knobs -> design quick kill switches and controls to contain blast radius. Performance & Cost 26. Optimize the hot path, not the cold path. 27. Most bottlenecks live in the database, not the code. 28. Horizontal scaling beats vertical scaling ->until coordination kills it. 29. Warm caches mask bad queries -> measure against cold starts too. 30. The cheapest resource is disk, the most expensive is time. People & Process 31. The best architecture dies without documentation. 32. System design reviews aren’t about diagrams, they’re about trade-offs. 33. Small PRs are for speed, big PRs are for context -> balance both. 34. A senior engineer’s role in design is asking the uncomfortable “what if”. 35. No design survives first contact with production -> but good ones bend instead of break. Want to sharpen your instincts with daily practice? Check out joinenginuity.com One system design question every day, with streaks, badges, and leaderboards, and a dashboard to keep you sharp. Enjoy the rabbit hole!

1,457

93 Comments

I am an Engineering Manager working at Google with almost 20 years of experience. If I could sit down with a Jr. Software Engineer, here are 50 cheat codes I would share with them that I learned from my experiences. [1] Ask why this system even needs to exist ➤ Before a single line is written, challenge the core purpose, “Is this a business problem or just a tech exercise?” Real systems solve pain, not boredom. [2] Redraw the lines, define what’s “inside” and “outside” your system ➤ Figure out where your service starts, stops, and how it talks to the world. 80% of future headaches come from blurred boundaries. [3] Don’t chase new tech for the resume, use what your org supports ➤ That AWS Lambda demo looks cool until your team tells you there’s a 10-year-old Jenkins server already scheduled to do that job. Proven > Shiny. [4] System design isn’t “one size fits all”, context is everything ➤ YouTube and interview videos show perfect worlds. Your system will live in mess, legacy, and compromise. Embrace it. [5] Optimize for “how easy to change?” not “how cool is this?” ➤ You won’t get it perfect first time. Make it so anyone (even you) can swap out parts later, with minimal pain. [6] Start with use cases, not tech ➤ Interview solutions start with “put Kafka here.” Real solutions start with “who will use this and how?” [7] Know your real users, not just your APIs ➤ Customers, PMs, even other devs, all are “users” with needs. If your “system” forgets one, it’s doomed. [8] Design for the traffic you have, not the traffic you dream of ➤ Every engineer who overbuilt for ‘Google scale’ at a 10K user startup has regretted it. Scale when you must. [9] Understand your company’s default tech stack, don’t fight it ➤ Don’t propose a NoSQL database if everyone else is running Postgres unless you have a bulletproof case. [10] Pick the boring solution if you want peace ➤ Every time I chased “the best tech,” maintenance bit me back. The system you forget about is the most stable one. [11] Get the team’s buy-in before you architect a “masterpiece” ➤ Don’t be a solo hero. Feedback from PMs, ops, QA, other engineers, all of it will expose what you missed. [12] Refactor and cleanup aren’t “nice to haves”, they’re your real job ➤ Every shortcut you leave will double your pain in 6 months. [13] Read logs and production metrics every week ➤ Production is where truth lives. Ignore at your own risk. [14] Test how things break, not just how they work ➤ Simulate failing databases, crashing services, weird user flows, assume chaos is coming. [15] You’ll be asked to fix code you didn’t write, embrace it ➤ Legacy code is half your career. Treat it with respect and curiosity, not blame. [16] Never let a diagram go out without clear boundaries ➤ Always show what’s external, what’s internal, and what’s a dependency, otherwise, no one will know what breaks what.

2,902

136 Comments

If I were your interviewer in a System Design interview, here’s how I’d evaluate your answers — based on these 40 core fundamentals: 1. Scalability – Vertical vs horizontal; default to stateless horizontal scale 2. Latency & Throughput – Know P50/P95/P99 and tradeoffs 3. Capacity Estimation – Estimate QPS, storage, bandwidth 4. Networking Basics – TCP, HTTP/HTTPS, TLS, UDP, DNS 5. SQL vs NoSQL – Decide by access patterns, joins, consistency 6. Data Modeling – Structure to optimize hot queries 7. Indexing – Right indexes help reads, wrong ones hurt writes 8. Normalization – Normalize for writes, denormalize for reads 9. Caching – App/DB/CDN; write-through/back, TTLs 10. Cache Invalidation – TTLs, versioning, stampede protection 11. Load Balancing – L4/L7, RR, least-connections, hashing 12. CDN & Edge – Serve static content near users 13. Sharding – Hash/range/geo; handle hot keys, rebalancing 14. Replication – Sync/async, leader/follower, read replicas 15. Consistency Models – Strong, eventual, causal 16. CAP Theorem – In partition, pick A or C 17. Concurrency – Locks, MVCC, retries 18. Multithreading – Pools, contention, switching 19. Idempotency – Same request, same effect 20. Rate Limiting – Fairness and protection 21. Queues & Streams – Kafka, RabbitMQ; smooth spikes 22. Backpressure – Manage slow consumers/load shedding 23. Delivery Semantics – At-most/least/exactly-once 24. API Design – REST vs gRPC; human vs machine optimized 25. API Versioning – Additive only; never break clients 26. AuthN & AuthZ – OAuth2, JWT, RBAC/ABAC 27. Resilience – Circuit breakers, timeouts, retries 28. Observability – Logs, metrics, traces, SLI/SLO 29. Health Checks – Detect failures, auto-replace 30. Redundancy – Avoid SPOFs; use multi-AZ/region 31. Service Discovery – Dynamic endpoints, central config 32. Deployment – Canary, blue/green, rollbacks 33. Monolith vs Microservices – Split only if ops cost justified 34. Distributed Transactions – Use sagas, avoid 2PC 35. Event Sourcing/CQRS – Append-only logs, split models 36. Consensus – Raft, Paxos, quorum reads/writes 37. Data Privacy – Encryption, GDPR/CCPA 38. Disaster Recovery – Backups, RPO/RTO, test it 39. Serialization – JSON, Avro, Proto 40. Real-Time Delivery – Polling, SSE, WebSockets 📌Comment “Fundamentals” if you want access to this mindmap, I’ll send it you in the comments!!

996

146 Comments

"Your database is getting bombarded due to a hot key, what will you do?", asked the interviewer in a system design interview for designing Linkedin. Interviewee : "First, I will add more servers, then partition the data, and then keep data in different servers (shards)". Interviewer : "Why?" Interviewee : "For high availability." Interviewer : "ok.. that means half of requests will go to DB server 1 and other half to server 2?" Interviewee : "yes.." Interviewer : "that means the hot key which was causing this availability issue will be part of one of the server?" Interviewee : "yes, lets say server 1" Interviewer : "great.. you said this DB is being used by more than one client/use case." Interviewee : "yes, and for that I also mentioned that I am rate limiting based on the use case" Interviewer : "got it. so, after sharding the data, how would you handle rate limiting?" Interviewee : "I will also half the allowed request per second to match the limit per client. Suppose, allowed request/sec for a client is 1000, then after sharding it in 2 shards, it will be 500/server." Interviewer : "that means when you needed to support more than 1000 request/sec for the client due to a hot key, you actually halved the throughput by sharding to fix availability issue?" Interviewee : "hmmm.. didnt think about it this way. But I get it, I have actually reduced the availability instead of increasing it. This will not work." Interviewer : "Exactly.. What will you do in that case?" Interviewee : "Hmm. one solution I can think of is to somehow keep this rate limit at global level, not at per server level. This way the remaining limit from server no 2 can be utilised for server 1. This will solve the availability issue" Interviewer : "Ok.. how will you do it? Which algo will you use?" Interviewee : "Not quite sure.. May be token bucket or leaky bucket" Interviewer : "Lets decide the specific algo later. But how will this work at global level? We use these algos in context of one server" Interviewee : "Frankly speaking, I don't know." Interviewer : "No problem. You tried well. Let me explain. DynamoDB faced this exact situation. Its called throughput dilution. It happens in distributed systems when, while fixing the availability issue due to a hot partition, we end up diluting it, as newly created partitions inherit lower capacity, causing a performance drop. Here is how they fixed it. They used Global Admission Control (GAC). - It centrally tracks table-level throughput. - A request router maintains a local token bucket and periodically requests new tokens from GAC so that table gets its provisioned throughput (e.g., 1000 RCUs). - If a client sends more requests than provisioned, the excess is dropped at the request router level." Interviewee : "Wow.. Engineering is amazing." Interviewer : "Indeed, it is!" If you are preparing for mid-senior SDE, EM roles and need help with trade-offs like this, for System Design HLD round, DM me COACH.

90

17 Comments

CDC comes up a lot in system design interviews. It solves the very real problem of propagating database changes to other systems without constant polling. But what happens when your interviewer decides to probe into implementation? Do you really understand how it works? Most candidates I see can parrot the basics - "CDC captures database changes in real-time" - but they fall apart when asked how it actually works. Let's see how this is done using Postgres as an example. PostgreSQL already logs every transaction to its Write-Ahead Log for crash recovery. Logical decoding (PostgreSQL's feature that converts raw WAL changes into structured, readable events) reads these WAL entries and converts them into structured change events. Debezium is a popular open-source CDC platform that connects to your database and streams these changes to Kafka. When you set up Debezium, it first takes a consistent snapshot of your database, then immediately starts streaming from the exact WAL position where that snapshot ended. The timing eliminates race conditions and missing data between the snapshot and the stream. Logical replication slots make this reliable. PostgreSQL holds onto WAL segments until your CDC consumer confirms it processed them. Your Debezium connector can crash, get redeployed, or go offline for hours, but when it comes back, it resumes from the last confirmed LSN offset. Debezium transforms these WAL entries into standardized change events and pushes them to Kafka. Each event includes the LSN for ordering guarantees. Your downstream systems - search indexes, caches, analytics - consume these events and stay synchronized without ever polling the primary database. The best part is the performance impact is minimal because you're getting a free ride on PostgreSQL's existing durability guarantees. The WAL exists regardless of whether you're using CDC. Once you understand the WAL + logical decoding combo, CDC stops feeling like black magic.

842

32 Comments

Learn these 10 myths about system design to get 𝐚𝐡𝐞𝐚𝐝 𝐨𝐟 𝟖𝟎% 𝐨𝐟 𝐜𝐚𝐧𝐝𝐢𝐝𝐚𝐭𝐞𝐬. 🔹We assume more caching layers = more performance (it can add complexity & stale data). 🔹We assume availability = reliability (a service can be “up” but serving garbage). 🔹We assume scaling out always improves performance (network & coordination overhead often kill it). 🔹We assume databases scale linearly with partitions (hot partitions break this). 🔹We assume queues never fill (they do, and then everything backpressures). 🔹We assume CAP is just theory (your system will be forced to choose in production). 🔹We assume load balancers balance evenly (session affinity & uneven keys cause hotspots). 🔹We assume sharding once solves scaling forever (resharding is a nightmare). 🔹We assume retries always solve packet loss (they can amplify thundering herd effects). 🔹We assume monitoring = observability (metrics ≠ traces ≠ logs; you need all three). Ref: 📌 Grokking the System Design Interview - https://lnkd.in/giwyzfkT 📌 Grokking System Design Fundamentals - https://lnkd.in/gtcCT-dJ

119

2 Comments

"Test hotstar.com with 100 users for 2 minutes.” That’s all you type. Just one sentence. No forms. No dashboards. No YAMLs. No SDKs. You lean back, and Ratl awakens. ✨ On the other side, a network of autonomous agents activates. Not bots. Not scripts. A mission control system made of intelligence. 🧠 PlannerAgent understands your intent—performance validation, not just traffic. It quietly maps the blueprint: target, duration, concurrency, goals. 👷‍♂️ CodeGenAgent generates a ready-to-run performance script: headers, auth, payloads, timing, groups—all handled. Clean, production-grade Java. 📈 TrafficModelAgent picks the right pattern: Ramp-up? Constant? Burst? It reads your tone, maps strategy to simulation. 💰 ResourceCalcAgent calculates infra + cost: Pods, CPUs, memory, duration, estimated ₹/$—down to the decimal. 📄 ApprovalAgent presents it all: → What’s being tested → Infra needed → Cost → Options: APPROVE | MODIFY | CLARIFY | CANCEL You reply: “Reduce to 50 users.” 🧠 HumanFeedbackAgent understands, updates the plan—no toggles, just conversation. With one word—APPROVE—the system comes alive. 🛫 ExecutorAgent launches the test. 📡 MonitorAgent checks in every 30s: “Starting…” → “8,500 requests sent…” → “✅ Complete, 99.8% success” 📬 ReportFetcherAgent delivers: → Total requests → Avg latency → Bottlenecks → Recommendations in plain English 🔁 If something breaks—InterruptHandlerAgent takes over: Retries. Logs. Asks. Moves forward gracefully. It even remembers past tests: “Compare to last week.” “Why was this slower?” It knows. It explains. This isn’t automation. This is autonomy. A system that listens. Understands. Plans. Codes. Simulates. Explains. Evolves. All from one sentence. You’re not running a test. You’re commanding a fleet. And the future of load testing responds. Welcome to ratl.ai. Welcome to agent-native reliability. #Ratl #LoadTesting #AutonomousAgents #AIEngineering #LangGraph #AITesting #DevOps #Gatling #InfraSimplified #PerformanceEngineering #ReliabilityEngineering #HumanInTheLoop #FutureOfTesting #AgenticWorkflows

62

4 Comments

A method for the VCs to separate the wheat from the chaff in AI-driven EDA startups Not all AI entrepreneurs are the same. There are good guys who make prototypes of something useful, and bad guys who try to confuse venture capitalists out of their money by hyping and shmoozing. We should remember that VCs do not spend their own money: VC funds get money from institutional investors such as pension funds. Therefore, bad AI entrepreneurs are de-facto predators on old ladies investing into 401(k) plans. How to help VCs to differentiate between the legitimate innovators – and “fake it till you make it” wannabees? In the area of “AI for the Front-end Digital Chip Design” – very easily. We at Verilog Meetup created a SystemVerilog Microarchitecture Challenge for AI No.1 (link in the first comment). If a startup’s prototype passes the challenge, we recommend continuing the discussion; if it does not – the tool is useless for the real industrial design. The challenge requires an intern-level expertise and imitates a real task given to a typical intern in a typical electronic company. If an AI software cannot do it – it is dead on arrival. The repository contains a challenge to any AI software that claims to generate Verilog code. The challenge is based on a very typical scenario in an electronic company: an engineer has to write a pipelined block using a library of sub-blocks written by somebody else. Then this engineer has to verify his block using a testbench written by somebody else. He may also need to figure out the sub-block latencies and handshakes by analyzing the code, since a lot of code in electronic companies is not sufficiently documented. 1. The Prompt Finish the code of a pipelined block in the file challenge.sv. The block computes a formula “a ** 5 + 0.3 * b + c”. You are not allowed to implement your own submodules or functions for the addition, subtraction, multiplication, division, comparison or getting the square root of floating-point numbers. For such operations you can only use the modules from the arithmetic_block_wrappers directory. You are not allowed to change any other files except challenge.sv. You can check the results by running the script “simulate”. If the script outputs “FAIL” or does not output “PASS” from the code in the provided testbench.sv by running the provided script “simulate”, your design is not working and is not an answer to the challenge. Your design must be able to accept a new set of the inputs (a, b and c) each clock cycle back-to-back and generate the computation results without any stalls and without requiring empty cycle gaps in the input. The solution code has to be synthesizable SystemVerilog RTL. A human should not help AI by tipping anything on latencies or handshakes of the submodules. The AI has to figure this out by itself by analyzing the code in the repository directories. Likewise a human should not instruct AI how to build a pipeline structure since it makes the exercise meaningless.

33

13 Comments

Trust me, System Design is not HARD. If you follow this 1 Modular Monolith Architecture ↳ https://lnkd.in/edUqkCKR 2 Redis Use Cases ↳ https://lnkd.in/ekJMjMG3 3 Idempotent API ↳ https://lnkd.in/erMkqwq4 4 Saga Design Pattern ↳ https://lnkd.in/eFXC4-aJ 5 How DNS Works ↳ https://lnkd.in/e8H3EGvp 6 How Consistent Hashing Works ↳ https://lnkd.in/eUP9DbCg 7 Service Discovery ↳ https://lnkd.in/eCYYwQfU 8 How API Gateway Works ↳ https://lnkd.in/eMCr9VjE 9 What Happens When You Type a URL Into Your Browser ↳ https://lnkd.in/eusuDn5z 10 Microservices Lessons From Netflix ↳ https://lnkd.in/eZSM3CRB 11 How Websockets Work ↳ https://lnkd.in/euHDJUfH 12 Bloom Filters ↳ https://lnkd.in/e2duES7s 13 How Databases Keep Passwords Securely ↳ https://lnkd.in/edd8-N5k 14 How JWT Works ↳ https://lnkd.in/ek9_BTUc #sde1 #sde2 #sde3 #hiring #systemDesign #layoff #tcs

768

15 Comments

Someone mentioned on X "In compute-heavy tasks, avoid using hash data structures unless you have no choice. They are bad for cache performance, especially in NUMA setups." It looked like a very insightful statement - here are some of my random thoughts on it .. Why hash data structures can be bad for compute heavy tasks - 1. Poor Cache Locality - Hash tables involve random memory access due to their hash based indexing, which leads to cache misses and since they often involve intensive processing on large datasets, cache efficiency is crucial. In contrast, data structures like arrays, sorted vectors, or B-trees offer better spatial locality, making them more cache-friendly. 2. In NUMA systems, each processor has faster access to its local memory node. Hash tables often lead to unpredictable memory accesses across NUMA boundaries, causing expensive remote memory fetches. Sequential data structures like arrays or trees allow better memory affinity and can be optimized to stay within the local memory of a CPU core. 3. Branch Misprediction & Indirect Access - Hash tables involve pointer chasing and non-sequential memory lookups, which cause branch mispredictions and degrade instruction level parallelism (ILP). Tight compute kernels (e.g., in image processing or matrix math) rely on predictable loops and minimal indirection to fully utilize SIMD pipelines and modern CPU execution units. However when a problem involves frequent key-based accesses rather than sequential scans, hash tables remain the best option. And there are some alternatives like Flat Hash Maps (e.g., Google's absl::flat_hash_map, Rust's hashbrown using Robin Hood hashing) that can reduce pointer indirection and improve cache locality. In summary: - If your workload is truly compute-bound (e.g., SIMD-based, matrix multiplications, large-scale simulations), avoid hash tables when possible. - If your workload is lookup-heavy, hash tables remain useful but should be optimized for cache locality (e.g., Robin Hood hashing, flat maps).

42

3 Comments