Deepit Patel

Most early engineers (in some cases even seniors) get confused with the word "consistency", but remember, Consistency in ACID is different from Consistency in CAP. Here's how they differ... ACID consistency is about the correctness of data within a single database and is enforced at the transaction level. CAP consistency is about the consistency of data across distributed nodes. So, ensuring that there are no blogs in the database whose author does not exist is ACID consistency; while ensuring that all the blogs are present across all the database nodes across all regions is CAP consistency. You use `NOT NULL`, `UNIQUE`, `CHECK`, and `FOREIGN KEY` constraints to implement ACID consistency. On the other hand, you use replication, gossip, locks, and consensus protocols to achieve strong or eventual consistency across nodes. Here's where it gets tricky in practice... When you are building a distributed application with multiple database replicas, you're dealing with BOTH types of consistency simultaneously. Your primary database enforces ACID consistency (no orphaned blogs), while your replication strategy determines CAP consistency (how quickly that constraint violation is reflected across all replicas). Here's the mental model that would make it easier for you to understand and remember. Think ACID = Is my data logically correct? Think CAP = Is my data the same everywhere? So, given that the same word is used to describe both, it is important to understand the context in which the word is used. Hope this helps.

872

23 Comments

35 practical thoughts on system design Core Principles 1. Every system is a trade-off -> you never get speed, cost, and simplicity all at once. 2. Latency compounds -> every millisecond added across layers adds up to user pain. 3. Scalability ≠ performance -> one handles growth, the other handles speed. 4. Read vs. write paths -> scaling each requires completely different strategies. 5. Design for change, not perfection -> requirements will shift. Databases & Storage 6. Indexes are levers -> high-selectivity columns are worth indexing, low ones often aren’t. 7. Replication helps reads, partitioning helps writes -> don’t confuse the two. 8. Dual writes are a lie -> without coordination, you will see drift. 9. Event stores > queues (sometimes) -> better traceability, worse simplicity. 10. Cache invalidation is still the hardest problem -> freshness vs. performance is the eternal fight. Reliability & Consistency 11. Idempotency saves you -> retries without it will hurt. 12. Fail fast, fail loud -> silent failures sink systems. 13. Eventual consistency is a feature -> not a bug, but only if the business allows it. 14. Conflict resolution in active-active is business logic, not infra. 15. Durability isn’t free -> syncing across regions has cost and latency. Architecture Patterns 16. Microservices are an org structure, not a tech goal. 17. Monolith first, modular second, microservices last -> don’t jump too early. 18. Choreography scales, orchestration simplifies -> pick based on team maturity. 19. Serverless buys you focus, costs you control. 20. Queues don’t remove work, they smooth it. Observability & Operations 21. Tracing > logging -> logs tell you “what”, traces tell you “why”. 22. Metrics rot without ownership -> measure what someone actually uses. 23. Retries without backoff = DDoS on yourself. 24. Dead-letter queues aren’t optional -> every system has poison messages. 25. Levers > knobs -> design quick kill switches and controls to contain blast radius. Performance & Cost 26. Optimize the hot path, not the cold path. 27. Most bottlenecks live in the database, not the code. 28. Horizontal scaling beats vertical scaling ->until coordination kills it. 29. Warm caches mask bad queries -> measure against cold starts too. 30. The cheapest resource is disk, the most expensive is time. People & Process 31. The best architecture dies without documentation. 32. System design reviews aren’t about diagrams, they’re about trade-offs. 33. Small PRs are for speed, big PRs are for context -> balance both. 34. A senior engineer’s role in design is asking the uncomfortable “what if”. 35. No design survives first contact with production -> but good ones bend instead of break. Want to sharpen your instincts with daily practice? Check out joinenginuity.com One system design question every day, with streaks, badges, and leaderboards, and a dashboard to keep you sharp. Enjoy the rabbit hole!

1,459

93 Comments

Imagine you just joined Google as an SWE. It’s your first week, and you’re overthinking everything. You’re surrounded by world-class engineers, new systems, and an overwhelming urge not to mess up. The pressure to prove yourself is real, and so is the anxiety…so what should you do? Over the years, I’ve moved across orgs, teams, and sometimes, entire companies. Here’s exactly what I did (and still do) to ramp up faster and enjoy the process without letting anxiety win → Don’t let yourself stay stuck. If you’re blocked for more than 30 minutes, ask for help. Share what you’ve tried, show initiative, and intent. → Read way more code than you’re assigned. Dive into related modules, historic PRs, and old design docs. The context you gain helps you spot patterns and avoid mistakes. → Ask about the story behind the system. Find out why things are built a certain way, what’s failed before, and what’s on the roadmap. Understanding the “why” beats memorizing the “what.” → Volunteer for unglamorous work. Take ownership of tests, docs, or nagging integration bugs. This is where you quietly learn the most and build your internal network. → Offer help before you’re an expert. Chime in when you see questions, review PRs, or share a script. Even small contributions get you noticed for the right reasons. → Connect with skip-levels and cross-team leads. Set up short intros to learn about the broader vision and team priorities. You’ll spot opportunities to add value beyond your immediate tasks. → Document what you learn as you go. Share notes, update READMEs, or post quick how-tos in team channels. Your future self and your teammates will thank you. → Shadow someone during incidents or launches. Sit in on debugging sessions or war rooms, even if you’re just observing. You’ll see how senior folks think, react, and communicate under pressure. → Keep a running list of “how did that break?” moments. Every time you see a system fail or a hotfix go live, dig in. Reverse-engineering past problems is the fastest way to understand the architecture. → Show up with curiosity and intent, every day. Don’t just tick off JIRA tickets. Look for the bigger picture. People notice when you genuinely want to learn and contribute, not just “do your job.” Some orgs assign mentors, some don’t. Either way, these habits put you in the driver’s seat. Ramping up is less about brilliance, more about consistent intent. And the ones who learn this early end up leading teams later.

708

52 Comments

System Design interviews aren’t about giving perfect answers — they’re about how you think. They're testing: ✅ Your ability to make trade-offs ✅ How you justify decisions ✅ How you handle failures ✅ And how clearly you can communicate Here’s how you can get better — consistently and practically: 🚀 Step-by-step guide to learning System Design: 🔹 1. Master the Basics: Start with understanding the building blocks: - Load Balancers - Databases (SQL vs NoSQL) - Caches (Redis, Memcached) - Queues (Kafka, RabbitMQ) - CDNs, Proxies, and APIs 📘 𝐑𝐞𝐜𝐨𝐦𝐦𝐞𝐧𝐝𝐞𝐝 𝐑𝐞𝐬𝐨𝐮𝐫𝐜𝐞: Grokking the System Design Interview (great for interview prep) 🔹 2. Learn to Think in Trade-offs: There’s no one “right” answer. Learn to ask: - Why this database over that? - What if traffic doubles overnight? - What if this service fails? 📘 𝐑𝐞𝐬𝐨𝐮𝐫𝐜𝐞: System Design Primer (GitHub repo) 🔹 3. Practice with Real-world Problems: Don’t just watch videos. Design on paper. Try these: - Design Instagram - Design Uber - Design WhatsApp - Design a URL shortener 🎥 𝐑𝐞𝐬𝐨𝐮𝐫𝐜𝐞: YouTube channels like Gaurav Sen, Hussein Nasser 🔹 4. Discuss with Peers / Mentors: Mock interviews or peer discussions can teach you what books can’t — clarity of thought and communication. 🤝 Tip: Join communities like Discord groups, Slack channels, or Reddit forums. 🔹 5. Start Small, Go Deep: Instead of trying to “scale to millions” from day 1, try designing for 100 users first. Then go: → Single server → Multi-server → Simple DB → Replication/Sharding → No cache → Smart caching strategy 🛠 That’s how real systems evolve. 🔹 6. Bonus: Build side projects Want to really understand system design? Build something small: - A chat app - A URL shortener - A blog platform You’ll run into real-world trade-offs fast. And learn way more than any tutorial. 📌 Remember: System Design is not a one-time topic. It’s a muscle — built by designing, reflecting, and iterating. No need to be perfect. Just be curious. Consistent. Thoughtful. Follow Upasana Singh for more such reads

80

Someone mentioned on X "In compute-heavy tasks, avoid using hash data structures unless you have no choice. They are bad for cache performance, especially in NUMA setups." It looked like a very insightful statement - here are some of my random thoughts on it .. Why hash data structures can be bad for compute heavy tasks - 1. Poor Cache Locality - Hash tables involve random memory access due to their hash based indexing, which leads to cache misses and since they often involve intensive processing on large datasets, cache efficiency is crucial. In contrast, data structures like arrays, sorted vectors, or B-trees offer better spatial locality, making them more cache-friendly. 2. In NUMA systems, each processor has faster access to its local memory node. Hash tables often lead to unpredictable memory accesses across NUMA boundaries, causing expensive remote memory fetches. Sequential data structures like arrays or trees allow better memory affinity and can be optimized to stay within the local memory of a CPU core. 3. Branch Misprediction & Indirect Access - Hash tables involve pointer chasing and non-sequential memory lookups, which cause branch mispredictions and degrade instruction level parallelism (ILP). Tight compute kernels (e.g., in image processing or matrix math) rely on predictable loops and minimal indirection to fully utilize SIMD pipelines and modern CPU execution units. However when a problem involves frequent key-based accesses rather than sequential scans, hash tables remain the best option. And there are some alternatives like Flat Hash Maps (e.g., Google's absl::flat_hash_map, Rust's hashbrown using Robin Hood hashing) that can reduce pointer indirection and improve cache locality. In summary: - If your workload is truly compute-bound (e.g., SIMD-based, matrix multiplications, large-scale simulations), avoid hash tables when possible. - If your workload is lookup-heavy, hash tables remain useful but should be optimized for cache locality (e.g., Robin Hood hashing, flat maps).

42

3 Comments

API design is an important step in any system design interview. You'll need to be able to: 1. Choose the right "API protocol" ie. REST, GraphQL, RPC 2. Follow proper protocol semantics 3. Handle follow up questions about pagination, security, versioning, etc. I'll admit, my bias is that interviewers who are nit-picky here are focused on the wrong things. BUT, that being said, it's easy enough to get right if you know the rules of the game. We recently launched a new deep dive on API Design complete with a YouTube video to match! Let me know what you think :) Article: https://lnkd.in/gih4Wyp3 Video: https://lnkd.in/gy_QNvRG

181

6 Comments