Maxime Mathy

🚀 Scaling Microservices with the Outbox Pattern (with Debezium) One of the biggest challenges in event-driven microservices is the dual-write problem: ✅ DB transaction succeeds but event publish fails ❌ Or event goes to Kafka but DB rolls back Result → inconsistent systems 😓 This is where the Outbox Pattern comes in! 🌟 👉 How it works: In the same DB transaction, save your business data (e.g., Order) and the event (e.g., OrderCreated) in an outbox table. A background process then reliably reads from the outbox table and publishes the event to Kafka, RabbitMQ, etc. 💡 Where Debezium fits in  Debezium is a CDC (Change Data Capture) tool built on top of Kafka Connect.  Instead of writing your own polling code for the outbox table, Debezium can: 👀 Watch DB transaction logs in real time 🔄 Capture inserts/updates/deletes in the outbox table 📤 Push those changes directly into Kafka topics ⚡ Guarantee ordering + exactly-once delivery (with Kafka + idempotent consumers) This makes the Outbox Pattern + Debezium super powerful: ✅ Atomicity → DB + outbox write are in one transaction ✅ Reliability → Events are never lost, even during failures ✅ Simplicity → No custom polling, Debezium does the heavy lifting 🔍 Real-world example: When a new order is placed 🛒: Insert order details + an “OrderCreated” event in the outbox table Debezium detects this new row from DB logs Publishes it into Kafka Inventory, Payment, and Notification services consume reliably 🎯 ✨ Pro Tips: Keep your outbox table clean using TTL or archiving. Tune Debezium connectors for low latency streaming. Pair with Kafka transactions for exactly-once guarantees. 💬 Have you tried Debezium + Outbox Pattern in production? ✅ Yes, works great ⚡ Yes, but with challenges (latency/cleanup) 🤔 Not yet, but exploring Would love to hear your experiences and lessons learned 👇 #Microservices #Debezium #Kafka #EventDriven #SystemDesign #OutboxPattern #SpringBoot #BackendEngineering #Java

118

7 commentaires

What people think tech leads do: • Architect solutions • Lead teams • Make decisions What tech leads actually do: • Attend meetings about meetings • Explain why "Kubernetes" isn't the answer • Say no to impossible deadlines • Translate dev speak to business speak • Google "how to motivate developers" • Question life choices Tech Lead Morning Routine: 1. Coffee 2. Check if prod is down 3. More coffee 4. Cancel unnecessary meetings 5. Fail to cancel meetings 6. Attend unnecessary meetings 7. Wonder why no code gets written What's your tech lead survival tip? Share your pain below 👇 *** P.S. The secret is more coffee. Always more coffee. ☕

209

19 commentaires

Finally published a new post after a month-long break from writing... Did you know that localhost is treated as a secure context even without TLS? I had no idea until very recently. This allows secure features to work in development, and you can also run multiple apps on localhost with subdomains + ports to separate them. This means you don't need HTTPS locally, most of the time. That said, when you do need local HTTPS, use Caddy server. The article shows how. As always, I hope you find it useful and you learn something new. https://lnkd.in/dc3V-QSG

57

1 commentaire

📝 In my last article, I want to reevaluate my thoughts and see whether the existence of GraalVM means Java is now suitable for CLI programs or not. During the article, we compare two implementations of a Money converter CLI tool in Java and Go to see the difference. 🔗 Article link: https://lnkd.in/dAxxqWhP

268

17 commentaires