re:Invent 2022 DAT326 Deep dive into Amazon Aurora and its innovations

Editor's Notes

• #4: Aurora is a cloud-native database engine. We designed it to meet the needs of enterprise with demanding requirements in terms of features, scaling and performance. We're talking about customers who need powerful and full-featured databases, but are getting tired of the legacy databases’ punitive licensing, their significant expense, their lack of cloud-native capabilities, etc. So Aurora is our answer to that need. At the SQL prompt, Aurora looks and feels just like Postgres or MySQL. But behind the scenes, it features a distributed, fault-tolerant, self-healing storage system that auto-scales up to 128TB per database. Each Aurora cluster replicates automatically across three Availability Zones, while at the same time, delivering high performance and availability with up to 15 low-latency read replicas. So in essence, Aurora is cloud-native, massively scalable and available database. It aims to address high-end enterprise use cases in a way that MySQL and Postgres can’t do by themselves. And Aurora aims to do it at a lower cost to customers than the legacy commercial databases.
• #127: DAT221- Thurs
• #128: Until today, RDS and Aurora supported solely a curated set of 85 plus PostgreSQL extensions. However, we heard from developers that they want access to the broader library of PostgreSQL extensions to use in production. TLE allows you to improve the time to market by allowing you to deploy extensions on Amazon Aurora on RDS as soon as you determine that an extension meets your needs. This is possible through AWS’ Shared Responsibility model. You no longer need to wait for AWS to support an extension to begin implementation because TLE extensions are considered part of your application. Previously, building a successful PostgreSQL extension required expert orchestration with C language. TLE uses popular PostgreSQL trusted languages, including JavaScript, PLpgSQL, and Perl, to improve extension builders’ productivity, letting developers efficiently create extensions. DBAs have control over who can install TLE extensions, making it possible for select application developers to test an extension prior to production use. Furthermore, all can rest assured that any defects in an extension’s code is limited to a single database connection.
• #129: TLE raises the bar on PostgreSQL extension creation and use. TLE is designed to provide you better safety, support for high performance programming languages, and removes AWS certification from your project. TLE is open source, so you can see what it does, and you can make it better. As we like to say at Amazon, it is still day one for this project. We would like this open-source project to become the standard for creating extensions for PostgreSQL, making it easier for developers to innovate. We recognize that an open-source project cannot be successful without the community. With this project, we can enable developers with tools to innovate faster and create a better experience for all PostgreSQL lovers.
• #158: For a long time, customers have been telling us that they like a lot of things about Performance Insights and CloudWatch, which let them explore all kinds of issues around database performance and troubleshooting. But you’ve also told us loud and clear that you’d like a little more help tracking down potential problems and even more importantly, figuring out what to do about them. That’s where DevOps Guru comes in. Since being released DevOps Guru has been helping customers by telling them about unusual and problematic performance behavior throughout their application stacks. This week we’re raising the bar for database diagnostics in DevOps Guru, bringing detailed database-specific capabilities to DevOps Guru. DevOps Guru for RDS goes several steps further than Performance Insights, by using machine learning to detect and diagnose performance problems in your databases, in order to help you fix those problems quickly.
• #161: Customers feedback has been consistent that they, that you, want to see the unique value GuardDuty provides expand to protect more of your AWS resources, and as we mentioned, perhaps most importantly your data. Protect your data in RDS, starting with Aurora – suspicious logins that we identified as a critical level of visibility to identify an early stage of threats to DBs that allows you to mitigate threats before the escalate, and further put your data at risk. Single click – org wide. With machine learning models that accurately detect suspicious logins to your RDS DBs.
• #163: Now I’d like to give you a look inside this new feature so you can understand how it detects threats. Perhaps you feel we glossed over the important part, where we detect suspicious activity. What is suspicious? How do we know?
• #166: Well what can you do if Guard Duty RDS Protection detects an issue? If Guard Duty RDS Protection tells you that connection attempts are coming from an atypical IP address range, you can tighten your security group posture to prevent unauthorized hosts from connecting, especially those outside your VPC. If Guard Duty RDS Protection tells you that a user you don’t recognize has connected to the system database, you can terminate that connection at the database, and rotate the credential that was used, preventing further connections. Any whenever Guard Duty RDS Protection detects unusual activity, it’s a great opportunity for your security team to review database audit records to determine if the issue is part of a larger pattern of misuse or abuse.
• #191: Fastest way to go from transaction to insights Fastest way to go from transaction to ML driven insights Easy and reliable Unify multiple sources
• #192: Automated data seeding Single-digit second replication lag Monitoring and recovery Low Latency Let’s take a closer looks at some other aspects of this integration. Creating a Redshift integration target, whether it’s a new of existing endpoint, is easy with zero-etl. Each Aurora database cluster is mapped into a Redshift database. A Redshift endpoint can support multiple integrations. Data can be ingested into Redshift in parallel, even as multiple concurrent queries are running in Redshift. Once the data is in Redshift, you can transform data with materialized views for improving performance. You can also further share data between Redshift clusters using Data Sharing.    We have designed this integration for easy maintenance. This integration adapts to Aurora side schema changes. Database or table additions and deletions are handled transparently. If a transient error is encountered, the integration automatically re-synchs after the recovery from the error. There is often a need for other permutations of data movement from one purpose built database to another, and we’ll turn to this topic next.