Webinar: Serverless Architectures with AWS Lambda and MongoDB Atlas
4 likes1,621 views
The document outlines the implementation of serverless architectures using AWS Lambda and MongoDB Atlas, detailing frameworks, use cases, and architectural considerations. It emphasizes microservices, event-driven processing, and the integration of security and monitoring features in serverless applications. Key topics include Lambda functions, API Gateway configuration, VPC peering, and pricing models for MongoDB Atlas and AWS services.
Webinar: Serverless Architectures with AWS Lambda and MongoDB Atlas
- 1. Serverless Architectures with AWS Lambda and MongoDB Atlas Ashley Townsend Senior Solutions Architect
- 2. Serverless? • Landscape • Use cases Going Serverless • What changes? • Considerations • MongoDB Atlas AWS & MongoDB Atlas • Simple API for Customer Single View • Lambda & API Gateway • MongoDB Atlas & Compass • Postman Agenda
- 3. Serverless
- 4. Big Iron Commodity Hardware Virtualized Containers Functions Where will my code run?
- 5. Serverless Frameworks and Platforms https://github.com/serverless/serverless https://www.zappa.io/ Chalice (awslabs) https://github.com/awslabs/chalice Frameworks for Cloud providers On-Prem PaaS - Now offering FaaS
- 6. 1. Cloud services have matured 2. BaaS à “SaaS-ification” 3. API’s are the glue 4. Containers – now per function 5. SysOps à DevOps à NoOps Less Ops, More Engineering 5 factors fueling Serverless Computing https://www.forbes.com/sites/janakirammsv/2016/02/28/five-factors-that-are-fueling-serverless-computing-part-1
- 8. Scheduled Jobs • Sequencing / Orchestration (AWS Steps?) Data Quality • Trigger à Identify à Pass to function Micro or Nano services • Clicks or Taps Event and IoT processing • Don’t worry about scaling App Servers Lightweight API’s • Focus of today! Good fit for Serverless?
- 10. Container-based Microservices Payments Service Product Catalog Service Shopping Cart Service Domains https://www.mongodb.com/blog/post/serverless-architectures-the-evolution-of-cloud-computing
- 11. Serverless Microservices Command Query Responsibility Segregation Think about: Fine or course grained Shared logic Start-up time! Packaging Deployment Versioning
- 12. CQRS pattern on Serverless Microservices GET API PUT PATCH POST DELETE … API API Key API Key Lambda Function(s) Lambda Function(s) Code Code Lambda Function(s) VPC Peering
- 13. Shape • Person • Insurance Policies • Shape changes per policy type • Addresses Operations via API • GET Customers with soon-to-expire policies, within a geo radius • GET Customers / by SSN, id, etc. • PATCH Update basic contact info (cell, email, …) Customer Single View - Insurance Industry (hypothetical) High-level architecture of a single view platform
- 14. MongoDB Atlas & AWS
- 15. Required MongoDB Services – Atlas!
- 16. mgeneratejs • https://github.com/rueckstiess/mgeneratejs • npm install -g mgeneratejs • Create template – generate data • Upload to Atlas via mongoimport • Hint: get connection string from Atlas UI! • Browse with Compass Generate dataset Template (InsuranceC360_Customers.json) mgeneratejs -n 100 InsuranceC360_Customers.json | mongoimport --host ”YOUR ATLAS CLUSTER" --numInsertionWorkers 4 --db WebinarCustomerSingleView --collection Customers -- authenticationDatabase admin --ssl --username YOURUSER --password YOURPASSWORD
- 17. IAM • Role with Lambda execute policies VPC • VPC • Security Groups – traffic rules • Internet Gateway – outside communication • VPC Peering Connection - Route Table Required AWS Services Lambda • VPC, Security Group and IAM role • Develop inline or upload deployment package (.zip) • Use MongoDB Driver – connect with MongoDB Atlas API Gateway • API definition • API Keys & Usage Plans • Resources and HTTP Methods • Map Routes to Lambda functions
- 18. VPC
- 19. MongoDB Atlas • Provision a Cluster – M10+ à need an assigned AWS region for VPC peer • Same AWS region – (I will use us-west-2) • Initiate VPC peer with AWS AWS VPC • Accept incoming Peering Connection • Update Route Table EC2 • Install MongoDB • Test connection from the MongoDB Shell to ensure VPC Peer is working • Optional but highly recommended – ensure VPC Peering is working before proceeding to Lambda MongoDB Atlas peered with your AWS VPC
- 24. Security Group
- 26. Verify VPC Peer works
- 27. Lambda
- 28. Role with lambda permissions (IAM)
- 29. Code packaging from __future__ import print_function import json import pymongo print('Loading function') print(’=== CONNECTING TO MONGODB ATLAS ===') connstr = ”ENTER YOUR MONGODB ATLAS CONNECTION HERE" MONGOCLIENT = pymongo.MongoClient(connstr, readPreference=’secondaryPreferred’) def GET_lambda_handler(event, context): … implement GET logic def POST_lambda_handler(event, context): … implement POST logic http://docs.aws.amazon.com/lambda/latest/dg/lambda -python-how-to-create-deployment-package.html
- 30. Lambda functions
- 31. Upload & configure function The handler function The role with lambda permissions The VPC (peered with Atlas) The security group that allows traffic At least 2 subnets
- 32. The code:
- 33. API Gateway
- 34. Recap: What we said we would do High-level architecture of a single view platform
- 35. Recap: What we said we would do GET API PUT PATCH POST DELETE … API API Key API Key Lambda Function(s) Lambda Function(s) Code Code Lambda Function(s) VPC Peering
- 36. Read API – GET /api/v1/customers
- 37. CUD API - PATCH /api/v1/customers
- 40. Access and throttling via API Keys
- 41. Testing It
- 43. Test with Postman #1
- 44. Test with Postman #2
- 45. Load Test
- 46. AWS CloudWatch
- 47. Connections and containers… http://docs.aws.amazon.com/lambda/latest/dg/lambda-introduction.html … AWS Lambda maintains the container for some time in anticipation of another Lambda function invocation. … the service freezes the container after a function completes, and thaws the container for reuse. If AWS Lambda chooses to reuse the container, this has the following implications: - Any declarations in your Lambda function code (outside the handler code, see Programming Model) remains initialized, providing additional optimization when the function is invoked again. For example, if your Lambda function establishes a database connection, instead of reestablishing the connection, the original connection is used in subsequent invocations. You can add logic in your code to check if a connection already exists before creating one.
- 48. MongoDB Atlas Monitoring and Alerts
- 49. MongoDB Compass
- 50. Considerations…
- 51. Scaling? Scaling Lambda No user intervention required - Default safety throttle of 100 concurrent executions per account per region. Functions invoked synchronously throw 429 error code. Functions invoked asynchronously can absorb reasonable bursts for approx. 15-30 minutes. If exhausted, consider using Simple Queue Service (SQS) or Simple Notification Service (SNS) as the Dead Letter Queue (DLQ). Read more at https://aws.amazon.com/lambda/faqs/ Scaling MongoDB Atlas On-Demand Zero downtime Upscale/Downscale: • Instance size • Storage size • IOPS • Replication factor.
- 52. Pricing? Lambda Costs Cost depends on requests (per million), request time, memory (GB) allocated to each function. First 1 million requests per month free - $0.20 per 1 million requests thereafter. $0.00001667 for every GB-second used. Additional AWS services imply cost (e.g. API Gateway, …) Read more at https://aws.amazon.com/lambda/pricing/ MongoDB Atlas Costs Cost depends on instance size, storage, iops, replication factor and backup retention. M0 free – great for you (no VPC peering, use IP whitelist) M10 starts at $0.08/hr – great for team Dev M30 starts at $0.54.hr – great for Production Read more at https://www.mongodb.com/cloud/atlas/pricing
- 53. Connections to MongoDB Atlas Encrypt using AWS KMS – see this blog post: https://www.mongodb.com/blog/post/serverless- development-with-nodejs-aws-lambda-mongodb-atlas Container freeze & recycle? • Connection outside lambda function helps • On scale à new containers, new connections • Ok if API is used in bursts, but maybe not ok if used seldomly • If not? Others? Local development? à Lambda emulators • python-lambda-local at https://pypi.python.org/pypi/python-lambda-local • lambda-local (node.js) at https://www.npmjs.com/package/lambda-local Serverless frameworks – evaluate them! … F500’s are! • Serverless Framework • Zappa • Chalice • More! - https://thenewstack.io/tns-guide-serverless- technologies-best-frameworks-platforms-tools/
- 54. Façade à Serverless Functions – logic querying backend API Backend à Traditional stateful layer - CRUD API to Data Stores Would this be a Serverless Architecture ?? Customer Single View - Insurance Industry (hypothetical) High-level architecture of a single view platform Stateful API Service Layer
- 55. Thank you!