AWS Quick Intro

AWS Quick Intro

• 1.
  AWS for scientificcomputing jss 2011-05-12
• 2.
  TOC • Overview: Cloud Eco System • Amazon Web Services for scientific computing • EC2 Deployment • S3, EBS and local Storage • Queueing / Messaging
• 3.
  Cloud Layers Software asa Service - SaaS running applications Platform as a Service - PaaS software environments and services Infrastructure as a Service - IaaS virtual machines and network
• 4.
  Elastic Compute Cloud(EC2) • Virtual Data Center • Instance Types • Regions • Availability Zones • Security Groups (Firewalling) Can go down just like anyone else • Amazon Machine Images (AMI)
• 5.
  EC2 Setup • Management: • AWS web console • CLI: ec2-tools • Python: boto (check dev/src/tba) • Use the setup script • Get SSH fingerprint from instance’s boot console • Use some form of parallel SSH for additional setup (Twisted conch works great, fabric for serial deployment)
• 6.
  The DNS Problem • Instances get new IP on boot • Solutions: • use Virtual Private Cloud (Amazon VPC) • auto-update hosts file • set up DNS server
• 7.
  Storage • Instance fast, cleared on boot, no charges • Elastic Block Storage (EBS) fast, networked, instant snapshot-able, N attachments per instance • Simple Storage Service (S3) HTTP access, CDN (CloudFront)
• 8.
  Messaging • Simple Queueing Service (Amazon SQS) • AMQP / RabbitMQ (c.f. knk’s recent rocketcircus) • Rolling your own (e.g. XML RPC) is fine, too
• 9.
  TLDR AWS for theoccasional number crunch: • 1/2 day to get basic setup running (longer for robust setups) • profit