EC2 101

  • Elastic Compute Cloud
  • Web service that provides resizable compute capacity in the cloud
    • Virtual machines in the cloud
  • Reduces the time required to obtain and boot new servers in minutes
    • Quickly scale capacity up and down as requirements change
    • Can add more instances behind a load balancer
    • Provision VMs in cloud in minutes
  • Pay for only capacity you use
    • No longer have to spend capital up front for traditional servers
    • Get exactly how much you need
  • Provides developers tools to build resilient apps and isolate themselves from common failure Scenarios

EC2 Pricing Options

  • On Demand
    • Allows you to pay a fixed rate by the hour (or by the second) with no commitment
    • Linux - by the second
    • Windows - by the hour
  • Reserved
    • Provides capacity reservation and offer a significant discount on the hourly charge for an instance
      • 1 or 3 year terms
  • Spot
    • Bid whatever price you want for instance capacity providing for even greater savings if your apps have flexible start and end times
  • Dedicated Hosts
    • Physical EC2 server dedicated for your use
    • Help reduce costs by allowing you to use your existing server-bound software licenses

On Demand

  • Users want the low cost and flexibility of Amazon EC2 without any up-front payment or long-term commitment
  • Apps with short term, spiky, or unpredictable workloads that can be interrupted
  • Apps being developed or tested on Amazon EC2 for the first time

Reserved

  • Apps with steady state or predictable usage
  • Apps that require reserved capacity
  • users can make up-front payments to reduce their total computing costs even further
    • If you pay for it all up front for 3 year contract, you get a huge discount.
    • Standard Reserved Instances (up to 75% off on-demand)
    • Convertible Reserved Instances (Up to 54% off on-demand) feature the capability to ch ange the attributes of the RI as long as the exchange results in the creation of a reserved instance of equal or greater value
    • Scheduled RIs are available to launch within the time window you reserve
      • Allows you to match your capacity reservation to a predictable recurring schedule that only requires a fraction of a day, week, or month.
      • Month end sale at the end of every month on a Friday
        • When that friday is over, simply scale back

Spot Instances

  • Apps that have flexible start and end times
  • Apps that are only feasible at very low compute prices
    • Genome
    • Pharmaceuticals
    • Chemical companies
    • Huge amounts of computing at 4 am on a sunday morning to save money
  • Urgent needs for a large amount of computing capacity

Dedicated Hosts

  • Regulatory requirements that may not support multi-tenant virtualization
    • Health field can't use multi-tenant virtualization
    • May expose data to someone else
  • Great for licensing which does not support multi-tenancy or cloud deployments
  • Purchased on demand (hourly)
  • Purchased as a reservation for up to 70% off the on-demand price

EC2 Instance Types

Note: Remember letters, not numbers

  • FIGHTDRMCPX
  • F - Field Programmable Gate Array
    • Genomics research, financial analytics, real-time video processing, big data, etc
  • I - High Speed Storage (IOPS)
    • NoSQL Databases, Data Warehousing, etc
  • G - Graphics Intensive
    • Video Encoding, 3D Application Streaming
  • H - High Disk Throughput
    • MapReduce based workloads, distributed file systems suich as HDFS and MapR-FS
  • T - Lowest Cost, General Purpose
    • Web Servers, Small Databases
  • D - Dense Storage
    • Fileservers, Data Warehousing, Hadoop
  • R - Memory optimized (RAM)
    • Memory intensive Apps/Databases
  • M - Main choice for general purpose apps
    • Application servers
  • C - Compute Optimized
    • CPU Intensive Apps/Databases
  • P - Graphics/General Purpose GPU (Pics)
    • Machine learning, Bit Coin Mining
  • X - EXtreme Memory (Memory Optimized)
    • SAP HANA, Apache Spark, etc

What is EBS?

  • Elastic Block Storage
    • Virtual Disk
  • Create storage volumes and attach them to an EC2 Instance
    • Done with the "AttachVolume" API Call
  • Once attached, you can create a file system on top of these volumes, run a database, or use them in any other way you would use a block device
  • EBS Volumes are placed in a specific AZ where they are automatically replicated to protect you from the failure of a single component
    • Doesn't exist on one physical disk
    • Spread across AZ incase of disk failure
    • Disk in cloud you attach to EC2
    • EBS attached to EC2 where OS is installed is the root volume
  • AZ Specific, not region specific

You can detach an Amazon EBS volume from an instance explicitly or by terminating the instance. However, if the instance is running, you must first unmount the volume from the instance.

If an EBS volume is the root device of an instance (EBS Backed), you must stop the instance before you can detach the volume.

Detaching an Amazon EBS Volume from an Instance

Creating an Instance with API Commands

For EBS-backed instances:

  • CreateImage creates and reigsters the AMI in a single request, so you don't have to register the AMI yourself

For non-EBS backed instances

  • RegisterImage
    • When you're creating an AMI, this is the final step you must complete before you can launch an instance from the AMI

EBS Volume Types

General Purpose SSD (GP2)

  • General purpose, balances both price and performance
  • Ratio of 3 IOPS per GM with up to 10k IOPS and the ability to burst up to 3k IOPS for extended periods of time for volumes at 3334 GiB and above
  • EXAM: Less than 10k IOPS = GP2 because of best performance and best price

Provisioned IOPS SSD (IO1)

  • Designed for IO intensive applications such as large relational or NoSQL databases
  • Use if you need more than 10k IOPS
  • EXAM: Really big NoSQL Database will use a lot of IOPS - Use Provisioned IOPS SSD
  • Can provision up to 20k IOPS per volume

Throughput Optimized HDD (ST1)

  • Big Data
  • Data Warehouses
  • Log processing
  • Cannot be Boot Volume (where OS is installed)
  • Can't be C Drive, Must be D Drive

Cold HDD (SC1)

  • Lowest cost storage for infrequently accessed workloads
  • Cannot be Boot Volume

Magnetic (Standard)

  • Bootable
  • Lowest cost per gigabyte of all EBS volume types that is bootable
  • Ideal for workloads where data is accessed infreuquently and apps where lowest storage cost is important
  • Legacy

EC2 Exam Tips

  • On Demand
    • Allows you to pay a fixed rate by the hour (or by the second) with no commitment
  • Reserved
    • Provides you with a capacity reservation, and offers a significant discount on the hourly charge for an instance
    • 1 or 3 year term contract
      • Longer contract more upfront more discount
  • Spot
    • Bid whatever price you want for instance capacity, providing greater savings if your apps have flexible start and end times
      • Bid a rate, say $100. When the spot price hits it, the instance is provisioned. When the price goes up above the rate, the instances are terminated

  • If a spot instance is terminated by EC2, you will not be charged for a partial hour of use.

    • However, if you terminate the instance yourself, you will be charged for the complete hour in which the instance ran.

  • Dedicated Hosts

    • Physical EC2 server dedicated to your use
    • Reduce costs by allowing you to use your existing server-bound software licenses
    • If rules require no multi-tenant virtualization
  • FIGHT DR MC PX
  • SSD (EBS Volumes)
    • General Purpose SSD
      • Balances price and performance for a wide variety of workloads
    • Provisioned IOPS SSD
      • Over 10k IOPS use this, below use general purpose.
  • Magnetic
    • Throughput Optimised HDD
      • Low cost HDD volume designed for frequently accessed, throughput-intensive workloads
    • Cold HDD
      • Lowest cost HDD volume designed for less frequently accessed workloads
    • Magnetic
      • Previous generation
      • Can be a boot volume

Instace Profiles and Profiles

Using roles to grant permissions to applications that run on EC2 instances requires a bit of extra configuration. An appliucation running on an EC2 instance is abstracted from AWS by the virtualized operating system. Because of this extra separation, an additional step is needed to assign an AWS role and its associated permissions to an EC2 instance and make them available to its applications.

This extra step is the creation of an instance profile that is attached to the instance. The instance profile contains the role and can provide the role's temporary credentials to an application that runs on the instance. Those temporary credentials can then be used in the application's API calls to access resources and to limit access to only those resources that the role specifies. Note that only one role can be assigned to an EC2 instance at a time, and all applications on the instance share the same role and permissions.

Using roles in this way has several benefits. Because role credentials are temporary and rotated automatically, you don't have to manage credentials, and you don't have to worry about long-term security risks. In addition, if you use a single role for multiple instances, you can make a change to that one role and the change is propagated autmatically to all the instances.

Imagine that you have an IAM user for working in the development environment and you occasionally need to work with the production environment at the command line with the AWS CLI. You already have an access key credential set avaialbe to you. This can be the access key pair that is assigned to your standard IAM user. Or, if you signed in as a federated user, it can be the access key pair for the role that was initially assigned to you. If your current permissions grant you the ability to assume a specific IAM role, then you can identify that role in a "profile" in the AWS CLI configuration files. That command is then run with the permissions of the specified IAM role, not the original identity.

Note that when you specify that profile in an AWS CLI command, you are using the new role. In this situnation, you cannot make use of your original permissions in the development account at the same time. The reason is that only one set of permissions can be in effect at a time.

Note: an instance profile is just a container for an IAM role that you can use to pass role information to an EC2 instance when the instance starts. This is different from an AWS CLI profile, which you can use for switching to various profiles. In addition, an instance profile is associated with the instance and not configured in the AWS CLI.

So if we have an EC2 instance with multiple environments, we may need different profiles for different environments (i.e. dev, stage, prod) that each have different permissions. Instace profile gives a role to the entire instance. CLI Profiles can be assumed to get different permissions, and only one is in effect at a time.

Extras

  • To view all categories of instance metadata from within a running instance, use the following URI:http://169.254.169.254/latest/meta-data/
  • When you create an encrypted EBS volume and attach it to a supported instance type, the following types of data are encrypted:
    • Data at rest inside the volume
    • All data moving between the volume and the instance
    • All snapshots created from the volume
    • All volumes created from those snapshots.
  • EBS Encryption
    • A snapshot of an encrypted volume is always encrypted
    • Restoring a Volume from an encrypted snapshot must be an encrypted volume
  • Amazon's SLA guarantees a monthly uptime percentage of at least 99.95% for Amazon EC2 and Amazon EBS within a region
  • You can sell reserved instances you no longer need on the Reserved Instance marketplace
  • If you want to use existing Windows Server licenses with EC2, you MUST use a Dedicated Host
  • When you launch an Instance in EC2, you can pass in data (or scripts) as user data
    • this will get run when the instance launches
  • EBS volumes support encryption in transit, but not at rest.