Extras

Extras while studying

Route 53

  • Record Types:
    • CNAME records can be used to map one name to another.
    • A records are used to point a domain or subdomain to an IP address
    • A Pointer (PTR) record resolves an IP address to a fully-qualified domain name (FQDN) as an opposite to what an A record does. PTR records are also called Reverse DNS records
    • Alias records let you route traffic to selected AWS resources, such as CloudFront distributions and Amazon S3 buckets. 3rd party websites do not qualify for these as we have no control over these.
  • Geoproximity routing lets Amazon Route 53 route traffic to your resources based on the geographic location of your users and your resources.

ECS

Amazon Elastic Container Service (Amazon ECS) is a highly scalable, high-performance container orchestration service that supports Docker containers and allows you to easily run and scale containerized applications on AWS. Amazon ECS eliminates the need for you to install and operate your own container orchestration software, manage and scale a cluster of virtual machines, or schedule containers on those virtual machines.

Amazon Elastic Container Service (Amazon ECS) is a highly scalable, fast, container management service that makes it easy to run, stop, and manage Docker containers on a cluster. You can host your cluster on a serverless infrastructure that is managed by Amazon ECS by launching your services or tasks using the Fargate launch type. For more control, you can host your tasks on a cluster of Amazon Elastic Compute Cloud (Amazon EC2) instances that you manage by using the EC2 launch type.

ECS Developer Guide

  • ECS_ENABLE_TASK_IAM_ROLE : Enables IAM roles for tasks for containers with the bridge and default network modes.
  • If you need to enable port mapping, you need to specify this as part of the container definition which is configured in the task definition.
  • Task group
    • set of related tasks
    • All tasks with the same task group anem are considered as a set when performing spread placement.
  • Task Placement Constraint
    • rule that is considered during task placement
    • Uses the cluster queries when placing tasks on container instances based on a specific expression.
      • It does not prpvide the actual expressions which are used to group those container instances
  • Task Placement Strategy
    • Algorithm for selecting instances for task placement or tasks for termination

ECS Task Placement Strategies

  • Binpack
    • places tasks based on the least amount of available CPU or memory. This minimizes the number of instances in use.
  • Random
    • place tasks randomly
  • Spread
    • Place tasks evenly based on the specified value. Accepted values are attribute key-value pairs, instanceID, or host

The Random task placement strategy is fairly straightforward as it doesn't require further parameters. The two other strategies, such as binpack and spread, take opposite actions. Binpack places tasks on as few instances as possible, helping to optimize resource itilization, while spread places tasks evenly across your cluster to help maximize availaability. By default, ECS uses spread with the ecs.availability-zone attribute to place tasks.

Random places tasks on instances at random yet still honors the other constraints that you specified, implicitly or explicitly. Specifically, it still makes sure that tasks are scheduled on instances with enough resources to run them.

ECS - IAM Permissions

By default, IAM users don't have permission to create or modify Amazon ECS resources, or perform tasks using the Amazon ECS API. This means that they also can't do so using the Amazon ECS console or the AWS CLI. To allow IAM users to create or modify resources and perform tasks, you must create IAM policies. Policies grant IAM users permissions to use specific resources and API actions. Then, attach those policies to the IAM users or groups that require those permissions.

When you attach a policy to a user or group of users, it allows or denies the users permission to perform the specified tasks on the specified resources. Likewise, Amazon ECS container instances make calls to the Amazon ECS and Amazon EC2 APIs on your behalf, so they need to authenticate with your credentials. This authentication is accomplished by creating an IAM role for your container instances and associating that role with your container instances when you launch them.

If you use an Elastic Load Balancing load balancer with your Amazon ECS services, calls to the Amazon EC2 and Elastic Load Balancing APIs are made on your behalf to register and deregister container instances with your load balancers

Fargate

  • AWS Fargate is a compute engine for Amazon ECS that allows you to run containers without having to manage servers or clusters. With AWS Fargate, you no longer have to provision, configure, and scale clusters of virtual machines to run containers. AWS Farget eliminates the need to manage a cluster of Amazon EC2 instances
    • If you don't want the hassle of managing the EC2 instances for Docker, we can use Fargate
    • Is serverless?

Cluster Query Language

When a task that uses the EC2 launch type is launched, Amazon ECS must determine where to place the task based on the requirements specified in the task definition, such as CPU and memory. Similarly, when you scale down the task count, Amazon ECS must determine which tasks to terminate. You can apply task placement strategies and constraints to customize how Amazon ECS places and terminates tasks. Task placement strategies and constraints are not supported for tasks using the Fargate launch type. By default, Fargate tasks are spread across Availability Zones.

Cluster queries are expressions that enable you to group projects. For example, you can group container instances by attributes such as Avalability Zone, instance type, or custom metadata. You can add custom metadata to your container instances, known as attributes. Each attribute has a name and an optional string value. You can use the built-in attributes provided by amazon ECS or define custom attributes.

After you have defined a group of container instances, you can customize Amazon ECS to place tasks on container instances based on group. Running tasks manually is ideal in certain situations. For example, suppose that you are develping a task but you are not ready to deploy the task with the service scheduler. Perhaps your task is a one-time or periodic batch job that does not make sense to keep running or restart when it finishes.

ECS Task Placement

Cluster Query Language

STS

  • "STS" : The AWS Security Token Service (STS) is a web service that enables you to request temporary, limited-privilege credentials for AWS Identity and Access Management (IAM) users or for users that you authenticate (federated users). However is not supported at the time with API Gateway.
  • Using sts decode-authorization-message, you are able to get more information about authorization exceptions happening with the CLI.
  • Credentials that are created by using account credentials can range from 900 seconds (15 minutes) up to a maximum of 3,600 seconds (1 hour), with a default of 1 hour. Hence you need to renew the credentials post expiry.
    • https://docs.aws.amazon.com/STS/latest/APIReference/API_GetSessionToken.html
  • Use decode-authorization-message to decode additional information about the authorization status of a request from an encoded message returned in response to an AWS request. If a user is not authorized to perform an action that was requested, the request returns a Client.UnauthorizedOperation response (an HTTP 403 response).
    • https://docs.aws.amazon.com/cli/latest/reference/sts/decode-authorization-message.html

STS API Calls:

  • GetSessionToken
    • Returns a set of temporary credentials for an AWS account or IAM user.
    • The credentials consist of an access key ID, a secret access key, and a security token
    • Tpyically, you use GetSessionToken if you want to use MFA to protect programmatic calls to specific AWS API operations like Amazon EC2 StopInstances
    • MFA-Enabled IAM users would need to call GetSessionToken and submit an MFA code that is associated with their MFA device
    • Using the temporary credentials returned from the call, IAM users can then make programmatic calls to API operations that require MFA authentication.
      • If you do not supply a correct MFA code, then the API returns an access denied error.
  • AssumeRoleWithWebIdentity
    • returns a set of temporray credentials for federated users who are authenticated through public identity providers sush as Amazon, Facebook, Google, or OpenID
      • Does not support MFA
  • AssumeRoleWithSAML
    • returns a set of temporary credentials for users who have been authenticated via a SAML authentication response
    • Provides a mechanism for tyin an enterprise identity store or directory to role-based AWS access without user-specific credentials or configuration
      • Does not support MFA
  • AssumeRole
    • creates a new session with temporary credentials
    • This session has the same permissions as the identity-based policies for that role

STS API Operations

Cognito Sync

  • Amazon Cognito Sync is an AWS service and client library that enables cross-device syncing of application-related user data. You can use it to synchronize user profile data across mobile devices and the web without requiring your own backend

Auto Scaling Groups

  • If you have a minimuim size of 1 and a max size of 5 EC2 launched across 3 AWS AZ, you can ensure you always have an EC2 instance running in any AZ by increasing the minimum instances to 2
    • This way, if an AZ goes down, you can spin up another EC2 instance in another AZ

Regions

  • If you don't select a region, then us-east-1 (US Virginia) will be used by default, which happens to be the first region AWS started in and where most of the AWS services reside.

SWF

  • Simple Work Flows
  • Last up to a year
  • Guaranteed to be processed only once and in the correct order
  • Humans can perform an activity task, but not a decision task
  • Use SWF for separating activity steps and allowing more control but increasing complexity
    • Use a step funciton for state management

SWF Options:

  • Markers
    • You can use markers to record events in the workflow execution history for application specific purposes. Markers are useful when you want to record custom information to help implement decider logic. For example, you could use a marker to count the number of loops in a recursive workflow.
  • Signals
    • Using signals enables you to inject information into a running workflow execution.
    • In some scenarios, you might want to add information to a running workflow execution to let it know that something has changed or to inform of an external event.
    • Any process can send a signal to an open workflow execution
      • for example, one workflow execution might signal another
  • Timers
    • Enables you to notify your decider when a certain amount of time has elapsed
  • Tags
    • Filter the listing of the executions when you use the visibility operations.

CodeStar

  • AWS CodeStar enables you to quickly develop, build, and deploy applications on AWS.
  • AWS CodeStar provides a unified user interface, enabling you to easily manage your software development activities in one place.
  • With AWS CodeStar, you can set up your entire continuous delivery toolchain in minutes, allowing you to start releasing code faster.
  • AWS CodeStar makes it easy for your whole team to work together securely, allowing you to easily manage access and add owners, contributors, and viewers to your projects.
  • Each AWS CodeStar project comes with a project management dashboard, including an integrated issue tracking capability powered by Atlassian JIRA Software.
  • With the AWS CodeStar project dashboard, you can easily track progress across your entire software development process, from your backlog of work items to teams’ recent code deployments.

Well Architected Framework

  1. Operational Excellence
  2. The operational excellence pillar includes the ability to run and monitor systems to deliver business value and to continually improve supporting processes and procedures.

Design Principles:

  • Perform operations as code
  • Annotate documentation
  • Make frequent, small, reversible changes
  • Refine operations procedures frequently
  • Anticipate failure

  • Learn from all operational failures

  • Security

  • Reliability
  • Performance Efficiency
  • Cost Optimization

Concept of Elasticity

In the traditional data center-based model of IT, once the infrastructure is deployed, it typically runs whtether it is needed or not, and all that capacity is paid for, regardless of how much it gets used. In the cloud, resources are elastic, meaning they can instantly grow or shrink to match the requirements of a specific application.

Elasticity allows you to match the supply of resources - which cost money - to demand. Because cloud resources are paid for based on usage, matching needs to itlization is critical for cost optimization. Demand includes both external usage, such as number of customers who visit a website over a given period, and internal usage, such as an application team using development and test environments.

There are two types of elasticity:

  1. Time Based
  2. Volume based

Time-based elasticity means turning off resources when they are not being used, such as a development environment that is needed only during business hours. Volume-based elasticity means matching scale to the intensity of demand, whether that's compute cores, storage sizes, or throughput. By combining monitoring, tagging, and automation, you can get the most value out of your AWS resources and optimize costs.

By taking advantage of volume-based elasticity, you can scale resources to match capacity. The best tool for accomplishing this task is Auto Scaling, which you can use to optimize performance by automatically increasing the number of EC2 instances during demand spikes and decreasing capacity during lulls to reduce costs. Auto scaling is well-suited for applications that have stable demand patterns and for ones that experience hourly, daily, or weekly variability in usage.

You can also use a combination of ELB and Auto scaling to maximize the elasticity of your architecture. Beyond Auto Scaling for Amazon EC2, you can use Application Auto Scaling to automatically scale resources for other AWS Services including:

  • Amazon ECS
  • Amazon EC2 Spot Fleets
  • Amazon EMR Clusters
  • Amazon AppStream 2.0 stacks and fleets
  • Amazon DynamoDB

For Amazon EC2 Spot Fleets, it can either launch instances (scale out) or terminate instances (scale in), within the range that you choose, in response to one or more scaling policies.

For Amazon DynamoDB, you can dynamically adjust provisioned throughput capacity in response to actual traffic patterns. Thjis enables a table or a global secondary index to increase its providionsed read and write capacity to handle sudden increases in traffic without throttling. When the workload decreases, Application Auto Scaling decreases the throughput so that you don't pay for unused provisioned capacity.

Systems Manager Parameter Store

AWS Systems Manager Parameter Store provides secure, hierarchical storage for configuration data management and secrets management. You can store data such as passwords, database strings, and license codes as parameter values. You can store values as plain text or encrypted data. You can then reference values by using the unique name that you specified when you created the parameter.

Parameter Store offers the following benefits and features:

  • Use a secure, scalable, hosted secrets management service (No servers to manage)
  • Improve your security posture by separating your data from your code
  • Store configuration data and secure strings in hierarchies and track versions
  • Control and audit access at granular levels
  • Configure change notifications and trigger automated actions
  • Tag parameters individually, and then secure access from different levels, including operational, parameter, Amazon EC2 tag, or path levels
  • Reference AWS Secrets Manager Secrets by using Parameter Store Parameters

Amazon QuickSight

Amazon Quicksight is a fast, cloud-powered business intelligence service that makes it easy to deliver insights to everyone in your organization. As a fully managed service, QuickSight lets you easily create and publish interactive dashboards that include ML insights. Dashboards can then be accessed from any device embedded into your applications, portals, and websites.

Amazon QuickSight has an ML Insights feature which leverages AWS's proven machine learning (ML) and natural language capabilities to help you gain deeper insights from your data. These powerful, out of the box features make it easy for anyone to discover hidden trends and outliers, identify key business drivers, and perform powerful what-if analysis and forecasting with no technical expertise or ML experience needed.

With ML Insights, you can accurately forecast your revenue with multiple levels of easonality (like sales with weekly and quarterly trends). QuickSight automatically excludes anomalies, like spikes due to a price drop, so they don't influence the predictions. QuickSight handles missing values so you don't have to worry about cleansing and repreparation your data beforehand. You can also perform interactive what-if analysis to determine the growth trajectory needed to meet business goals.

What is Amazon QuickSight

Glacier Retrieval

There are three options for retrieving data with varying access times and cost

  • Standard Retrievals
    • allow you to access any of your achives within several hours
    • Standard retrievals typically complete within 3-5 hours
    • This is the default option
  • Bulk Retrievals
    • Glacier's low cost retrieval option
    • Use to retrieve large amounts of data, even petabytes, inexpensively in a day
    • Bulk retrievals typically complete within 5-12 hours
  • Expedited retrievals
    • allow you to quickly access your data when occasional urgent requests for a subset of archives are required.
    • Typically made within 1-5 minutes

Certificate Management

To enable HTTPS connections to your website or application in AWS, you need an SSL/TLS server certificate. For certificates in a Region supported by AWS Certificate Manager (ACM), it is recommended that you use ACM to provision, manage, and deploy your server certificates. In unsupported Regions, you must use IAM as a certificate manager.

ACM is the preferred tool to provision, manage, and deploy your server certificates. With ACM, you can request a certificate or deploy an existing ACM or external certificate to AWS resources. Certificates provided by ACM are free and automatically renew. In a supported region, you can use ACM to manage server certificates from the console or pgorammatically.

Use IAM as a certificate manager only when you must support HTTPS connections in a Region that is not supported by ACM. IAM securely encrypts your private keys and stores the encrypted version in IAM SSL certificate storage. IAM supports deploying server certificates in all Regions, but you must obtain your certificate from an external provder for use with AWS. You cannot upload an ACM certificate to IAM. Additionally, you cannot manage your certificates from the IAM console.

If you got your certificate from a third-party CA, import the certificate into ACM or upload it to the IAM certificate store.

CloudHSM

AWS CloudHSM provides hardware security modules in AWS Cloud. A hardware security mdoule (HSM) is a computing device that processes cryptographic operations and provides secure storage for cryptographic keys.

When you use an HSM from AWS CloudHSM, you can perform a vaeriety of cryptographic tasks:

  • Generate, store, import, export, and manage cryptographic keys, including symmetric keys and asymmetric key pairs.
  • Use symmetric and asymmetric algorithms to encrypt and decrypt data.
  • Use cryptographic hash functions to compute message digests and hash-based message authentication codes (HMACs).
  • Cryptographically sign data (including code signing) and verify signatures.
  • Generate cryptographically secure random data

You should consider using AWS CloudHSM instead of AWS KMS if you require:

  • Keys stored in dedicated, third-party validated hardware security modules under your exclusive control.
  • FIPS 140-2 compliance
  • Integration with applications using PKCS#11, Java JCE, or Microsoft CNG interfaces
  • High-performance in-VPC cryptographic acceleration (bulk crypto)

Managing Keys in AWS CloudHSM

AWS CloudHSM FAQs

AWS WAF

AWS WAF is a web application firewall that lets you monitor the HTTP and HTTPS requests that are forwarded to an amazon API Gateway API, Amazon CloudFront or ALB. WAF also lets you control access to your content. Based on conditions that you specify, such as the IP addresses that requests originate from or the values of query strings, API Gateway, CloudFront or an ALB responds to requests either with the requested content or an HTTP 403 status code (Forbidden). You can also configure CloudFront to return a custom error page when a request is blocked.

At the simplest level, AWS WAF lets you choose one of the following behaviors:

  • Allow all requests except the ones that you specify
    • This is useful when you want CloudFront or an ALB to serve content for a public website, but you also want to block requests from attackers
  • Block all requests except the ones that you specify
    • This is useful when you want to serve content for a restricted website whose users are readily identifiable by properties in web requests, such as the IP addresses that they use to browse to the website
  • Count the requests that match the properties that you specify
    • When you want to allow or block requests based on new properties in web requests, you first can configure AWS WAF to count the requests that match those properties without allowing or blocking those requests. This lets you confirm that you didn't accidentally configure AWS WAF to block all the traffic to your website. When you're confident that you specified the correct properties, you can change the behavior to allow or block requests.

AWS Secrets Manager

AWS Secrets manager is an AWS service that makes it easier for you to manage secrets. Secrets can be database credentials, passwords, third party AIP keys, and even arbitrary text. You can store and control access to these secrets centrally by using the Secrets manager console, the Secrets Manager CLI, or the Secrets Manager API and SDKs.

In the past, when you created a custom application that retrieves information from a database, you typically had to embed the credentials for accessing the database directly in the application. When it came time to rotate the credentials, you had to do much more than just create new credentials. You had to invest time to update the application to use the new credentials. Then you had to distribute the updated application. If you had multiple applications that shared credentials and you missed updating one of them, the application would break. Because of this risk, many customers have chosen not to regularly rotate their credentials, which effectively substitutes one risk for another.

Secrets manager enables you to replace hardcoded credentials in your code (including passwords), with an API call to Secrets Manager to retrieve the secret programmatically. This helps ensure that the secret can't be compromised by someone examining your code, because the secret simply isn't there. Also, you can configure Secrets Manager to autmatically rotate the secret for you according to a schedule that you specify. This enables you to replace long-term secrets with short-term ones, which helps to significantly reduce the risk of compromise.

AWS App Sync

AWS App Sync simplifies application devlopment by letting you create a flexible API to securely access, manipulate, and combine data from one or more data sources. AppSync is a managed service that uses GraphQL to make it easy for applications to get exactly the data they need.

With AppSync, you can build scalable applicatoins, including those requiring real-time updates. on a range of data sources such as NoSQL data stores, relational databases, HTTP APIs, and your custom data sources with AWS Lambda. For mobile and web apps, AppSync additionally provides local data access when devices go offline, and data synchronization with custmizable conflict resoluation, when they are back online.

AWS AppSync is quite similar to Cognito Sync which is also a service for synchronizing application data across devices. It enables user data like app preferences or game state to be synchronized as well. However, the key difference is that AppSync also extends these capabilities by allowing multiple users to synchronize and collaborate in real time on shared data.

Consolidated Billing

The consolodated billing feature in AWS Organizations allows you to consolodate payment for multiple AWS accounts for multiple AISPL accounts. Each organization in AWS organizations has a master account that pays the charges for all the member accounts. If you have access to the master account, you can see a combined view of the AWS charges that are incurred by the member accounts. You can also get a cost report for each member account.