aws.markdown

AWS

• Five Pillars
• Core Concepts
• IAM
• Cognito
  • User Pool Authentication Flow
  • Tokens
• Signing Requests
• VPC (Virtual Private Cloud)
  • Network ACLs vs. Security Groups
  • TODO: Private vs. Public subnets
• EC2
  • EC2 Storage
• Glacier
• RDS
• DynamoDB
• Aurora
• API Gateway
  • CORS
• Developer Tools

Five Pillars

1. Operational Excellence

   • Infrastructure as Code

     • CloudFormation
     • CDK

   • Observability

     • Collection:
       • Infrastructure-level: CloudWatch
       • Application-level: CloudWatch Custom Metrics
       • Account-level: CloudTrail
     • Analytics: CloudWatch Logs Insight, Athena, RDS, RedShift, Elasticsearch Service
     • Action: CloudWatch Alarms/Dashboards

2. Security: Zero Trust, Principle of least privilege

   • IAM
   • Network security: VPC, ALB, WAF, Security Group
   • Data Encryption
     • In transit: HTTPS, ALB
     • At rest: KMS, CMK, CloudTrail

3. Reliability

   Minimize the blast radius of any individual component

   • Fault Isolation
     • Resource and Request
     • Availability Zone
     • Region
   • Limits
     • Service Quotas

4. Performance Efficiency

   • Selection
     • Compute
     • Storage
     • Database
     • Network
   • Scaling
     • Vertical
     • Horizontal

5. Cost Optimization

   OpEx vs. CapEx

   • Pay for Use
     • Right Sizing
     • Serverless
     • Reservations
     • Spot Instances: for fault-tolerent workloads
   • Cost Optimization Lifecycle

Core Concepts

• Regions

  Geographical locations, such as US West (N. California) Asia Pacific (Sydney)

• Availability Zones

  Each region has multiple availability zones, each zone means an isolated infrastructure (separate power grid, etc), but a single zone can span multiple data centers;

IAM

Identity and Access Management

• The account you created when signing up AWS is the root account, you should avoid using root account as much as possible;
• You can create multiple IAM users, which can be used to login to AWS console, or access AWS resources through CLI, SDK, API;
• Access control is specified by policies;
• Policies can be attached to IAM users, groups or roles;
• IAM Roles are intended to be used temporarily, it can be granted to
  • IAM users in another account;
  • code running on a EC2 instance;
  • or an AWS service that needs access to resources in your account, eg. when you create an Cloud9 environment, an IAM role "AWSServiceRoleForAWSCloud9" will be created, it allows Cloud9 to call other AWS services for you

Cognito

User Pool Authentication Flow

• The flow in the above image is generalized to two steps: InitiateAuth and RespondtoAuthChallenge;
• The 'Challenge' can be more than one, and can be customized by AWS Lambda triggers;

On client Side

• User enters their username and password to the app;
• The app calls the InitiateAuth method with the user's username and SRP details (generated by a SDK);
• The app calls the RespondToAuthChallenge method:
  • if the call succeeds, it returns the users tokens and the authentication flow is completed;
  • if another challenge is required, no tokens are returned, instead a session is returned;
• If a session is returned in the last step, the app calls RespondToAuthChallenge again, with the session and the challenge response (e.g. MFA code);

Tokens

After successful authentication of a user, Cognito issues three tokens to the client:

• ID Token

  • It is represented as a JSON Web Token (JWT), contains identity claims of the user, such as name, family_name, phone_number, etc, which can be used in the app;
  • Can be used to authenticate users against your resource servers or server applications;
  • When used outside of the application against your web APIs, you must verify the signature of the ID token before you can trust any claims inside the token;
  • ID tokens expires one hour after the user authenticates;
  • Structure of an ID token:
    • Header: kid, alg
    • Payload: iss, sub, aud, token_use, auth_time
    • Signature: used for verification

• Access Token

  • Is also a JWT, does not include all of the user's identity info;
  • Primary purpose of the access token is to authorize operations in the context of the user in the user pool, e.g. use it to update or delete user attributes;
  • Can also be used with any of your web APIs to make access control decisions and authorize operations in the context of the user;
  • Expires in one hour;

• Refresh Token

  • Used to retrieve new tokens;
  • By default, the refresh token expire 30 days after the user authenticates;
  • Mobile SDKs refresh tokens automatically;

Signing Requests

Most request to AWS need signing;

• If you use AWS CLI or any AWS SDK, then the signing is done automatically;

• If you are using a custom solution, you need to do the signing yourself;

  • Signing is done by adding a signature to HTTP headers or url parameters;
  • The signature is a hash generated by using the access keys (including Access Key ID and Secret Access Key), and some request info;

VPC (Virtual Private Cloud)

This is a basic feature of AWS, a lot of other services (EC2, RDS, S3, etc) are deployed into VPC and are protected by it.

• A private, virtual network in the AWS Cloud (similar to an on-premise network);
• Complete control (isolate and expose resources in VPC);
• Several layers of security controls (allow and deny specific internet and internal traffic);
• Other services deployed into VPC inherit security of the VPC;

Features:

• VPC lives within a Region, can span multiple AZs
• Subnets: used to divide a VPC, each subnet lives within an AZ;
• Route tables: control traffic going out of the subnets;
• Internet Gateway (IGW): allow access to the Internet from a VPC;
• NAT Gateway: allow private subnet resources to access Internet;
• Network Access Control List (NACL): control access to subnets;

Basic Network Knowledge:

• Private IP ranges:

  • 10.0.0.0/8: 10.0.0.0 to 10.255.255.255
  • 172.16.0.0/12: 172.16.0.0 to 172.31.255.255
  • 192.168.0.0/16: 192.168.0.0 to 192.168.255.255

• Other special IPs:

  • Loopback: 127.0.0.0/8, 127.0.0.0 to 127.255.255.255
  • Not used: 0.0.0.0/8

Network ACLs vs. Security Groups

These two should be used together to implement network security rules, ACLs apply to subnet and security groups apply to individual instances;

	Network ACLs	Security Groups
Applies to	Subnets	Individual instances (EC2, RDS, etc)
Default	"Allow All" for both in/out	"Deny All" inbound
Settings	Can specify both allow and deny rules	Can only specify allow rules
Attributes	Stateless (you have to manually edit both inbound/outbound rules)	Stateful (if you open inbound port 80, it will open outbound port 80 automatically)

TODO: Private vs. Public subnets

EC2

Steps to launch an EC2 instance:

• Select an image (AMI), which determines how many CPUs and Memory are available;
• Configure networking (VPC) and IAM Role;
• Add storage;
• Add Tags;
• Configure Security Group (firewall rules);

EC2 Storage

Storage options available:

• EBS (Elastic Block Store) (like a portable disk)

  • Durable, block-level storage, suitable for data that requires frequent and granular updates, e.g. running a database;

  • Independent of EC2 instances, can be attached and detached any time;

  • Can be created as encrypted volumes;

  • Can create a snapshot of an EBS volume, which is stored in Amazon S3, then you can create an EBS volume from a snapshot, and attach it to another instance;

  • Create a volume:

    # Create a filesystem
    mke2fs /dev/xvdb
    # Mount it
    mount /dev/xvdb /mnt
    # Unmount
    unmount /mnt

• EC2 Instance Store (like a hard disk in the case)

  • Storage from disks that are physically attached to the host computer;
  • Data only persists during the life of the associated instance;

• EFS (Elastic File System) (like a remote network disk)

  • Scalable, can be mounted to the file system;
  • Accessible to multiple EC2 instances at the same time, unlike EBS, which is only accessible to one instance at a given time;

• S3 (like dropbox)

  • Data is available through an Internet API;
  • Not attached to an EC2 Instance, you can access it anywhere;
  • Can be used to store EBS snapshots and instance store-backed AMIs;

Glacier

Data archive solution

• Long-term storage at low-cost;
• 99.9999% durability;
• Suitable for data that only need infrequent access;

RDS

The DB program can be: MySQL, Amazon Aurora, MariaDB, PosgreSQL, SQL Server, Oracle

• Usually a DB instance is put in a private subnet, only accessible to the app;
• You can configure a slave db in another subnet/AZ;
• You can also configure read replicas to offload the master instance;

DynamoDB

Managed NoSQL database, that means you can add attributes to any data without schema changes.

Two types of query:

• QUERY: search by key, it can locate a partition easily, it's efficient;
• SCAN: search items by attribute, which scans each row, less efficient;

The table's primary key is made up of one or two attributes that uniquely identify items, it includes the partition key and optionally a sort key:

Aurora

Amazon Aurora is a MySQL-compatible enterprise-class database. Aurora supports up to 64TB of auto-scaling storage capacity, 6-way replication across three availability zones, and 15 low-latency read replicas.

API Gateway

API Gateway can be used to integrate with Lambda Functions, HTTP endpoints, or a Mock service;

Access Control

• IAM Permissions

  API Gateway has two components, each has its own permission model:

  • Management component: create, deploy, manage an API;

    IAM permissions policies need to be attached to the user;

  • Execution component: call a deployed API or to refresh the API caching;

    To allow an API caller to call an API method, you need to do two things:

    • Create IAM policies that permit the API method to be invoked by this API caller;
    • Set the API method's authorizationType property to AWS_IAM;

  When an API is integrated with an AWS service (e.g. AWS Lambda) in the back end, API Gateway must also have permission to access the integrated AWS service on behalf of the API caller. To grant these permissions,

  1. Create an IAM role of the AWS service for API Gateway type (looks like it should be an AWS managed role with the name AWSServiceRoleForAPIGateway), API Gateway is the trusted entity of this role;
  2. Attach to this role appropriate IAM permissions policies for calling integrated AWS services;

• Amazon Cognito User Pools

  To enable this:

  • Create an authorizer of the COGNITO_USER_POOLS type and configure an API method to use that authorizer;
  • The API client must first sign the user in to the user pool, obtain an identity or access token for the user;
  • Then the client call the API method with one of the tokens, typically set to the request's Authorization header;

CORS

• Add an OPTIONS method to your endpoint, respond with at least the following headers:

  • Access-Control-Allow-Methods
  • Access-Control-Allow-Headers
  • Access-Control-Allow-Origin

• The actual CORS enabled methods must also return the Access-Control-Allow-Origin: 'request-originating server addresses' header in at least its 200 response, if you are using a proxy integration, your backend must return this header;

• CORS can be enabled for resource or just a method under the resource;

• CORS can be enabled automatically by a menu action in the console, so you do not need to do the above steps by yourself;

Developer Tools

• CodeStar - a dashboard that integrates CodeCommit, CodeBuild, CodeDeploy, CodePipeline;
• CodeCommit - git repo hosting, like GitHub;
• CodeBuild - auto build and test;
• CodeDeploy - code deploy;
• CodePipeline - the whole CI/CD process;
• Cloud9 - online IDE;