Elastic Transcoder is a media Transcoder in cloud.
Converts the original media files into different formats which will be compatible with smartphones, tablets and PCs. It optimizes the source media file.
Provides transcoding presets for popular output formats, which means we don’t need to worry about which setting works best for devices.
Pay based on the minutes and the resolution of the media which is transcoded.
How it works,
Upload the media file in the S3 bucket, Pass it to the lambda function, lambda function will invoke Elastic Transcoder.
Elastic Transcoders will then further process and convert it to different media format.
Then it is moved back to S3, it also creates thumbnail of the media file.
API caching will cache the endpoint response. Thus, reducing the number of requests.
Feature of API Cache
Its low cost and Efficient
Scales Effortlessly and automatically.
Increase performance.
You can throttle requests to prevent requests.
Connects to cloud watch to log all requests.
Same origin policy
It is an important web application security model, under this policy a web browser permits scripts contained in a first web page to access data in a second web page, only if both web pages have the same origin (Which means same domain name).
CORS (Cross-Origin Resource Sharing)
CORS is on the way where the server at the other end (Not the client code in the browser) can relax the same origin policy.
This policy is a mechanism that allows restricted resources (e.g. font) on a web page to be requested from another domain outside the domain from which the first resource was served.
If you are using JavaScript/AJAX that uses multiple domains with API Gateway, ensure that you have enabled CORS on API Gateway.
The flow of the API Gateway or how it is implemented.
ECS is a container orchestration service that supports Docker and Windows containers. It has deep integration with AWS services IAM, VPC, and Route53.
ECS is a fully managed clustered platform that allows us to run the Docker images in the cloud.
The container is more like a virtual operating environment.
It has a standardized unit with everything the software needs to be run
It is also well suited for microservice, applications are created independently with stateless components
The architecture of Docker Container
Advantages of Container and Micro Services
Highly Scalable, we can scale only those resources which have max out, no need to scale out the whole application infrastructure.
Fault Tolerance, Error on one component doesn’t impact the other components.
Easy to maintain, Easier to maintain compared to the monolithic applications.
Fargate or EC2
We have the option to choose the container service,
The cluster of the virtual machines, ECS will allow us to run the container on the cluster of virtual machines. It provides you the control over installation, configuration and management of your environment.
Fargate can be used for serverless container where you don’t need to worry about the underlying EC2 instance.
ECR – ECR stores the container images, which will be used by the Docker.
ECS Application
ECS is used by AWS Sagemaker, LEX and Amazon.com which use ECS for the recommendation engine.
To prepare your application to run on Amazon ECS, you must create a task definition.
The task definition is a text file (in JSON format) that describes one or more containers (up to a maximum of ten) that form your application.
It specifies various parameters for your application. For example, these parameters can be used to indicate which containers should be used, which ports should be opened for your application, and what data volumes should be used with the containers in the task.
An Amazon ECScluster is a logical grouping of tasks or services.
Elastic Cloud Compute is a virtual server based on Windows /Linux Platform in AWS.
Amazon Elastic Compute Cloud has reduced the time to get the server instance up and boot in minutes. This allows us to quickly scale up/down the capacity as per the computing requirements.
EC2 Types
On Demand – This allows you to pay fixed rate by hours/Seconds with no upfront commitments. This not available in seconds for windows servers.
This is perfect for users that want low cost and flexibility of EC2. Without any upfront payments or commitments.
Applications with short term, spiky or unpredictable loads that cannot be interrupted.
Application developed and tested on EC2 for the first time.
Reserved – Provides with capacity reservations and offers discount on the hourly charge for an instance, 1to 3-year terms
Application with steady state or predictable usage.
Application that requires reserve capacity.
User can pay upfront to reduce the total computing cost further down.
Types of RI
Standard RIs – Up to 75% off on-demand RI.
Convertible RIs – Up to 54% off on-demand RI. It allows you to change the attributes of the RI if the exchange results in creation of Reserved Instance of equal or greater value.
Scheduled RIs – These are time bound RIs, it will be available during the reserve time window. This option allows you to match your capacity reservation to a predictable recurring schedule that only requires a fraction of a day, a week, or a month.
Spot – Enables you to Bid whatever price you want for instance capacity, providing for even greater saving if the applications have flexible start and end time (No Peak Hour).
Application having flexible end and start time.
Applications those are only feasible at very low computing price.
Users with an urgent need for large amount of additional computing capacity.
If spot instance is terminated by Amazon, you will not be charged for the partial hour. But if you terminate the instance, you will be charged for that complete hour.
Dedicated Host – Physical EC2 server dedicated for your use. Dedicated Hosts can help you to save the cost by allowing using the existing server bound software.
Useful for regulatory requirement that doesn’t support multi-tenant or cloud deployments
Great for licensing which doesn’t support multi-tenant or cloud deployments.
Can be purchased on-Demand (Hourly)
Can be purchased as reservation for up to 70% off the On-Demand.
AmazonEBS allows you to create storage volume and attach them to Amazon EC2 Instance. Once attach you can create file system on top of these volumes, you can run them as database or use them in any other way you would use a block device.
Amazon EBS is placed in a specific AZ, where they are automatically replicated to protect you from failure of a single component.
EBS volume types
General Purpose SSD (GP2)
General purpose balances both price and performance.
Ratio of 3 IOPS per 3 GB with up to 10000 IOPS and the ability to burst up to 3000 IOPS for extended period for volumes at 3334 Gib and above.
Provisioned IOPS SSD (101)
Designed for I/O Intensive applications such as large relational or NoSQL database.
Use if you need more than 10,000 IOPS
Can provision up to 20000 IOPS per Volume.
Throughput optimized HDD (ST1)
These cannot be a boot volume, these are additional volumes attached to the root volume.
Big data, data warehousing, log processing
Cold HDD (SC1)
Lowest cost storage for infrequent access workload.
File server.
Magnetic (Standard)
Lowest cost per GB of all EBS volume Type that is a bootable.
Magnetic volume is ideal where the work load were data are accessed infrequently and application where the lowest cost is important.
Each subnet is assigned to one AZ, you cannot have more than 1 subnets per AZ. One Subnet = One AZ
AMI(Amazon Machine Images) – are snapshots of different flavor virtual machines.
On EC2 Create page, advance section we can add boot strap scripts. It may contain the Unix Bash commands.
On Add Storage page, we can add the root EBS volume as well as the additional volumes. In root Volume we can add only General Purpose (GP2), Provisioned IOPS SSD (IO1) and Magnetic.
These root volumes are used to load the OS.
Volumes are nothing but virtual HDD.
Delete on termination if checked will delete the volume assigned upon termination of the EC2 Instance.
Security Groups are Virtual files. It defines the traffic allowed from your web servers, like which port/protocol is allowed or denied for the instance.
0.0.0.0/0 will allow access to all.
Public key and private key pair will be shared upon creation of EC2 Instance. Public key is your Padlock and private key is the key to Pad Lock.
These key pair needs to be downloaded and saved, as it will not be shown later. You can regenerate the keys.
Encryption can be done only for the attached volume, we cannot encrypt the Root volume device.
The EBS root volume can be encrypted using the third-party tool like Bit Locker. This can be achieved while creating the Amazon AMI or through the API.
Login through SSH terminal for EC2 Instance,
To connect through the Windows system to AWS EC2 instance, we need to use Putty and Puttykeygen.
The Private and Public key generated by AWS EC2 are in pem extension format, and for putty we need ppk, here we use putty keygen tool to convert the. pem file to. ppk file.
To connect to putty, use IP address (Public IP address) of your instance as hostname
Windows 10 provides SSH access through CMD, you can directly run the below command to SSH your EC2 instance using Pem file.
ssh -i <path to your .pem file> ec2-user@<your EC2 Public IP address>
In case of ubuntu as OS, replace ec2-user with ubuntu@<your EC2 Public IP address>
Upgrading EBS volume Type
You cannot have the EC2 instance in one availability zone and the EBS Volume in another availability zone. Both should be in the same AZ.
We cannot modify the volume for StandardVolume Type, standard is pure magnetic volume. All other volume type can be modified.
To change the AZ for any volume, we will need to create a snapshot first for that EC2 instance and then create a volume for the snapshot. While creating volume for the snapshot, you can change the volumetype as well as the AZ for the new volume.
Snapshot also allows the EC2 instance to move from one region to another one. Create a snapshot and then move the snapshot to the new region.
Through snapshot we can create images. These images are stored under AMI.
To delete the AMI, select the images and click on deregister.
Volume vs Snapshot – Security
Snapshots of Encrypted volumes are Encrypted automatically.
Volumes restored from encrypted snapshots are encrypted automatically.
You can share snapshots but only when they are un-encrypted.
These snapshots can be shared with other AWS account or can be made public.
Additional Details
Volumes Exist on EBS
Volumes are nothing but Virtual HDD
Snapshot exists on S3, they are point in time copy of the volumes.
Snapshots are incremental – That means only those blocks that have changed since your last snapshot are moved to S3.
First snapshot takes time to load, as it will capture all data.
To create a snapshot for the EBS volumes that serve as a root device, you should stop the instance before taking the snapshot.
However, you can take snaps while instance is running.
You can create AMIs from the EBS-backed instance and snapshots.
You can change the EBS volume sizes on the fly, including changing the size and volume type.
To move an EC2 Volume from one AZ/Region to another, take a snap or an image of it, then copy it to the new AZ/Region.
AWS offers three different types of load balancers in AWS, they are as below,
Application Load Balancer
Network Load Balancer
Classic Load Balancer
Application Load Balancer – They are best suited for load balancing the HTTP and HTTPS traffic. They operate at layer 7 (TCP layers) and are application-aware. They are intelligent and can create advance request routing, sending specific requests to specific web servers.
Note –
One subnet is equal to one availability zone.
You will need to register the target (Load balancer) to the instance to assign it instance.
Network Load Balancer – They are required for load balancing where extreme performance is required. They operate at layer 4. They are capable of handling millions of requests per second while maintaining ultra-low latency.
Classic Load Balancer – They are legacy Elastic load balancers. It can be used for load balancing HTTP/HTTPS applications and it uses layer 7 specific features, such as X-forwarder and sticky sessions. You can use strict layer 4 load balancing for applications that purely rely on the TCP protocol.
Classic load balancer error 504, if your application throws error code 504 (Gateway Timeout). This means the application may have issues at the application layer or database layer.
X-Forwarded-For – This helps to identify the public IP address under the private cluster. This is part of the classic Load balancer. ELB can pass the public IPv4 address of the client to the EC2 instance.
Notes:
Load balancers do not provide any public IP address; instead, they will have a DNS name. You will need to update the DNS in cname instead of IP address.
Instances monitored on the ELB are reported as
In-service
Out-service
Health checks verify if the instance is healthy by talking to them.
All AMIs are either categorized as Amazon EBS or backed by an instance store.
For EBS volume – The root device for an instance launched from the AMI is an Amazon EBS volume created from an amazon EBS snapshot.
For Instances Store Volume – The root device is an instance launched from the AMI is an instance store volume created from a template stored in Amazon S3.
AMI can be selected based on
Region (Region and Availability zone)
Operating Systems
Architecture (32 and 64 bits)
Launch permission
Storage for root device (Root Device Volume)
Instance Store (Ephemeral Storage)
EBS backend volume
Operating System
AWS provides a variety of OS, which we can select and install as per our requirements.
Architecture Type
Root Device Type
EBS vs Instance Store
Instance store volumes are sometimes called Ephemeral storage.
Instance store volumes cannot be stopped You will lose the data on this instance if it is stopped.
The EBS-backed instances can be stopped, without data loss.
You can reboot both types of the Instance, without data loss.
Upon termination, both Root volumes will be deleted. But in EBS volumes, we can tell AWS to keep the root device volume.
AWS Route53 service is used to manage the DNS records for your infrastructure.
SOA record stores information about
The name of the server that supplied the data of the zone.
The administrator of the zone
The current version of the data file.
The default number of seconds for the time-to-live file on the resource record
Alias Records
Alias records are used to map resource record sets in your hosted zone to ELB, CloudFront Distribution, or S3 buckets that are configured as websites.
Alias record works like CNAME record in that you map one DNS name (www.example.com) to another target DNS name (elbtest.elb.amazonaws.com).
Key Differences – A CNAME can’t be used for naked domain names (Zone apex record). You can’t have a CNAME pointing to the naked domain name i.e. http://example.com. It must be either an A record or an alias.
Route 53 is a global service and is not restricted to regions.
Routing policy in AWS
Simple Routing
Weighted Routing
Latency – Based routing
Failover Routing
Geolocation Routing
MultiValue Routing
Simple Routing policy
This is the default routing policy when we create a new record set.
It is good when we have only one record set.
It basically works around robin policy, for two instances each of them will appear one after the another.
Weighted routing policy
As we can see from the example, with weighted routing we can route the traffic based on the weight to certain instances.
Latency Based Routing
Latency-based routing allows you to route your traffic based on the lowest network latency for your end-users. (fastest response time).
To use latency-based routing, you create a latency resource record set for the amazon EC2 or (ELB) resource in each region that hosts your website. When AWS route 53 receives a query for your website. It selects the latency resource record set for the region that gives the user the lowest latency. Route 53 then responds with the value associated with that record set.
Failover Routing Policy
Failover routing policies are used when you have to create an Active/Passive setup.
Route 53 will monitor the health of your primary site using the health check page.
The Heath check page is the endpoint of your instances.
We need to create health checks and configure the endpoints to be monitored.
Geo-Location Routing Policy
Geo Location Routing lets you choose where your traffic will be sent based on the geographical location of the users.
Multivalue Answer Routing
You can have health check endpoints for each EC2 instance and if anyone fails you can remove it.
