Category: Application Service

Application services

API Gateway Overview

admin

API Gateway

  • We have two type of the API
    • REST API – Representational State Transfer. It uses JSON. It is also used in AWS.
    • SOAP API – The existing one, it uses XML.
  • AS API Gateway is a fully manage services used by the developers to publish, maintain, monitor and secure API at any scale.
  • The code can be in EC2 or in serverless Lambda or DynamoDB.
  • It provides us HTTPS endpoints to define a RESTFUL API.
  • Each API end point can point to different targets.
  • It is scalable.
  • We can track and control usage using the API key.
  • Throttle request to prevent attack.
  • We can maintain multiple version of the API.
  • You can use cloudwatch to monitor the logs of API Gateway.
  • Although API gateway supports REST method, it do support legacy SOAP method. But that will be custom configuration.

Configuring the API

  • Define the API (Container)
  • Define the resources and nested resources (URL paths)
  • For each resources
    • Select the supported HTTP method
    • Set the security
    • Choose target (EC2, Lambda or DyanamoDB)
    • Set request and response transformation.
  • Deploy API to Stage
  • It will use the API gateway domain by default
  • You can use custom domain
  • Now supports AWS certificate manager (get free SSL/TLS certs)

API caching

  • It is very similar to cache, instead of hitting to the target, the frequent requests are cached at endpoint and if someone requests the same information, the cached information is sent back.
  • It has a TTL set.
  • This decreases the latency.

Same-origin policy

  • It is an important concept in web application security.
  • It allows first webpage scripts to access the second page script, only if they are in the same region.
  • This is managed and handled by the browsers
  • This helps in avoiding the XSS attacks
  • This is ignored by tool like Postman and CURL.

CORS Cross-Origin Resource Sharing

  • This is one way in which the server at one end can relax the same origin policy.
  • This will allow the restricted resource on one web application to be shared /requested from another domain outside the domain through which the request is been served.
  • Browser makes an HTTP OPTIONS call for an URL. OPTIONS are GET, PUT, POST
  • Server returns the option that says “These domains are approved to GET this URL”
  • Error – “Origin policy cannot be read at the remote resource “; you will need to enable CORS at the AI Gateway.

Advance API Gateway

  • We can use the API Gateway Import feature to import from an external definition into API gateway.
  • With import API you can either create a new API by submitting a POST request that includes swagger definition in the payload and end point configuration or you can update an existing API by using a PUT request that contains a swagger definition in payload.
  • You can update the existing API definition by overwriting it or you can merge a definition with existing one.
  • You can specify the options in mode query parameter in the request URL.

API throttling

  • By default, API gateway has a steady state request rate to 10000 requests per second.
  • The maximum concurrent request is 5000 request across all API with an account.
  • If the request exceeds 5000 or 10000 limit, you will get an error “429 too many request”

Kinesis Overview

admin
  • Streaming data, it’s a data generated continuously with thousands of sources, which sends the data in small size and record continuously. E.g. listed below,
    • Purchases from online store amazon.com
    • Stock price
    • Game data
    • Social network data
    • Geospatial data (Uber data) constantly sends the data about the cab and users location.
    • IOT data (Sensor Data).
  • Kinesis is a platform where we send the stream data, it helps to load and analyze the streaming data. It also helps in building your own custom applications as per the business needs.
  • Kinesis offers three core services
    • Kinesis stream
    • Kinesis Firehose
    • Kinesis Analytics
  • Kinesis retains the data for 24 hours and you can increase that to 7 days retention.
  • Kinesis Stream
    • They store the data in Shard, then the data is passed to EC2.
    • Once processed in EC2, the data can be moved further to different storage media like DynamoDB, S3, EMR and Redshift.
    • Kinesis streams consists of Shards.
    • 5 transactions per second to read.
    • 2 MB /Sec max data read rate.
    • 1000 records per second to write up to maximum 1 MB per second including the partition key.
    • The total capacity of the stream is the sum of the capacities of its Shard.
  • Kinesis Firehose
    • In Firehose, the data is acquired from different media like your personal laptop/S3 etc. and passed to Firehose, in Firehose it is analyzed and processed and then passed to S3. The data analytics is optional.
    • The data is retained for 24-hour default and can be extended till 7 days.
    • You can pass the data to Redshift but it should go through the S3. You can also pass the data to elastic search.
  • Kinesis analytics
    • It allows you to run the SQL queries; you can store the data of SQL further to S3/Redshift and Elasticsearch Cluster.
    • It is a web analysis tool used with SQL type query.

SWF (Simple Workflow Service)

admin
  • In SWF the task is assigned only once and never duplicated.
  • SWF coordinates work across distributed application components.
  • It helps in implementing complex business processes and work flows.
  • It is suitable for long running execution and not for short batch jobs etc.
  • Enables complex interaction within different application on different platform, AWS or on-premise infrastructure and between different users.
  • Maximum work flow can be 1 year and it is measured in seconds.
  • Features of SWF
    • Tasks executed with no duplicate
    • Routing and queueing of the task handled by the SWF.
    • It also provides timeout and status of the task.
    • The task workflow can have child process, and a list of hierarchy process.
    • It handles user data Input and execution result output.
  • SWF components
    • A workflow is control flow logic for execution of the task.
    • A domain contains of workflow or workflows.
    • Tasks can be performed by executable code, a webservices call or end user input. They can be performed parallel or serially.
    • Actors interact directly with SWF to coordinate tasks.
  • SWF Actors
    • Actors can be workflow starter, decider or activity workers
    • Starters can initiate the execution of work flow.
    • Deciders implement work flow logic and notify SWF changes during the workflow execution
    • Activity Workers perform activity tasks of the workflow.   
  • SWF Task
    • Must be registered using either the console or the RegisterActivityType action (API/CLI).
    • When scheduled you can specify a task list (queue)
    • Decision and Activity task has separate lists (queue)
    • Task can be assigned to activity workers through task routing if required.
  • Application communication can be established to SWF through,
    • SDK
    • SWF API HTTP POST
    • Flow Framework (Java or Ruby)
    • CLI
  • SWF and SQS
    • SWF presents a task oriented API and SQS offers a message oriented API
    • SWF ensures that the task is assigned only once and never duplicated, with SQS you may get duplicate message and you will need to ensure that a message is process only once.
    • SWF keeps tracks of all the tasks and events in an application, in SQS we should implement our own application level tracking especially if the application uses multiple queues.

SNS (Simple Notification Service)

admin
  • SNS is a push mechanism, as supposed to be SQS which is pull mechanism.
  • It pushes cloud notification directly to the mobile devices.
  • SNS can deliver the message via SMS/Email to SQS or any HTTP end point.
  • To avoid message loss, message published in the SNS are stored redundantly across multiple availability zone.  
  • SNS Topic
    • SNS allows to group multiple recipients using the topics. A topic is an access point for allowing recipients to dynamically subscribe for identical copies of the same notification.
    • One topic can support deliveries to multiple end point types – e.g. you can group together iOS, Android and SMS recipients.
  • SNS Benefits
    • Instantaneous push based delivery (no Pulling)
    • Simple API and easy integration with applications.
    • Flexible message delivery over multiple transport protocol.
    • Inexpensive, pay-as-you-go model with no up-front costs
    • Web-based AWS management console offers simplicity of a point-and-click interface.
  • SNS vs SQS
    • Both are messaging services in AWS.
    • SNS is Push
    • SQS is Pull
  • SNS Pricing
    • $0.50 per 1 million SNS Requests.
    • $0.06 per 100000 Notification deliveries over HTTP.
    • $0.75 per 100 notifications delivered over SMS.
    • $2.00 per 100000 notifications delivered over email.

Elastic Transcoder

admin
  • Elastic Transcoder is a media Transcoder in cloud.
  • Converts the original media files into different formats which will be compatible with smartphones, tablets and PCs. It optimizes the source media file.
  • Provides transcoding presets for popular output formats, which means we don’t need to worry about which setting works best for devices.
  • Pay based on the minutes and the resolution of the media which is transcoded.
  • How it works,
  • Upload the media file in the S3 bucket, Pass it to the lambda function, lambda function will invoke Elastic Transcoder.
  • Elastic Transcoders will then further process and convert it to different media format.
  • Then it is moved back to S3, it also creates thumbnail of the media file.

API Gateway

admin

API Gateway

  • API caching will cache the endpoint response. Thus, reducing the number of requests.
  • Feature of API Cache
    • Its low cost and Efficient
    • Scales Effortlessly and automatically.
    • Increase performance.
    • You can throttle requests to prevent requests.
    • Connects to cloud watch to log all requests.
  • Same origin policy
    • It is an important web application security model, under this policy a web browser permits scripts contained in a first web page to access data in a second web page, only if both web pages have the same origin (Which means same domain name).
  • CORS (Cross-Origin Resource Sharing)
    • CORS is on the way where the server at the other end (Not the client code in the browser) can relax the same origin policy.
    • This policy is a mechanism that allows restricted resources (e.g. font) on a web page to be requested from another domain outside the domain from which the first resource was served.
    • If you are using JavaScript/AJAX that uses multiple domains with API Gateway, ensure that you have enabled CORS on API Gateway.

The flow of the API Gateway or how it is implemented.