Lambda Functions

  • Lambda by Amazon Web Services (AWS) is an analogous serverless solution.
  • Lambda can be used internall as well as for model deployments (we are focusing on the latter).
  • We will repeat setting up the weather app and the recommender model, using the CLI (command line interface tools)

aws

Aside: Setting up an IAM user

  • TBD

Hello World in Lambda

  • Select the lambda service.

lambda_start

  • Pick the python 3.7 runtime.

lambda_scratch

  • You will see the green bar indicating a successful creation.

lambda_start_success

  • We will initially not changed the code.

lambda_hello_world

  • And create a dummy test.

lambda_hello_world_test

  • The result looks like this.

lambda_hello_world_test_result

Setting up an API Gateway

  • We will slightly change the function so that it forms a proper HTTP response.

lambda_api0

  • Under the Designer section, we will ‘Add Trigger’ and select ‘API Gateway’.

lambda_api1

lambda_api2

  • We choose HTTP API and security to be open for the time being.

lambda_api3

  • A block will appear on the designer tab in the Lambda function page.

lambda_api4

  • Once you click on the API Gateway link provided, you will see the response as expected:

lambda_api5

lambda_api6

See this documentation for more details.

Set up a Programmatic User to Access S3

  • What is IAM? IAM (Identity and Access Management) is a very useful tool for an organization to administer and control access to various resources.

    • We will create a programmatic user, similar to the service account in GCP.

  • What is S3? S3 (Simple Storage Service) is a very popular service used by many large and small companies and individuals.

    • For example, a typical data science work flow may involve ETL work on databases (such as Amazon Redshift) and then storing outputs on S3.

  • Start with finding the IAM service.

iam

  • Click on create a new user.

iam1

  • We will make this user have programmatic access (for example via Python later on).

iam2

  • For now, we let the user have full read and write access to S3.

iam3

  • Lets review the user setup.

iam4

  • In this step, copy the access key and secret key that will be needed for the programmatic access. They will not be shown again, so store them safely and securely.

iam5

  • Here is the description of the user we just created.

iam6

Set up AWS CLI

The AWS Command Line Interface (CLI) is a unified tool to manage your AWS services. With just one tool to download and configure, you can control multiple AWS services from the command line and automate them through scripts.

  • Head over to https://aws.amazon.com/cli/ and download a suitable installer for your OS.

  • Install it and open your terminal. If it is in the PATH, you should be able to verify the following:

    which aws
#/usr/local/bin/aws #your location could be different.
aws --version
#aws-cli/2.0.45 Python/3.7.4 Darwin/18.6.0 exe/x86_64

  • Using the access and secret keys to configure aws using the aws configure command.

  • Test S3 access by issing the following commmand: aws s3 ls

    $ aws s3 ls
2020-09-03 12:27:48 chicagodatascience-com
2020-09-03 12:02:25 theja-model-store

  • Instead of programmatically creating a bucket (with the aws s3 mb s3://bucket-name) we will do it using the GUI below.