Introduction

TLDR

Problem

  • Environmental compatibility is a headache, in addition to scaling, security, maintenance and upgrade of software and hardware.
    • For instance, in the serverless examples, the need for pickle5 instead of pickle was due to such incompatibility.
  • For hosted environments, we have to work hard on the devops to ensure the environments are the same.
  • For serverless, we did this via the requirements file (Cloud Functions) and locally installing python packages (Lambda functions)

Solution

  • Containers

    • they allow you to run arbitrary programs on new environments as long as they have a common piece of software (e.g., docker).

  • Their use cases include:

    • Webapp deployments (real time dashboards)
    • Model deployments (our objective)
    • Packaging up projects (like a snapshot) and archiving so you can return to the project with the runtime/environment all set to go.

Containers

  • Containers are tools to produce reproducible environments by isolating and packaging everything that your program needs, independent of the target/host environment where you intended to run the program.
  • Many tasks can be isolated and served in such a way, including model deployments.
  • A Container is a process with features to keep it isolated from the host system.
  • Because of their architecture, they are much lighter on resources compared to a full blown VM. You could think of them as stripped down virtual machines (VMs). Obviously this comes with some trade-offs as well.
  • Containers with our code/program/model in it can be distributed to team members and can be publicly released with minimal environment incompatibility issues.
  • One key variant of this solution is Docker. There are others, but we will stick with this one.
  • In summary, they are useful because:
    • consistency: in app development as well as its delivery
    • portable: local machine or cloud
    • lightweight: multiple containers on single host

Elastic Container Service by AWS

  • A proprietary solution by AWS
  • Elastic Container Service (ECS) similar to Lambda functions, and allows one to be agnostic to devops details (which server, server configs, security etc).
  • Lifts certain restrictions of Lambda functions:
    • Not specific to the runtimes available in Lambda functions
    • No memory limit (e.g., 256MB)
  • In ECS, we have to define the container explicitly, and then the rest (scaling, fault-tolerance) is behind the scenes.

Note: While exploring ECS, keep a tab on billing (check every so often)!

What we will study

  • Docker local experimentation
  • Run docker on EC2/AWS
  • Use ECS (and Elastic Container Registry/ECR) on AWS
  • The end goal is a functionality that is almost serverless, but has more work, potentially costing less and also allowing for more flexibility.