Serverless Deployments

A. TLDR

Models do not need to be complex, but it can be complex to deploy models. - Ben Weber (2020)

Problem

  • We have to take care of provisioning and server maintenance while deploying our models.
  • We have to worry about scale: would 1 server be enough?
  • How to minimize the time to deploy (at an acceptable increase in cost)?
  • How can a single developer or data science/analytics professional manage a complex service?

Solution

  • Software that abstracts away server details and lets you serve your model (any function actually) with few lines of code/UI.

  • The software automates

    • prrovising servers
    • scaling machines up and down
  • load balancing

    • code versioning

    • Our task is then to specify the requirements (pandas, pytorch).

B. Our Objective

  • Write serverless functions that generate predictions when they get triggered by HTTP requests.
  • We will work with:
    • AWS
    • GCP
  • We will deploy
    • a keras model, and
    • a sklearn model.

C. Managed Services

  • Cloud is responsible for abstracting away various computing components: compute, storage, networking etc
  • Minimizes thinking about dev/staging vs production.
  • Note: what we did last class, ssh’ing into a virtual private server (VPS) would be considered as a hosted deployment, which is the opposite of managed deployment
  • For example, serverless technology was introduced in 20152016 by AWS (Amazon web) and GCP (Google cloud). It contrasts with VPS based deployment. Similarly, AWS ECS (Elastic Container Service) managed solution contrasts with the hosted/manual Docker on VPS setup.

When is a managed solution a bad idea?

  • No need for quick iteration (company cares about processes and protocols)
  • Need a high speed service
  • No need to scale system arbirarily
  • Cost conscious or have an in-house developer.