Serverless Deployments

A. TLDR

Models do not need to be complex, but it can be complex to deploy models. - Ben Weber (2020)

Problem

  • We have to take care of provisioning and server maintenance while deploying our models.
  • We have to worry about scale: would 1 server be enough?
  • How to minimize the time to deploy (at an acceptable increase in cost)?
  • How can a single developer or data science/analytics professional manage a complex service?

Solution

  • Software that abstracts away server details and lets you serve your model (any function actually) with few lines of code/UI.

  • The software automates

    • prrovising servers
    • scaling machines up and down

  • load balancing

    • code versioning

    • Our task is then to specify the requirements (pandas, pytorch).

B. Our Objective

  • Write serverless functions that generate predictions when they get triggered by HTTP requests.
  • We will work with:
    • AWS
    • GCP
  • We will deploy
    • a keras model, and
    • a sklearn model.

C. Managed Services

  • Cloud is responsible for abstracting away various computing components: compute, storage, networking etc
  • Minimizes thinking about dev/staging vs production.
  • Note: what we did last class, ssh’ing into a virtual private server (VPS) would be considered as a hosted deployment, which is the opposite of managed deployment
  • For example, serverless technology was introduced in 2015⁄2016 by AWS (Amazon web) and GCP (Google cloud). It contrasts with VPS based deployment. Similarly, AWS ECS (Elastic Container Service) managed solution contrasts with the hosted/manual Docker on VPS setup.

When is a managed solution a bad idea?

  • No need for quick iteration (company cares about processes and protocols)
  • Need a high speed service
  • No need to scale system arbirarily
  • Cost conscious or have an in-house developer.