Serverless Deployments
A. TLDR
Models do not need to be complex, but it can be complex to deploy models. - Ben Weber (2020)
Problem
- We have to take care of provisioning and server maintenance while deploying our models.
- We have to worry about scale: would 1 server be enough?
- How to minimize the time to deploy (at an acceptable increase in cost)?
- How can a single developer or data science/analytics professional manage a complex service?
Solution
B. Our Objective
- Write serverless functions that generate predictions when they get triggered by HTTP requests.
- We will work with:
- We will deploy
- a keras model, and
- a sklearn model.
C. Managed Services
- Cloud is responsible for abstracting away various computing components: compute, storage, networking etc
- Minimizes thinking about dev/staging vs production.
- Note: what we did last class, ssh’ing into a virtual private server (VPS) would be considered as a hosted deployment, which is the opposite of managed deployment
- For example, serverless technology was introduced in 2015⁄2016 by AWS (Amazon web) and GCP (Google cloud). It contrasts with VPS based deployment. Similarly, AWS ECS (Elastic Container Service) managed solution contrasts with the hosted/manual Docker on VPS setup.
When is a managed solution a bad idea?
- No need for quick iteration (company cares about processes and protocols)
- Need a high speed service
- No need to scale system arbirarily
- Cost conscious or have an in-house developer.