Didn’t do this

Thanks to Florient Chouteau and Dennis Wilson

for their work on this subject.

I took most of the content from theirs:

• https://supaerodatascience.github.io/OBD/1_1_overview.html
• https://supaerodatascience.github.io/OBD/slides/1_1_cloud_computing.html
• https://supaerodatascience.github.io/OBD/slides/1_2_cloud_usage.html
• https://supaerodatascience.github.io/OBD/slides/1_3_gcp.html

But it’s a bit bigger…

Google Cloud Data Center locations

Cloud Definition

The cloud is a real physical place - accessed over the internet - where a service is performed for you or where your stuff is stored. Your stuff is stored in the cloud, not on your device because the cloud is not on any device; the cloud lives in datacenters. A program running on your device accesses the cloud over the internet. The cloud is infinite, accessible from anywhere, at any time

Todd Hoff in “Explain the Cloud like I’m 10”

Cloud Computing

Using cloud resources for any purpose, web server, storage, computations.

• The cloud is a set of cloud providers
• Renting cloud services
• Increasingly abstracted from the physical hardware

A portion of AWS services

Quizz

What’s the Cloud?

• Answer A: Still don’t know…
• Answer B: A magic place where my photos and alikes are
• Answer C: Someone else computer
• Answer D: Several big Data centers around the world that host data and service

Answer link Key: zq

How?

Definition

In computing, virtualization refers to the act of creating a virtual (rather than actual) version of something, including virtual computer hardware platforms, storage devices, and computer network resources.

Wikipedia

Basically we are running software on “abstract hardware” which is a “portion” of a real computer (“bare metal”)

Desktop virtualization

Server virtualization

Virtualization evolution

Definitions

• Hypervisor (VMWare, Virtualbox, KVM): A hypervisor is a program for creating and running virtual machines
• Virtual Machine: A virtual machine is the emulated equivalent of a computer system that runs on top of another system
• Containers: Isolated environments that share the same underlying OS (more this afternoon) & resources

Opens up new possibilities

Hardware abstraction

• Hardware Abstraction (“download more RAM”)
• Fine-grained resource allocation / sharing
• Decouple maintenance of hardware from maintenance of software

Reliability, security…

Quizz

Virtualization allows… (multiple choices)

• Answer A: Hardware abstraction, download a machine with more ram
• Answer B: Resources sharing optimization
• Answer C: Building immersive video games
• Answer D: Reliability
• Answer E: Reproducibility

Answer link Ket: un

AWS

Once upon a time…

Amazon (the e-commerce store) has “scaling” issues

Idea

2002-2003; The idea

Building an infrastructure that is completely standardized, completely automated, and relied extensively on web services for things like storage

http://blog.b3k.us/2009/01/25/ec2-origins.html

So …

Basically Amazon became very good at running scalable infrastructure as services

• For themselves…
• … but also for other partners (target)

And that infrastructure is often there to answer peak load…

Let’s sell it !

Public, hybrid, private

Different layers of abstractions

Abstraction examples

• Using data storage service like google cloud storage without managing the infrastructure ?aaS
• Using google drive ?aaS
• Renting a server with hard drive and storing data ?aaS

Quizz

What means IaaS?

• Answer A: I am a Sociopath
• Answer B: Information as a System
• Answer C: Information as a Service
• Answer D: Infrastructure as a Service

Answer link Key: gk

Public

Public (European)

Academic, public founded:

Private/on premise

Leaders

In France

A technical evolution

• More Virtualization
• More API
• More Managed Services

Access and operating computing power

• Outsourcing infra, maintenance, security, development of new services
  • Less (?) operation cost (or at least less burden)
• Pay-per-use, pay as you go, change of economical model
• “Infinitely scalable” for common folks
• “No need to plan out” infrastructure
  • Enabling innovation
  • Power in the hands of developpers/builders

Technical benefits

• Infrastructure as Code
• Continuous Deployment
• Infinite resources
• On demand scalability and resources

Changing the way we interact with hardware

We interact with cloud providers using APIs…

gcloud compute --project=deeplearningsps instances create ${INSTANCE_NAME} \
    --zone=${ZONE} \
    --machine-type=n1-standard-8 \
    --scopes=default,storage-rw,compute-rw \
    --maintenance-policy=TERMINATE \
    --image-family=ubuntu-1804-lts \
    --image-project=ubuntu-os-cloud \
    --boot-disk-size=200GB \
    --boot-disk-type=pd-standard \
    --accelerator=type=nvidia-tesla-p100,count=1 \
    --metadata-from-file startup-script=startup_script.sh

Infrastructure as Code

• Infrastructure is now managed via text files
• Data is securely stored on storage
• So we store code + urls on git… and everything is reproducible ! (not that simple)
• We use automated deployment tools (terraform, gcp deployment manager…)

Pet vs Cattle

Quizz

Why Cloud computing is a usage revolution (multiple choices)?

• Answer A: Infinite resources (CPU, memory, storage)
• Answer B: It’s cheaper
• Answer C: Infrastructure managed as Code
• Answer D: Pay per use
• Answer E: It gives all our data to Google and Amazon

Answer link Key: sm

Object store

High througput storage system with horizontal scalability

Compute as a service

• Start and stop compute resources when needed
  • IaC tools like Terraform, Ansible
  • Mesos/Nomad and other tools.
• Kubernetes as a Service
  • With autoscaling features: add VMs if needed
  • Dask, Spark and other processing tools plugged in
• Function as a Service
  • Just put a bit of code, everything else is handled.
• HPC as a Service
  • With POSIX filesystem (sometimes over Object store)
  • And high performance network.

HPC as a Service

AWS ParallelCluster

Is using cloud computing less expensive?

• Depend on your {normal / peak} utilization
• Access to latest hardware without investment
• Fully utilized hardware is more expensive on the cloud
• CLOUD HYGIENE !
  • Watch for unused services / storage
  • Shutdown machines when not used
  • Services stack up…

Is using cloud computing more secure / safer.

• The best engineers in the world working on it
• Secure regions / private cloud…
• Your data somewhere in some datacenter…
• “Dependency” towards your cloud provider…
• Still need to handle security inside you resources

Presentation

• One of the main cloud provider
• Behind AWS in SaaS (serverless…)
• More “readable” product line (for a Cloud Provider…)
• Very good “virtual machine” management
  • per second billing
  • fine-grained resource allocation

Services

Concepts: Zones and regions

Concepts: Projects

• Access (Enabling API/Services)
• Ressources (Quota by project)
• Networking
• Billing

Concepts: Identity and Access Management (IAM)

Interacting with GCP: The Console

Interacting with GCP: SDK & Cloud Shell

• Using the gcloud CLI: https://cloud.google.com/sdk/install
• Using Google Cloud Shell: A small VM instance you can connect to with your browser

Objectives

Everyone has its credits/coupons?

• Create your GCP account, configure your credentials, ensure everything is working
• With the coupons you got, you should see $50 of credits (in the billing tab)
• Interact with Google cloud console: check all the services
• Start a first VM instance through the console, watch prices and all the options
• Create a Google Storage bucket and upload content
• Connect to google cloud shell and interact with it
• Follow this Google codelabs to deploy a Website!

We’ll do more this afternoon!