Adding GPUs to instances

Check and request for an increase in your quotas for GPUs:

• Search for GPU. If you have 0 in your allocated quotas, select the type of GPU and region and click on edit the quotas. Fill in the request form (https://console.cloud.google.com/iam-admin/quotas?project=packt-gcp).

There are several restrictions when it comes to using GPUs on Google Compute. GPUs are not available in shared or pre-emptible machines. GPU instances are terminated for regular (weekly) maintenance events. See https://cloud.google.com/compute/docs/gpus/ for up-to-date information on the restrictions. See also https://cloud.google.com/compute/docs/gpus#introduction to learn what machine types are available based on your desired GPU count.

To create a VM with GPU from the console:

1. Go to the VM console and click on Create Instance
2. Select a zone that is GPU compatible
3. Click on customize the machine type and again on the GPUs link
4. Select the Number of GPUs and the associated type you require:

Similarly you can create a GPU-enabled instance with gcloud with the following command:

$ gcloud compute instances create [INSTANCE_NAME] \
--machine-type [MACHINE_TYPE] --zone [ZONE] \
--accelerator type=[ACCELERATOR_TYPE],count=[ACCELERATOR_COUNT] \
--image-family [IMAGE_FAMILY] --image-project [IMAGE_PROJECT] \
--maintenance-policy TERMINATE --restart-on-failure \
--metadata startup-script='[STARTUP_SCRIPT]'

Where --accelerator type= specifies the type of GPU and count= specifies the number of GPUs.

For instance, this command will create an Ubuntu 1604 instance with one NVIDIA? Tesla? K80 GPU and two vCPUs in the us-east1-d zone. The startup-script metadata instructs the instance to install the CUDA toolkit with its recommended driver version:

$ gcloud compute instances create gpu-instance-1 \
--machine-type n1-standard-2 --zone us-east1-d \
--accelerator type=nvidia-tesla-k80,count=1 \
--image-family ubuntu-1604-lts --image-project ubuntu-os-cloud \
--maintenance-policy TERMINATE --restart-on-failure \
--metadata startup-script='#!/bin/bash
echo "Checking for CUDA and installing."
# Check for CUDA and try to install.if ! dpkg-query -W cud
a-8-0; then
 curl -O http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_8.0.61-1_amd64.deb
 dpkg -i ./cuda-repo-ubuntu1604_8.0.61-1_amd64.deb
 apt-get update
 apt-get install cuda-8-0 -y
fi'

The startup script installs the right CUDA driver for the Ubuntu. For other drivers and operating systems, follow the instructions at https://developer.nvidia.com/cuda-downloads.

Once the driver has finished installing, you can verify that it is properly installed:

1. Do ssh into your instance
2. Type nvidia-smi to see your driver version and how much GPU memory you have

The command nvcc --version shows the current CUDA version.