AZ-104 Deploy and manage Azure compute resources

My notes from the learning path AZ-104 Deploy and manage Azure compute resources of Microsoft Certified: Azure Administrator Associate on Microsoft Docs

Introduction to Azure virtual machines

Size of the VM

Option	Description	Size Series
General purpose	balanced CPU-to-memory ratio. Ideal for testing and development, small to medium databases, and low to medium traffic web servers.	B, Dsv3, Dv3, DSv2, Dv2
Compute optimized	high CPU-to-memory ratio. Suitable for medium traffic web servers, network appliances, batch processes, and application servers.	Fsv2, Fs, F
Memory optimized	high memory-to-CPU ratio. Great for relational database servers, medium to large caches, and in-memory analytics.	Esv3, Ev3, M, GS, G, DSv2, Dv2
Storage optimized	high disk throughput and IO. Ideal for VMs running databases.	Ls
GPU	heavy graphics rendering and video editing. are ideal options for model training and inferencing with deep learning.	NV, NC, NCv2, NCv3, ND
High performance computes	the fastest and most powerful CPU virtual machines with optional high-throughput network interfaces.	H

Azure Automation Services

Azure Automation allows you to automate management tasks with ease. These services include:

• Process Automation allows you to set up watcher tasks that can respond to events that may occur in your datacenter.

• Configuration Management allows you to track software updates for the operating system that runs on your VM and take action as required.

  • Microsoft Endpoint Configuration Manager is used to manage your company's PC, servers, and mobile devices and can be extended to your Azure VMs.

• Update Management is used to manage updates and patches for your VMs.

  • Update management incorporates services that provide process and configuration management.

  • Update management can be enabled for a VM directly from your Azure Automation account or from the virtual machine pane in the portal.

Availability Set

Availability set is a logical feature used to ensure that a group of related VMs are deployed so that they aren't all subject to a single point of failure and not all upgraded at the same time during a host operating system upgrade in the datacenter.

• Microsoft offers a 99.95% external connectivity service level agreement (SLA) for multiple-instance VMs deployed in an availability set.
• When you place VMs into an availability set, Azure guarantees to spread them across Fault Domains and Update Domains.
  • A fault domain is a logical group of hardware in Azure that shares a common power source and network switch.
  • An update domain is a logical group of hardware that can undergo maintenance or be rebooted at the same time.

Failover  Across Locations

Azure Site Recovery replicates workloads from a primary site to a secondary location with two significant business advantages:

1. Azure as a destination for recovery, thus eliminating the cost and complexity of maintaining a secondary physical datacenter.

2. Simple to test failovers for recovery drills without impacting production environments. 

Create a Linux virtual machine in Azure

Storage Options

There are two levels of SSD storage available:

• Choose Standard SSD disks if you have normal workloads but want better performance.
• Choose Premium SSD disks if you have I/O intensive workloads or mission-critical systems that need to process data very quickly.

Mapping Storage to Disks

Two virtual hard disks (VHDs) will be created for your Linux VM:

1. The operating system disk: This is your primary drive, and it has a maximum capacity of 2048 GB. It will be labeled as /dev/sda by default.

2. A temporary disk: This provides temporary storage for the OS swap files or any apps. The disk is /dev/sdb and is formatted and mounted to /mnt.

• Azure virtual disk sizes are measured in Gibibytes (GiB), which are not the same as Gigabytes (GB); one GiB is approximately 1.074 GB (equals to 2³⁰ bytes).

Unmanaged vs. managed disks

• With unmanaged disks, you are responsible for the storage accounts that are used to hold the VHDs that correspond to your VM disks.
  • You pay the storage account rates for the amount of space you use.
  • A single storage account has a fixed rate limit of 20,000 I/O operations/sec.
  • This means that a single storage account is capable of supporting 40 standard virtual hard disks at full throttle.
• Managed disks are the newer and recommended disk storage model.
  • Azure creates and manages both the disk and the storage it uses so you don't have to worry about storage account limits.
  • RBAC supported
  • Snapshot and backup supported

Authentication Method for SSH

Generating a key pair

ssh-keygen -t rsa -b 4096

Install public key in an existing VM named myserver with a user azureuser

ssh-copy-id -i ~/.ssh/id_rsa.pub azureuser@myserver

Creating a Linux VM with the Azure Portal

Virtual Network

•  You can create a new one and configure the:

  • Address space: The overall IPV4 space available to this network.
    • If the VNet will be connected to other VNets, you must select address ranges that are not overlapping.
    • You can use unrouteable IP addresses such as 10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16, or define your own range.
  • Subnets: The first subnet to subdivide the address space - it must fit within the defined address space. 
    • Subnets is used to segregate your network into more manageable sections. 
    • Azure reserves the first four addresses and the last address in each subnet for its use.
• By default, Azure creates a virtual network, network interface, and public IP for your VM.

VM IP Addresses

• Unless you’ve set up a site-to-site VPN to Azure, your Azure VMs won’t be accessible from your local network.
• Public IP addresses in Azure are dynamically allocated by default. That means the IP address can change over time.
• You can pay more to assign static addresses.

Connect to a Linux virtual machine with SSH

• Useful commands:
  • ls -la / to show the root of the disk
  • ps -l to show all the running processes
  • dmesg to list all the kernel messages
  • lsblk to list all the block devices - here you will see your drives

Initialize data disks

• Any additional drives you create from scratch need to be initialized and formatted.

identify the disk

dmesg | grep SCSI

initialize the disk /dev/sdc

(echo n; echo p; echo 1; echo ; echo ; echo w) | sudo fdisk /dev/sdc

write a file system to the partition

sudo mkfs -t ext4 /dev/sdc1

mount the the drive to the file system

sudo mkdir /data && sudo mount /dev/sdc1 /data

Install the Apache web server

Update the local package index

sudo apt-get update

Install Apache server

sudo apt-get install apache2 -y

Check the status if the daemon will start automatically

sudo systemctl status apache2 --no-pager

Network and security settings

• By default, new VMs are locked down.

Network Security Group

• Network security groups (NSGs) are used to enforce and control network traffic rules at the networking level.
• Security groups can be associated to a network interface (for per host rules), a subnet in the virtual network (to apply to multiple resources), or both levels.
• Each security group has a set of default rules that cannot be modified but can be overridden.

Security Rules

• For inbound traffic, Azure processes the security group associated to the subnet and then the security group applied to the network interface.
• Outbound traffic is handled in the opposite order (the network interface first, followed by the subnet).
• Security groups are optional at both levels. If no security group is applied, then all traffic is allowed by Azure.
• The rules are evaluated in priority order, starting with the lowest priority rule. Deny rules always stop the evaluation.
  • if a network interface rule blocks an outbound request, any rules applied to the subnet will not be checked.
• The last rule is always a Deny All rule which is a default rule.
• SMTP (port 25) is a special case. Depending on your subscription level and when your account was created, outbound SMTP traffic may be blocked.

Configure network settings

• Always make sure to lock down ports used for administrative access.
• An even better approach is to create a VPN to link the virtual network to your private network and only allow RDP or SSH.
• You can also change the port used by SSH to be something other than the default. Keep in mind that changing ports is not sufficient to stop attacks. It simply makes it a little harder to discover.

Create a Windows virtual machine in Azure

Storage Options

Mapping storage to disks

By default, two virtual hard disks (VHDs) will be created for your Windows VM:

1. The Operating System disk. This is your primary or C: drive and has a maximum capacity of 2048 GB.

2. A Temporary disk. This provides temporary storage for the Windows paging file or any apps. It is configured as the D: drive by default.

Create a Windows virtual machine

• Windows password must be at least 12 characters long. It must have three of the following:
  • one lower case character
  • one uppercase character
  • one number
  • one special character that is not '\' or '-'. 

Use RDP to connect to Windows VMs

• If you're connecting over VPN or ExpressRoute, you can select the internal IP address.

• When you connect, you'll typically receive two warnings. These are:

  • Publisher warning - caused by the .rdp file not being publicly signed.
  • Certificate warning - caused by the machine certificate not being trusted.
• The .rdp file can be signed using RDPSIGN.EXE and the machine certificate placed in the client's Trusted Root Certification Authorities store.

Connect to the VM with RDP

• The first time you connect to a Windows server VM, it will launch Server Manager
• This is where we would add the Web Server role (IIS) to the server. 

Install custom software

We have two approaches:

1. First, this VM is connected to the Internet. If the software you need has a downloadable installer, you can open a web browser in the RDP session, download the software, and install it. 
2. If your software is custom, you can copy it from your local machine over to the VM to install it.

Initialize data disks

• Any additional drives you create from scratch will need to be initialized and formatted with the Disk Management tool

Configure network settings

• By default, new VMs are locked down.
• Apps can make outgoing requests, but the only inbound traffic allowed is from :
  • the virtual network (e.g. other resources on the same local network)
  • and from Azure's Load Balancer (probe checks).

Manage VMs with the Azure CLI

Create a virtual machine

The Azure CLI includes the vm command to work with VMs. The most common subcommands include:

az vm create is used to create a virtual machine in a resource group. There are several parameters but the four parameters that must be supplied are:

Here is an example:

az vm create \
  --resource-group [sandbox resource group name] \
  --location westus \
  --name SampleVM \
  --image UbuntuLTS \
  --admin-username azureuser \
  --generate-ssh-keys \
  --verbose

• add the --verbose flag to see progress while the VM is being created.
• the Azure CLI tool waits while the VM is being created. You can add the --no-wait option to return immediately
• We are specifying the administrator account name through the --admin-username flag to be azureuser. If you omit this, the az vm create command will use your current user name.
• generate-ssh-keys parameter is used for Linux distributions and creates a pair of security keys that can be used for SSH.
• Two files are placed into the .ssh folder on both your machine and in the VM.
• If you already have an SSH key named id_rsa in the target folder, then it will be used rather than having a new key generated.
• Once finished, you will get a JSON response which includes the current state of the virtual machine and its public and private IP addresses assigned by Azure:

{
  "fqdns": "",
  "id": "/subscriptions/20f4b944-fc7a-4d38-b02c-900c8223c3a0/resourceGroups/Learn-2568d0d0-efe3-4d04-a08f-df7f009f822a/providers/Microsoft.Compute/virtualMachines/SampleVM",
  "location": "westus",
  "macAddress": "00-0D-3A-58-F8-45",
  "powerState": "VM running",
  "privateIpAddress": "10.0.0.4",
  "publicIpAddress": "40.83.165.85",
  "resourceGroup": "2568d0d0-efe3-4d04-a08f-df7f009f822a",
  "zones": ""
}

Explore other VM images

This will output the most popular images that are part of an offline list built into the Azure CLI. 

az vm image list --output table

You can get a full list by adding the --all flag to the command. it is helpful to filter the list with the --publisher, --sku or –-offer options.

az vm image list --sku Wordpress --output table --all

az vm image list --publisher Microsoft --output table --all

Some images are only available in certain locations

az vm image list --location eastus --output table

you can also create and upload your own custom images to create VMs based on unique configurations

Sizing VMs properly

The available sizes change based on the region you're creating the VM in.

az vm list-sizes --location eastus --output table

You can specify size of the VM in the creation command:

az vm create \
    --resource-group [sandbox resource group name] \
    --name SampleVM2 \
    --image UbuntuLTS \
    --admin-username azureuser \
    --generate-ssh-keys \
    --verbose \
    --size "Standard_DS5_v2"

• If you didn't specify a size when creating VM, Azure selected a default general-purpose size of Standard_DS1_v2
• Your subscription tier enforces limits on how many resources you can create, as well as the total size of those resources. For example: you are capped to 20 virtual CPUs with the pay-as-you-go subscription, and only 4 vCPUs for a free tier. 

Resize an existing VM

Before a resize is requested, we must check to see if the desired size is available in the cluster our VM is part of.

az vm list-vm-resize-options \
    --resource-group [sandbox resource group name] \
    --name SampleVM \
    --output table

• If the size we want isn't available in our cluster, but is available in the region, we can deallocate the VM. This command will stop the running VM and remove it from the current cluster without losing any resources.

Resize command:

az vm resize \
    --resource-group [sandbox resource group name] \
    --name SampleVM \
    --size Standard_D2s_v3

once it's done, it will return a new JSON configuration.

Query system and runtime information about the VM

This command will return all virtual machines defined in this subscription.

az vm list --output table

you can specify json (the default), jsonc (colorized JSON), or tsv (Tab-Separated Values) as the --output type

Getting the IP address

{
    "name": "Barney",
    "age": 25
}

az vm list-ip-addresses -n SampleVM -o table

Getting VM details

az vm show --resource-group [sandbox resource group name] --name SampleVM

This will return a fairly large JSON block with all sorts of information about the VM.

Adding filters to queries with JMESPath

• Instead, we can turn to a built-in query language for JSON called JMESPath.
• JMESPath is an industry-standard query language built around JSON objects.
• check out the online tutorial available on the JMESPath.org site.

For example, given the object:

{
  "people": [
    {
      "name": "Fred",
      "age": 28
    },
    {
      "name": "Barney",
      "age": 25
    },
    {
      "name": "Wilma",
      "age": 27
    }
  ]
}

For example, people[1] would return:

{
    "name": "Barney",
    "age": 25
}

For example, adding the qualifier people[?age > '25'] would return:

[
  {
    "name": "Fred",
    "age": 28
  },
  {
    "name": "Wilma",
    "age": 27
  }
]

by adding a select: people[?age > '25'].[name] that returns just the names:

[
  [
    "Fred"
  ],
  [
    "Wilma"
  ]
]

Filtering our Azure CLI queries

For example, we can retrieve the admin user name:

az vm show \
    --resource-group [sandbox resource group name] \
    --name SampleVM \
    --query "osProfile.adminUsername"

to retrieve all the IDs for your network interfaces, you can use the query:

az vm show \
    --resource-group [sandbox resource group name] \
    --name SampleVM \
    --query "networkProfile.networkInterfaces[].id"

• This query technique can be used to pull a value out of your Azure account and store it in an environment or script variable.
• Add flag --output tsv parameter (which can be shortened to -o tsv) to return the results in tab-separated values that only include the actual data values with tab separators.

Start and stop your VM with the Azure CLI

Stopping a VM

az vm stop \
    --name SampleVM \
    --resource-group [sandbox resource group name]

We can verify it has stopped by attempting to ping the public IP address, using ssh, or through the vm get-instance-view command.

Typing the following command into Azure Cloud Shell to see the current running state of your VM:

az vm get-instance-view \
    --name SampleVM \
    --resource-group [sandbox resource group name] \
    --query "instanceView.statuses[?starts_with(code, 'PowerState/')].displayStatus" -o tsv

This command should return VM stopped as the result.

Starting a VM

az vm start \
    --name SampleVM \
    --resource-group [sandbox resource group name]

You can verify the status should return VM running.

Restarting a VM

Use the vm restart command.

Install software on your VM

Install NGINX web server

SSH to your VM and use this command:

sudo apt-get -y update && sudo apt-get -y install nginx

Retrieve our default page

curl -m 10 <PublicIPAddress>

This command will fail because the Linux virtual machine doesn't expose port 80 (http).

Use the following command to open up port 80:

az vm open-port \
    --port 80 \
    --resource-group [sandbox resource group name] \
    --name SampleVM

Run the curl command again and it should return data.