Azure Linux Support Team

Why proactively ensuring you have access to GRUB and sysrq in your Linux VM could save you lots of down time?

Vittorio Franco Libertucci — Tue, 23 Oct 2018 10:25:15 GMT

Quite simply, to speed up the recovery time of your IaaS Linux Virtual Machine most recovery options will be possible if GRUB is configured.

Recovering Linux Virtual Machines in Azure

You will, at some point have to perform some type of recovery in your Linux VM unless you are very lucky, this inevitable. The reasons for performing a VM recovery are numerous and can be anything such as:

• corrupt file systems/kernel/MBR • failed kernel upgrades • incorrect fstab configurations • firewall configurations • lost password • mangled sshd configurations files • Networking configurations • Many others

You should take some time and verify if you can access GRUB on your VM's deployed in Azure. Official Microsoft docs Ensure you take backups of files before making changes. Watch this video to see how you can quickly recover you Linux VM once you have access to GRUB There are a number of methods to aid recovery however in a Cloud environment this has been challenging and time is of the essence we understand you need to restore services at lightning speed. With the introduction of the Azure Serial Console you have the power to interact with your Linux VM and manipulate many aspects including how the kernel will boot. The more experienced Linux/Unix sys admins will appreciate the single user and emergency modes that are accessible via the Azure Console making Disk Swap and VM deletion for many recovery scenarios redundant. The method of recovery depends on the problem you are facing, for example a lost or misplaced password can be reset through Azure Portal options -> Reset Password . This communicates with the Linux Guest other possible tools such as Custom Script and other mechanisms are available however these options require that the Linux waagent be up and in a healthy state. By ensuring that you have access to the Azure Serial Console and GRUB a password or an incorrect configuration change can take a matter of minutes instead of hours. You could even force the VM to boot from an alternative kernel should you have multiple kernels on disk in the scenario where your primary kernel becomes corrupt.

suggested order of recovery methods • Azure Serial Console • Disk Swap - can be automated using either:

Power Shell Recovery Scripts

bash Recovery Scripts

• Legacy Method

Challenge: Not all Linux Azure VMs are configured by default for GRUB access and neither are they all configured to be interrupted with the sysrq commands. Some older distros such as SLES 11 Linux VM's are not configured to display Login prompt in the Azure Serial Console In this current draft article I will review various distros and document findings on how to configure GRUB and what's the impact. (RH and SUSE to complete)

How to configure Linux VM to accept SysRq keys

Ubuntu GRUB configuration

Ubuntu 12.04

Force the kernel to a bash prompt

Single User mode

Ubuntu Recovery Mode

How to configure Linux VM to accept SysRq keys

The sysrq key is enabled on some newer Linux distros by default , whilst on other it could be configured for accepting values only for certain SysRq features and on others it could be disabled completely especially in many older versions. The SysRq feature is useful for rebooting a crashed or hung VM directly from the Azure Serial Console, I also find it extremely helpful in gaining access to the the GRUB menu, alternatively restarting a VM from another Portal window or ssh session might drop your current console connection thus expiring GRUB Timeouts to which are used to display the GRUB menu. The VM must be configured to accept a value of 1 for the kernel parameter which enables all functions of sysrq or 128 which allows reboot/poweroff [video width="1920" height="1080" mp4="https://msdnshared.blob.core.windows.net/media/2018/10/enable-sysrq-for-Azure-Serial-Console1.mp4"][/video] To configure the VM to accept a reboot via SysRq commands on the Azure Portal you will need to set a value of 1 for the kernel parameter kernel.sysrq For this configuration to persist a reboot it can be configured by adding entries to the file sysctl.conf echo kernel.sysrq=1 >> /etc/sysctl.conf To configure the kernel parameter dynamically sysctl -w kernel.sysrq=1 If you do not have root access or sudo is broken you can't enable SysRq from a shell prompt? You can enable the sysrq in such a scenario using the Portal. This can be beneficial if the sudoers.d/waagent file has become broken or an admin has deleted it. Using the Azure Portal Operations -> Run Command -> RunShellScript feature, requires the waagent process be healthy you can then inject this command to enable sysrq sysctl -w kernel.sysrq=1 ; echo kernel.sysrq=1 >> /etc/sysctl.conf Once completed you can then try accessing SysRq and should see that a reboot is possible.

Select Reboot and Send SysRq Command

The system should log a reset message such as this

Ubuntu GRUB configuration

By default you should be able to access GRUB by holding down Esc key during the VM boot, however in testing , the GRUB menu was not always presented. To force and keep the GRUB menu on screen in the Azure Serial Console you can test and use one of these 3 options. Option 1 update this file /etc/default/grub.d/50-cloudimg-settings.cfg and it will keep the GRUB menu on screen for ever how long you specify. GRUB_TIMEOUT=0 to GRUB_TIMEOUT=5 Option 2 We can also achieve similar behavior by making changes to this file /etc/default/grub and observe 3 seconds to hit Esc Comment these 2 lines: #GRUB_HIDDEN_TIMEOUT=0 #GRUB_HIDDEN_TIMEOUT_QUIET=true And add replace with GRUB_TIMEOUT_STYLE=countdown Option 3 This configuration gives you 5 seconds to hit Esc to acess GRUB and does perform a countdown to select the kernel Update the file /etc/default/grub GRUB_HIDDEN_TIMEOUT=5 #GRUB_HIDDEN_TIMEOUT_QUIET=true #GRUB_TIMEOUT_STYLE=countdown GRUB_TIMEOUT=10 For all these options you need to recreate GRUB configurations by running: update-grub

Ubuntu 12.04

Ubuntu 12.04 will allow to see serial console but does not allow to interact. You will not see a login: prompt. The settings required by each Ubuntu version for configuring serial console can be found here Testing 12.04 was sufficient to 1) Create a file called /etc/init/ttyS0.conf containing the following:

# ttyS0 - getty
#
# This service maintains a getty on ttyS0 from the point the system is
# started until it is shut down again.
start on stopped rc RUNLEVEL=[12345]
stop on runlevel [!12345]

respawn
exec /sbin/getty -L 115200 ttyS0 vt102

2) Ask upstart to start the getty

sudo start ttyS0

Red Hat GRUB configuration

RHEL 7.4+

The default /etc/default/grub configuration on these versions is adequately configured

GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL="serial console"
GRUB_SERIAL_COMMAND="serial"
GRUB_CMDLINE_LINUX="console=tty1 console=ttyS0 earlyprintk=ttyS0 rootdelay=300"
GRUB_DISABLE_RECOVERY="true"

Simply enable the SysRq key sysctl -w kernel.sysrq=1;echo kernel.sysrq=1 >> /etc/sysctl.conf;sysctl -a | grep -i sysrq

RHEL 7.2+

The file to modify is /etc/default/grub - a default config looks like this:

GRUB_TIMEOUT=1
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="console=tty1 console=ttyS0 earlyprintk=ttyS0 rootdelay=300"
GRUB_DISABLE_RECOVERY="true"

change the following lines in /etc/default/grub GRUB_TIMEOUT=1 to GRUB_TIMEOUT=5 GRUB_TERMINAL_OUTPUT="console" to GRUB_TERMINAL="serial console" also add this line: GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1" /etc/default/grub should now look similar to this:

GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL="serial console"
GRUB_CMDLINE_LINUX="console=tty1 console=ttyS0 earlyprintk=ttyS0 rootdelay=300"
GRUB_DISABLE_RECOVERY="true"

Complete and update grub configuration using grub2-mkconfig -o /boot/grub2/grub.cfg set the SysRq kernel parameter: sysctl -w kernel.sysrq=1;echo kernel.sysrq=1 >> /etc/sysctl.conf;sysctl -a | grep -i sysrq For the more adventurous: I also configured GRUB and SysRq using a single line via the Run Command feature it worked well but please backup your files before running this.

cp /etc/default/grub /etc/default/grub.bak; sed -i 's/GRUB_TIMEOUT=1/GRUB_TIMEOUT=5/g' /etc/default/grub; sed -i 's/GRUB_TERMINAL_OUTPUT="console"/GRUB_TERMINAL="serial console"/g' /etc/default/grub; echo "GRUB_SERIAL_COMMAND=\"serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1\"" >> /etc/default/grub;grub2-mkconfig -o /boot/grub2/grub.cfg;sysctl -w kernel.sysrq=1;echo kernel.sysrq=1 /etc/sysctl.conf;sysctl -a | grep -i sysrq

RHEL 6.x

The file to modify is /boot/grub/grub.conf make the necessary modifications and you should have a configuration like this (the timeout=15 is used for demo purpose a reasonable value is 5)

#boot=/dev/vda
default=0
timeout=15
splashimage=(hd0,0)/grub/splash.xpm.gz
#hiddenmenu
serial --unit=0 --speed=9600
terminal serial
terminal --timeout=5 serial console

The last line, terminal --timeout=5 serial console will further increase GRUB timeout by adding a prompt for 5 seconds “Press any key to continue.” displayed in the console. GRUB menu should appear on screen for the configured timeout=15 without the need to press Esc. Make sure to click in the Console in the Browser to make active the menu and select the required kernel

Force the kernel to a bash prompt

no password for root access in single user? If you do not have root password and single user requires you to have a root password, you can boot the kernel replacing the init program with a bash prompt - this can be achieved by appending init=/bin/bash Remount your / (root) file system RW using the command mount -o remount,rw / Now you can perform root password change or many other Linux configuration changes Restart the VM with /sbin/reboot -f

Single User mode

Alternatively you might need to access the VM in single user or emergency mode Select the kernel you wish to boot or interrupt using arrow keys Enter the desired mode by appending the keyword "single" or "1" to the line displaying "Ubuntu" on Red Hat systems you can also append rd.break

Ubuntu Recovery Mode

Additional recovery and clean up options are available for Ubuntu via GRUB however these are only are accessible if you configure kernel parameters accordingly. Failure to configure this kernel boot parameter would force the Recovery menu to be sent to the Azure Diagnostics and not to the Azure Serial Console. You can obtain access to the Ubuntu Recovery Menu by following these steps: Interrupt the BOOT Process and access GRUB menu Select Advanced Options for Ubuntu Select the line displaying (recovery mode) do not press enter but press "e" Locate the line that will load the kernel and substitute the last parameter "nomodeset" with destination as console=ttyS0 linux /boot/vmlinuz-4.15.0-1023-azure root=UUID=21b294f1-25bd-4265-9c4e-d6e4aeb57e97 ro recovery nomodeset to linux /boot/vmlinuz-4.15.0-1023-azure root=UUID=21b294f1-25bd-4265-9c4e-d6e4aeb57e97 ro recovery console=ttyS0 Press Ctrl-x to start and load the kernel If all goes well you will see these additional Options which can help perform other recovery options

/var/empty/sshd must be owned by root and not group or world-writable

Vittorio Franco Libertucci — Tue, 30 Jan 2018 07:28:22 GMT

Problem Statement:

You are attempting to ssh into a Linux VM and are unsuccessful. We have seen customers modify key sshd configuration files and upon restarting sshd or rebooting the VM connectivity is lost.

symptoms/errors experienced

The serial or console log may exhibt errors such as this: SuSE /var/empty must be owned by root and not group or world-writable. startproc: exit status of parent of /usr/sbin/sshd: 255 Failed CentOS Starting sshd: /var/empty/sshd must be owned by root and not group or world-writable. [FAILED]

Resolution options:

There are a few methods to change the permissions on a Linux file when you don’t have ssh access. 1 - Portal - Run a Linux Custom Script into the VM which issues the Linux commands chmod on the file that you need to change via portal.azure.com 2 - Manually - Delete the VM, keep the disks, mount OS disk to another temporary VM and update the files on the temporary VM. Recreate the VM from disk 3 - Serial Console - Use the Serial Console feature if available for the region your VM is located in 4 - CLI - The command Line Interface can also be used to run BASH commands into your VM

1. Portal

This option relies on theWAagent process being up. From the serial log or portal check if the agent looks to be up.

Navigate to Properties of the VM in the portal to see if the WAagent is ok

Navigate to portal.azure.com and locate you VM's settings. Go to "Extensions" -> "Add" select "Custom Script For Linux" Select the "Create" button Populate the "Script File" field with the attached script (download update_perms.txt to your local PC/MAC and rename to update_perms.sh) Note you might have to update the script to reflect your OS - this attached script runs on Red Hat variants. Notifications will update when the Script has completed - The STATUS should display Success In the guest VM the example script update_perms.sh , changes the permissions on the file /var/empty/sshd from 777 to 755 and sets the owner and group to root:root Wait for script to run. The script will take a few minutes. The Linux Agent receives the request and hands off to the correct extension. If you are able to ssh back into your VM and are curious to see what happened during executing of the update_params.sh, look in the direcory /var/log/azure/Microsoft.OSTCExtensions.CustomScriptForLinux/ and check the file extension.log

2. Manually attach your VM's disks to a temporary VM

Delete the VM, keep the disks, mount OS disk to another temporary VM and update the files (permissions and ownership) on the temporary VM. Recreate the VM from disk. Instructions on this method can be found here

3. Serial Console

At time of writing, Jan 2018, the feature of "Serial Console" was available only in West Central US The console will eventually be rolled out to all regions. This feature allows you to access the VM via a console as if you were "on-prem" or "bare metal" - permitting you to modify files without using Custom Script and without having to delete the VM.

4. CLI

The Command Line Interface (CLI) can be used to inject commands into a VM. There are 2 versions of CLI - latest documentation is located here In this example, using the newer CLI v2 version we are resetting the /var/empty/sshd permissions and ownership using the customScript extension via CLI. The sshd service is also restarted. You will need to replace yourvm with the name of Virtual Machine and yourrg with your Resource Group

vmname=yourvm;rg=yourrg;timestamp=`date +%d%Y%H%M%S`;az vm extension set --resource-group $rg --vm-name $vmname --name customScript --publisher Microsoft.Azure.Extensions --settings "{'commandToExecute': 'bash -c \'chmod 755 /var/empty/sshd;chown root:root /var/empty/sshd;systemctl start sshd;ps -eaf | grep sshd\'','timestamp': "$((timestamp))"}"

AZ CLI: How to delete and re-deploy a VM from VHD (unmanaged disk)

Marco A Bicca — Sun, 07 May 2017 19:17:04 GMT

You should be able to use the built-in CLI on the Azure portal, for more details on how to use it, please check: Azure Cloud Shell (quickstart) If not, you can always install it locally on your workstation or also using docker, for details, please follow the steps described here: Install the Azure CLI For the steps to work properly, please, make sure you are logged into the Azure CLI: az login NOTE: You will get a code which you then have to type in the box using the link https://aka.ms/devicelogin and proceeding with logging in to your Azure account. Once you go through that you can go back to the prompt. In this example, the following variables will be used: Resource Group: ARMGROUP Original VM Name: LinuxVM Location: southcentralus NIC Name: LinuxVM-NIC VHD URI: https://ARMSTORAGE.blob.core.windows.net/vhds/linuxvm.vhd If you want to keep some information from the VM before deleting: az vm show -g ARMGROUP -n LinuxVM > ORIGINAL_VM.txt Deleting a VM on ARM using the CLI: az vm delete -g ARMGROUP -n LinuxVM Recreating the VM from the Original VHD:

az vm create -g ARMGROUP -n LinuxVM --nics LinuxVM-NIC --use-unmanaged-disk --attach-os-disk https://storageaccount.blob.core.windows.net/vhds/linuxvmoriginal2017.vhd --size Standard_D1_V2 -l southcentralus --os-type Linux

After the command is executed you should be able to see the VM on the Azure Portal. You might need to re-enable diagnostics in the particular VM.

How to: Deploy a basic wordpress on Ubuntu 16.04 LTS on Azure

Marco A Bicca — Mon, 10 Apr 2017 22:16:36 GMT

How to deploy a simple Wordpress instance using Ubuntu 16.04 LTS, Nginx, MariaDB and Memcache in Azure. This article is destined to help understand a basic Wordpress deployment on Azure using Ubuntu 16.04 LTS, Nginx with PHP-fpm (for PHP support), MariaDB and Memcache In this example, we are deploying a Standard_F2S VM with premium disk, 2 CPU’s, 4GB of memory and also a local 7GB SSD (Ephemeral disk) which we will use for swap configuration (4GB). Considering you already have your VM deployed with the latest Ubuntu 16.04 LTS, the steps we will describe are: 1) Installing Nginx 2) Installing MariaDB 3) Installing PHP7 / PHP-FPM / Memcache 4) Installing Wordpress 5) Enabling Memcache 1) Installing Nginx sudo apt-get update sudo apt-get install nginx You can verify Nginx is correctly listening on port 80 by issuing: sudo netstat -anp | grep :80 tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN XXXX/nginx –g daemo This should indicate that Nginx is listening on port 80 on all IP’s 2) Installing MariaDB sudo apt-get install mariadb-server mariadb-client You will be asked to provide a password for the mariadb root account, please make sure you note that down. NOTE: We highly recommend using the command “sudo mysql_secure_installation” but keep in mind that doing that will add a validation plugin for MySQL passwords which require certain levels of security where you can choose from LOW to STRONG. 3) Installing PHP / PHP-FPM / Memcached sudo apt-get install php7.0-fpm sudo apt-get install php-memcache memcached sudo apt-get install php7.0-mysql php7.0-curl php7.0-gd php7.0-mbstring php7.0-mcrypt php7.0-xml php7.0-xmlrpc Turn off cgi.fix_pathinfo in php7.0-fpm: sudo sed -i.bak 's/;cgi.fix_pathinfo=1/cgi.fix_pathinfo=0/g' /etc/php/7.0/fpm/php.ini sudo systemctl restart php7.0-fpm.service 4) Installing Wordpress

Connect to MySQL and create a database and a user: mysql -u root -p CREATE DATABASE wordpress DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci; GRANT ALL ON wordpress.* TO ‘wpuser’@’localhost’ IDENTIFIED BY ‘password’; FLUSH PRIVILEGES; quit
sudo –i cd /var/www/html curl –O https://wordpress.org/latest.tar.gz tar xzvf latest.tar.gz cd wordpress mv * .. rm -rf latest.tar.gz wordpress chown –R www-data:www-data /var/www/html chmod g+w /var/www/html/wp-content chmod g+w /var/www/html/wp-content/themes chmod g+w /var/www/html/wp-content/plugins
Setup the Wordpress configuration file curl –s https://api.wordpress.com/secret-key/1.1/salt/ copy the content from the output vi /var/www/html/wp-config-sample.php
Replace the lines below with the output from the output above:define('AUTH_KEY', 'put your unique phrase here'); define('SECURE_AUTH_KEY', 'put your unique phrase here'); define('LOGGED_IN_KEY', 'put your unique phrase here'); define('NONCE_KEY', 'put your unique phrase here'); define('AUTH_SALT', 'put your unique phrase here'); define('SECURE_AUTH_SALT', 'put your unique phrase here'); define('LOGGED_IN_SALT', 'put your unique phrase here'); define('NONCE_SALT', 'put your unique phrase here');
Rename the config file: mv /var/www/html/wp-config-sample.php /var/www/html/wp-config.php
Edit the file again to configure MySQL and replace the variables with the corresponding values:define('DB_NAME', 'wordpress'); /** MySQL database username */ define('DB_USER', 'wpuser'); /** MySQL database password */ define('DB_PASSWORD', 'password');
Edit Nginx default website configuration: vi /etc/nginx/sites-available/default
Replace the location / area in the file with the content below, including the other block:

location / { # try_files $uri $uri/ =404; try_files $uri $uri/ /index.php?q=$uri&$args; } location ~ \.php$ { try_files $uri =404; fastcgi_pass unix:/var/run/php/php7.0-fpm.sock; include /etc/nginx/fastcgi_params; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; } Check to see if nginx configuration is correct and if it is, restart the service: nginx –t systemctl restart nginx Assuming you have a Network Security Group with a rule to allow port 80 inbound to the VM, you can then access Wordpress from the external IP of the VM and finish the setup: http://IP 5) Enabling MemCache sudo -i cd /tmp wget https://downloads.wordpress.org/plugin/memcached.3.0.1.zip apt-get install unzip unzip memcached.3.0.1.zip cd memcached cp object-cache.php /var/www/html/wp-content chown www-data:www-data /var/www/html/wp-content/object-cache.php To check stats in memcached you can use: watch "echo stats | nc 127.0.0.1 11211" After finishing the setup through the web interface in Wordpress you should be able to access the website at the http://IP address or the management interface at http://IP/wp-admin using the user/password that you configured during the Wordpress installation. Happy blogging!

How to: Add SWAP to Linux VM's on Azure

Marco A Bicca — Mon, 10 Apr 2017 21:58:48 GMT

Every virtual machine (VM) on Azure has what we call a temporary (ephemeral) disk which is recommended to be used ONLY as temporary storage and that includes SWAP files or data that does not need to be available upon a reboot or saved , the data stored in this drive will be lost. To create a swap file in the directory that's defined by the ResourceDisk.MountPoint parameter, you can update the /etc/waagent.conf file by setting the following three parameters: ResourceDisk.Format=y ResourceDisk.EnableSwap=y ResourceDisk.SwapSizeMB=xx Where the size is in MB, so for instance, to create a SWAP file of 4GB you could use these lines: ResourceDisk.Format=y ResourceDisk.EnableSwap=y ResourceDisk.SwapSizeMB=4096 Before creating a SWAP file using the WAAgent, we also recommend looking at how much space you have available for the temporary disk by checking this link where on each category of VM we display the local SSD or local HDD storage available. References: How to add a swap file in Linux Azure virtual machines Optimizing your Linux VM on Azure

How to: Resize Linux osDisk partition on Azure

Marco A Bicca — Mon, 03 Apr 2017 21:02:25 GMT

Most Linux images on Azure will by default have 30GB size for the osDisk and that is something that you might want to resize moving forward.

NOTE: We recommend to store applications and data on a separate data disk not only because of space but also for performance reasons.

Resizing the partition is really simple on Azure and we can be done by the following procedure: a) Stopping and deallocating the VM on the portal or Azure CLI / PowerShell; b) Editing the disk on the portal and setting the size to the desired value; c) Updating the partition information within the Linux OS which might be slightly different depending on the partitions and distribution but the concept is very similar , which is basically deleting and recreating the root partition.

NOTE: You must create a backup or a snapshot of the VHD prior to the resizing steps , you can use http://storageexplorer.com or also AZCopy once the VM is deallocated when using unmanaged disks. For managed disks, you can use the Azure Portal to create a snapshot of the disk.

Steps to make the required partition changes are listed below per distribution and have been tested in the specific releases listed: For Red Hat 7.3 / CentOS 7.3 and Oracle 7.3:

1) Login to the VM using SSH, we can check the size of the disk by using: sudo dmesg | grep -i sda

We should see something similar to the output below (assuming you changed the size to 100GB on the portal for the disk): [ 3.100914] sd 2:0:0:0: [sda] 209715200 512-byte logical blocks: (107 GB/100 GiB) 2) To proceed with the partition resize, we will use: sudo fdisk /dev/sda

type: p this will show both partitions /dev/sda1 and /dev/sda2 which are basically partitions 1 and 2

type: d then 2 (to delete partition 2) type: n then p, 2 (to recreate partition 2) you can accept the default values type: w (to save the new partition) type: q (to exit fdisk) sudo reboot (to reboot the VM so the partition is updated)

3) To finalize the resize, after the reboot, execute the command:

For Red Hat 7.3 and CentOS 7.3: sudo xfs_growfs /dev/sda2

For Oracle 7.3: sudo btrfs filesystem resize max /

4) Use the command: df -h to verify its size: Filesystem Size Used Avail Use% Mounted on /dev/sda2 100G 1.6G 98G 2% /

For Ubuntu 16.04 LTS: No extra steps are necessary, simply deallocating the VM, changing the size on the portal and starting the VM backup will work. You can verify the size after the VM is up by using: df -h Filesystem Size Used Avail Use% Mounted on /dev/sda1 97G 1.4G 96G 2% / For Debian 8:

1) Login to the VM using SSH, we can check the size of the disk by using: sudo dmesg | grep -i sda

2) To proceed with the partition resize, we will use: sudo fdisk /dev/sda

type: p this will show partition /dev/sda1

type: d type: n then p, 1 (to recreate partition 1) you can accept the default values NOTE: if you receive a message about deleting or keeping a signature on the disk, just answer N type: w (to save the new partition) type: q (to exit fdisk) sudo reboot (to reboot the VM so the partition is updated) 3) To finalize the resize, after the reboot, execute the command: resize2fs /dev/sda1 4) Use the command: df -h to verify its size: Filesystem Size Used Avail Use% Mounted on /dev/sda1 99G 698M 94G 1% / For SUSE 12 SP2 (Premium):

1) Login to the VM using SSH, we can check the size of the disk by using: sudo dmesg | grep -i sda

2) To proceed with the partition resize, we will use: sudo fdisk /dev/sda type: p this will show both partitions /dev/sda1 and /dev/sda2 which are basically partitions 1 and 2

3) To finalize the resize, after the reboot, execute the command: resize2fs /dev/sda2

4) Use the command: df -h to verify its size: Filesystem Size Used Avail Use% Mounted on /dev/sda2 98G 1.2G 92G 2% /

For CoreOS (Stable): No extra steps are necessary, simply deallocating the VM, changing the size on the portal and starting the VM backup will work. CoreOS has a few extra partitions and the extra space will be added to the root partition (usually /dev/sda9).

Azure Linux Extensions: Custom Script for Linux

Marco A Bicca — Sun, 12 Feb 2017 16:45:03 GMT

You have probably seen some extensions that are available on Azure, some of them listed/documented here: Azure Linux Extensions In this article, we will cover a few examples that you can use with the "Custom Script for Linux" extension. This extension basically allows you to run a script inside a Linux VM, so as long as the Linux Agent (WAAGENT) is working correctly you should be able to use this extension without problems. You can run this extension from the portal , so on your Azure Resource Manager (ARM) VM, you can access that by clicking: VM Name > Extensions > Add > Select "Custom Script for Linux" > Create After that all you will need is the upload the script you want to run and then change the "Command" line to the appropriate one to execute that file, so in this case we have created a bash script called adduser.sh , in this case the command line would be changed to: bash adduser.sh like in the screenshot below: Click OK and the script will start to be deployed and executed inside your Linux VM. In most cases, you will need to be familiar with shell scripting in Linux and also we highly recommend testing the script extensively before you run it and keep in mind to create a script that will handle any prompts or questions, since you won't have access to it while it gets executed. As a few simple examples, let's assume you want to deal with any of these issues: 1) Disabling the UFW firewall on a Ubuntu Linux VM that is blocking access to the VM 2) Fixing wrong permissions on SSH Host Keys which are not allowing you to login to a given VM 3) Manually adding a new account to a VM when using the portal is not working properly for some reason 1) Disabling the UFW firewall on a Ubuntu Linux VM Create an empty text file called "disableufw.sh" and add these lines into it: #!/bin/bash ufw disable logger "ufw disabled by custom script" This script will basically run the command ufw disable which disables the Ubuntu firewall and it will also log a message "ufw disabled by custom script" in the system logs using the logger command. 2) Fixing wrong permissions on SSH Host Key files Create an empty text file called "fixsshkeys.sh" and add these lines into it:

#!/bin/bash
chmod 640 /etc/ssh/ssh_host_ecdsa_key ssh_host_ed25519_key
chmod 644 /etc/ssh/ssh_host_ecdsa_key.pub ssh_host_ed25519_key.pub ssh_host_rsa_key.pub
chmod 600 /etc/ssh/ssh_host_rsa_key
logger "SSH keys permission reset successfully"

This script will reset the the file permissions on the specified files above to its defaults. 3) Manually adding a new account with sudo access and SSH Key. Create an empty text file called "adduser.sh" and add these lines into it: For CentOS / RHEL and Oracle distributions

#!/bin/bash
# Make sure you use a username that is lowercase.
USERNAME="rescue"
# Creates the user account using the name provided
useradd -m $USERNAME
logger "Azure custom script - adding $USERNAME"
# Creates SSH configuration folder for user.
mkdir -p /home/$USERNAME/.ssh
logger "Azure custom script - creating SSH configuration folder"
# Adding the SSH-Key data to the authorized_keys file for the given user.
# Replace the content "SSH-PUBLIC-KEY" with the public key for your user.
echo SSH-PUBLIC-KEY >> /home/$USERNAME/.ssh/authorized_keys
logger "Azure custom script - adding SSH key for user $USERNAME"
# Adding user account to sudoers
echo "$USERNAME ALL = (ALL) ALL" > /etc/sudoers.d/$USERNAME

For SUSE distributions

#!/bin/bash
# Input your desired password# Make sure you use a username that is lowercase.
USERNAME="rescue"
# Creates the user account using the name provided
useradd -m $USERNAME -G users
logger "Azure custom script - adding $USERNAME"
# Creates SSH configuration folder for user
mkdir -p /home/$USERNAME/.ssh
logger "Azure custom script - creating SSH configuration folder"
# Adding the SSH-Key data to the authorized_keys file for the given user.
# Replace the content "SSH-PUBLIC-KEY" with the public key for your user.
echo "SSH-PUBLIC-KEY" >> /home/$USERNAME/.ssh/authorized_keys
logger "Azure custom script - adding SSH key for user $USERNAME"
# Adding user account to sudoers.
echo "$USERNAME ALL = (ALL) NOPASSWD: ALL" > /etc/sudoers.d/$USERNAME
logger "Azure custom script - creating $USERNAME sudoers file"

For Ubuntu / Debian distributions

#!/bin/bash
# Make sure you use a username that is lowercase.
USERNAME="rescue"
# Creates the user account using the name provided.
useradd -m -G sudo $USERNAME
logger "Azure custom script - adding $USERNAME"
# Creates SSH configuration folder for user.
mkdir -p /home/$USERNAME/.ssh
logger "Azure custom script - creating SSH configuration folder"
# Adding the SSH-Key data to the authorized_keys file for the given user.
# Replace the content "SSH-PUBLIC-KEY" with the public key for your user.
echo "SH-PUBLIC-KEY" >> /home/$USERNAME/.ssh/authorized_keys
logger "Azure custom script - adding SSH key for user $USERNAME"

The scripts above you basically create an account named "rescue" and it will use the SSH public key added to login, just make sure you replace the field "SSH-PUBLIC-KEY" with the contents of your SSH public key. For more reference around the extension and other ways to execute it such as using Azure PowerShell or Azure CLI, you can check this page: Custom Script for Linux

Linux Recovery: Fixing non-boot issues related to Kernel problems using chroot

Marco A Bicca — Sun, 09 Oct 2016 21:19:30 GMT

In a previous article, it was described how to manually fix kernel related issues that could cause a Linux VM to not boot properly. There is also an option where you can use steps to use CHROOT and then use tools to automatically fix these same issues without the need to manually edit files. Assuming you have your system ready after going to the steps described in this article, meaning you just went through the sequence of steps and you finally executed: chroot /rescue For a quick example, let's assume you had a non-boot scenario after updating the kernel on a CentOS 7.2 to version: 3.10.0-327.36.1.el7.x86_64 You can quickly see which kernels you still have installed by running: cd /rescue/boot/grub2/ grep -i linux16 grub.cfg The output should be similar to: linux16 /boot/vmlinuz-3.10.0-327.36.1.el7.x86_64 root=UUID=3b5d406c-7fa6-458f-815c-9684bb9fe148 ro console=tty1 console=ttyS0,115200n8 earlyprintk=ttyS0,115200 rootdelay=300 net.ifnames=0 LANG=en_US.UTF-8 linux16 /boot/vmlinuz-3.10.0-327.18.2.el7.x86_64 root=UUID=3b5d406c-7fa6-458f-815c-9684bb9fe148 ro console=tty1 console=ttyS0,115200n8 earlyprintk=ttyS0,115200 rootdelay=300 net.ifnames=0 Which means you have the current one 36.1 as well as the 18.2 installed. You can then easily roll back to 18.2 by executing: grep -i menuentry grub.cfg Output should be similar to: menuentry 'CentOS Linux (3.10.0-327.36.1.el7.x86_64) 7 (Core)' --class centos --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-3.10.0-327.18.2.el7.x86_64-advanced-3b5d406c-7fa6-458f-815c-9684bb9fe148' { menuentry 'CentOS Linux (3.10.0-327.18.2.el7.x86_64) 7 (Core)' --class centos --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-3.10.0-327.18.2.el7.x86_64-advanced-3b5d406c-7fa6-458f-815c-9684bb9fe148' { With that you can then execute: grub2-editenv list This will display: saved_entry=CentOS Linux (3.10.0-327.36.1.el7.x86_64) 7 (Core) grub2-set-default "CentOS Linux (3.10.0-327.18.2.el7.x86_64) 7 (Core)" This will change the kernel back to the 18.2 version, now after this you can safely go through the final steps to exit the CHROOT environment: The steps to exit the chroot environment and umount everything are: exit cd / umount /rescue/proc/ umount /rescue/sys/ umount /rescue/dev/pts umount /rescue/dev/ For Debian and Ubuntu distributions you will need to umount (run): umount /rescue/run/ umount /rescue The finally you can detach the disk and use one of our articles to rebuild the VM using either PowerShell or Azure CLI: Azure PowerShell: How to delete and re-deploy a VM from VHD Azure CLI: How to delete and re-deploy a VM from VHD

Linux Recovery: Manually fixing non-boot issues related to Kernel problems

Marco A Bicca — Sun, 09 Oct 2016 20:37:23 GMT

You have a Linux VM that had kernel changes applied recently such as a kernel upgrade and is no longer starting up properly due to kernel errors during the boot process.

Kernel messages will vary; some examples could be:

no root device found,
kernel timeouts,
null pointer,
kernel panic errors.

Most of the time the recovery steps will be similar requiring that you need to either install a newer kernel or simply roll-back to the previous version manually.

Example of error messages that you can see

A) No root device found

dracut Warning: No root device "block:/dev/disk/by-uuid/6d089360-3e14-401d-91d0-378f3fd09332" found dracut Warning: Boot has failed. To debug this issue add "rdshell" to the kernel command line. dracut Warning: Signal caught! Kernel panic - not syncing: Attempted to kill init! Pid: 1, comm: init Not tainted 2.6.32-504.12.2.el6.x86_64 #1 Call Trace: [] ? panic+0xa7/0x16f [] ? do_exit+0x862/0x870 [] ? fput+0x25/0x30 [] ? do_group_exit+0x58/0xd0 [] ? sys_exit_group+0x17/0x20 [] ? system_call_fastpath+0x16/0x1b

B) Example of a kernel timeout error

INFO: task swapper:1 blocked for more than 120 seconds. Not tainted 2.6.32-504.8.1.el6.x86_64 #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. swapper D 0000000000000000 0 1 0 0x00000000 ffff88010f64fde0 0000000000000046 ffff88010f64fd50 ffffffff81074f95 0000000000005c2f ffffffff8100bb8e ffff88010f64fe50 0000000000100000 0000000000000002 00000000fffb73e0 ffff88010f64dab8 ffff88010f64ffd8 Call Trace: [] ? __call_console_drivers+0x75/0x90 [] ? apic_timer_interrupt+0xe/0x20 [] ? vprintk+0x251/0x560 [] schedule_timeout+0x192/0x2e0 [] ? process_timeout+0x0/0x10 [] schedule_timeout_uninterruptible+0x1e/0x20 [] msleep+0x20/0x30 [] prepare_namespace+0x30/0x1a9 [] kernel_init+0x2e1/0x2f7 [] child_rip+0xa/0x20 [] ? kernel_init+0x0/0x2f7 [] ? child_rip+0x0/0x20

C) Example of a kernel null point error

Pid: 242, comm: async/1 Not tainted 2.6.32-504.12.2.el6.x86_64 #1 Call Trace: [] ? kmem_cache_create+0x538/0x5a0 [] ? mutex_lock+0x1e/0x50 [] ? attribute_container_add_device+0x104/0x150 [] ? storvsc_device_alloc+0x4e/0xa0 [hv_storvsc] [] ? scsi_alloc_sdev+0x1fc/0x280 [] ? scsi_probe_and_add_lun+0x4d9/0xe10 [] ? kobject_set_name_vargs+0x6d/0x70 [] ? mutex_lock+0x1e/0x50 [] ? attribute_container_add_device+0x104/0x150 [] ? get_device+0x19/0x20 [] ? scsi_alloc_target+0x2d0/0x300 [] ? __scsi_scan_target+0x121/0x740 [] ? scsi_scan_channel+0x87/0xb0 [] ? scsi_scan_host_selected+0xb0/0x190 [] ? do_scsi_scan_host+0x91/0xa0 [] ? do_scan_async+0x1c/0x150 [] ? async_thread+0x116/0x2e0 [] ? default_wake_function+0x0/0x20 [] ? async_thread+0x0/0x2e0 [] ? kthread+0x9e/0xc0 [] ? child_rip+0xa/0x20 [] ? kthread+0x0/0xc0 [] ? child_rip+0x0/0x20 BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 IP: [] storvsc_device_destroy+0x20/0x50 [hv_storvsc] PGD 0

D) Example of a Kernel Panic error

Invalid op code: 0000 [#2] [11427.908676] --- end trace 61a458bb863d7f0f ]--- Kernel panic - not syncing: attempted to kill the idle task!

Recovering from these issues

In all cases we will need to go through the normal procedure of deleting the affected Linux VM, keep its OSDisk, attach it to a new VM running the same version as the impacted VM or at least the same distribution and then, depending on the issue we will have two options of repairing which are described below.

1) Simply editing configuration files to roll-back the kernel and boot from a previous working setup, Linux boot loaders usually have more than one entry which defines which kernel to use for booting, that usually gets updated every time you perform an upgrade to reference the new installed kernel.

2) Go through a CHROOT process, by attaching the impacting VM OSDisk to a temporary new VM and running known tools (apt-get/yum/zypper) to install/reinstall a kernel. This is the easiest and usually fastest way to repair since you do not have to manually edit files.

Example, a Linux VM running CentOS 6.6 has loaded a kernel version 2.6.32-504.16.2.el6.x86_64, this can be seen with the command: uname -a

Linux vfldev 2.6.32-504.16.2.el6.x86_64 #1 SMP Wed Apr 22 06:48:29 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

The boot loader configuration file that knows of the kernel version to load, in the case for CentOS the file is /boot/grub/grub.conf and on newer versions /boot/grub2/grub.cfg

Looking closer at the grub.conf we can see that the current loaded version is referenced in the first 4 lines highlighted in blue.

1) Title > Simply the title that shows up in the menu (not applicable in cloud environments) 2) The root disk 3) The kernel command line being loaded at boot time and its parameters 4) The path for the initrd file that will be loaded to boot and usually matches the kernel too

An example of grub.conf with reference to 4 Linux versions:

title CentOS (2.6.32-504.16.2.el6.x86_64) root (hd0,0) kernel /boot/vmlinuz-2.6.32-504.16.2.el6.x86_64 ro root=UUID=6d089360-3e14-401d-91d0-378f3fd09332 rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM numa=off console=ttyS0 earlyprintk=ttyS0 rootdelay=300 crashkernel=auto initrd /boot/initramfs-2.6.32-504.16.2.el6.x86_64.img

title CentOS (2.6.32-504.12.2.el6.x86_64) root (hd0,0) kernel /boot/vmlinuz-2.6.32-504.12.2.el6.x86_64 ro root=UUID=6d089360-3e14-401d-91d0-378f3fd09332 rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM numa=off console=ttyS0 earlyprintk=ttyS0 rootdelay=300 crashkernel=auto initrd /boot/initramfs-2.6.32-504.12.2.el6.x86_64.img

title CentOS (2.6.32-504.8.1.el6.x86_64) root (hd0,0) kernel /boot/vmlinuz-2.6.32-504.8.1.el6.x86_64 ro root=UUID=6d089360-3e14-401d-91d0-378f3fd09332 rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM numa=off console=ttyS0 earlyprintk=ttyS0 rootdelay=300 crashkernel=auto initrd /boot/initramfs-2.6.32-504.8.1.el6.x86_64.img

title CentOS (2.6.32-431.29.2.el6.x86_64) root (hd0,0) kernel /boot/vmlinuz-2.6.32-431.29.2.el6.x86_64 ro root=UUID=6d089360-3e14-401d-91d0-378f3fd09332 rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM numa=off console=ttyS0 earlyprintk=ttyS0 rootdelay=300 initrd /boot/initramfs-2.6.32-431.29.2.el6.x86_64.img

To change the boot loader (grub.conf) and force the Linux VM to load a different kernel manual intervention is required, below you can find steps to do that change.

NOTE: We highly recommend making a backup of the VHD from the inaccessible VM before going through the steps for the recovery process, you can make a backup of the VHD by using Microsoft Storage Explorer, available at http://storageexplorer.com

A = Original VM (Inaccessible VM) B = New VM (New Recovery VM)

Stop VM A via Azure Portal
For Resource Manager VM, we recommend to save the current VM information before deleting
- Azure CLI: azure vm show ResourceGroupName LinuxVmName > ORIGINAL_VM.txt
- Azure PowerShell: Get-AzureRmVM -ResourceGroupName $rgName -Name $vmName
Delete VM A BUT select “keep the attached disks” NOTE: The option to keep the attached disks is only available for classic deployments, for Resource Manager deleting a VM will always keep its OSDisk by default.
Once the lease is cleared, attach the Data Disk from A to VM B via the Azure Portal, Virtual Machines, Select “B”, Attach Disk
On VM “B” eventually the disk will attach and you can then mount it.
Locate the drive name to mount, on VM “B” look in relevant log file note each Linux is slightly different.
- grep SCSI /var/log/kern.log (ubuntu, debian) grep SCSI /var/log/messages (centos, suse, oracle, redhat)
Mount the attached disk onto mountpoint /rescue df -h mkdir /rescue For Red Hat 7.2+ mount -o nouuid /dev/sdc2 /rescue For CentOS 7.2+ mount -o nouuid /dev/sdc1 /rescue For Debian 8.2+, Ubuntu 16.04+, SUSE 12 SP4+ mount /dev/sdc1 /rescue
Mount the OS disk on /rescue on VM B, and modify the /rescue/boot/grub.conf file.Modified grub.conf

#title CentOS (2.6.32-504.16.2.el6.x86_64) #root (hd0,0) #kernel /boot/vmlinuz-2.6.32-504.16.2.el6.x86_64 ro root=UUID=6d089360-3e14-401d-91d0-378f3fd09332 rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM numa=off console=ttyS0 earlyprintk=ttyS0 rootdelay=300 crashkernel=auto #initrd /boot/initramfs-2.6.32-504.16.2.el6.x86_64.img

Another item to check is if the device specified by the UUID value from the boot loader file, grub.conf exists.

Taking our example of the OS disk mounted on /rescue look in /rescue/dev/disk/by-uuid

We can clearly see that there is a corresponding UUID file found on disk highlighted in pink that is referenced in the grub.conf file

The file is actually a symbolic link as denoted by the l at the start of the attributes lrwxrwxrwx and is pointing to the OSDisk sda1

Even if this file is missing, testing deletion of the symbolic link is recreated when the system starts up.

It is possible to create symbolic links manually you would need to know if sda1 is your boot device and the corresponding UUID.

cd /rescue/dev/disk/by-uuid ln -s ../../sda1 6d089360-3e14-401d-91d0-378f3fd09332

The sda1 file is known as a block device in Linux seen by the ls output and denoted by a b in the attributes brw-rw---- or by running the file command.

You could also check that this file is present again testing on a centos 6.5 this file also gets recreated if missing.

cd /rescue/dev/disk/by-uuid ls -ltr ../../sda1 brw-rw---- 1 root disk 8, 1 Apr 30 14:37 ../../sda1

file ../../sda1 ../../sda1: block special

2) If the VM still does not boot after having tried to boot from a previous kernel next you could try to rebuild the initramfs and copy over a new compressed image of the Linux kernel this is seen in the example in the grub file.

Initramfs file initrd /boot/initramfs-2.6.32-504.16.2.el6.x86_64.img

Kernel file /boot/vmlinuz-2.6.32-504.16.2.el6.x86_64

Normally, in an on-premise environment you would boot a system from a recovery cd. In Cloud environments we have to attach the OS disk to a temporary VM of the same OS and Version for recovering or manipulating system files for a no-boot scenario, this is reinforced even more if we are going to attempt a copy or recreate initramfs and kernel files

Once the OS disk is attached to a temp VM on /rescue (first secure any data by copying off the OS disk)

Revert previously made changes to grub.conf if you had commented out entries referring to the first kernel to boot re-instate them.

Then proceed to rebuild initramfs.

mv /rescue/boot/initramfs-2.6.32-504.8.1.el6.x86_64.img /rescue/boot/initramfs-2.6.32-504.8.1.el6.x86_64.old-img

On the temporary CentOS 6.6 Linux VM we were unable to locate the exact same initramfs, hence building and using the latest version available on the temp.

dracut /rescue/boot/initramfs-2.6.32-504.8.1.el6.x86_64.img 2.6.32-504.8.1.el6.x86_64

Then to be safe copy relevant vmlinuz file and finally update the grub.conf to reflect new kernel values: ls -ltr /lib/modules/

drwxr-xr-x. 7 root root 4096 Apr 14 2014 2.6.32-431.11.2.el6.x86_64 drwxr-xr-x. 7 root root 4096 Jun 4 2014 2.6.32-431.17.1.el6.x86_64 drwxr-xr-x. 7 root root 4096 Sep 25 2014 2.6.32-431.29.2.el6.x86_64 drwxr-xr-x. 7 root root 4096 Nov 18 18:54 2.6.32-504.1.3.el6.x86_64 drwxr-xr-x. 7 root root 4096 Mar 25 19:28 2.6.32-504.12.2.el6.x86_64

dracut /rescue/boot/initramfs-2.6.32-504.12.2.el6.x86_64.img 2.6.32-504.12.2.el6.x86_64

ls -ltr /rescue/boot/initramfs-2.6.32-504.12.2.el6.x86_64.img -rw-------. 1 root root 19354168 May 6 15:39 /rescue/boot/initramfs-2.6.32-504.12.2.el6.x86_64.img

cp /boot/vmlinuz-2.6.32-504.12.2.el6.x86_64 /rescue/boot/

ls -ltr /rescue/boot/vmlinuz*

-rwxr-xr-x. 1 root root 4128368 Nov 22 2013 /rescue/boot/vmlinuz-2.6.32-431.el6.x86_64 -rwxr-xr-x. 1 root root 4128688 Jan 3 2014 /rescue/boot/vmlinuz-2.6.32-431.3.1.el6.x86_64 -rwxr-xr-x. 1 root root 4129872 May 7 2014 /rescue/boot/vmlinuz-2.6.32-431.17.1.el6.x86_64 -rwxr-xr-x. 1 root root 4131984 Sep 9 2014 /rescue/boot/vmlinuz-2.6.32-431.29.2.el6.x86_64 -rwxr-xr-x. 1 root root 4153008 Jan 28 21:40 /rescue/boot/vmlinuz-2.6.32-504.8.1.el6.x86_64 -rwxr-xr-x. 1 root root 4152720 May 6 15:44 /rescue/boot/vmlinuz-2.6.32-504.12.2.el6.x86_64

vi /rescue/boot/grub/grub.conf

title CentOS (2.6.32-504.12.2.el6.x86_64) root (hd0,0) kernel /boot/vmlinuz-2.6.32-504.12.2.el6.x86_64 ro root=UUID=6d089360-3e14-401d-91d0-378f3fd09332 rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet numa=off console=ttyS0 earlyprintk=ttyS0 rootdelay=300 initrd /boot/initramfs-2.6.32-504.12.2.el6.x86_64.img

cd / umount /rescue

Detach the disk from VM B via the Azure portal
Recreate the original VM A from the repaired VHD

For a Classic VM:

Recreate the original VM A (Create VM from Gallery, Select My Disks) you will see the Disk referring to VM A – Select the original Cloud Service name.

For a Resource Manager VM you will need to use either Powershell or Azure CLI tools, the articles below have steps to recreate a VM from its original VHD:

Azure PowerShell: How to delete and re-deploy a VM from VHD Azure CLI: How to delete and re-deploy a VM from VHD

Unsupported: How to install xRDP for GUI remote access on Linux

Marco A Bicca — Mon, 26 Sep 2016 18:44:19 GMT

You want to access a Linux VM on Azure using a GUI through Remote Desktop or similar tools.

The installation and configuration of 3rd party tools are beyond the scope of Azure Linux support but we have here a few notes and best practices to enable xRDP if needed.

There are other ways of achieving similar results with other tools, in this article we are covering specifically xRDP.

When using the standard configuration in xRDP the port needed to be opened by using an Endpoint (Classic) or Network Security Groups (Resource Manager) will be 3389 TCP, the same used for Microsoft Windows remote access.

There are a few security implications to keep in mind when enabling xRDP but we will be covering a few best practices at the end of this article.

Steps to install xRDP are listed below.

For Ubuntu/Debian: sudo -i apt-get update apt-get install xrdp

For Red Hat, CentOS and Oracle: You need to install a 3rd party repository (EPEL) which is usually not supported by the vendors but here you can find more information: https://fedoraproject.org/wiki/EPEL

Make sure you install the one for your specific version (6.x/7.x).

After that repository is successfully installed then you can proceed with the steps below.

Red Hat / CentOS and Oracle version 7.x: sudo -i yum install xrdp tigervnc-server xterm systemctl enable xrdp systemctl enable xrdp-sesman

Red Hat / CentOS and Oracle version 6.x sudo -i yum install xrdp tigervnc-server xterm service xrdp restart chkconfig xrdp on

Once you can connect using xRDP and you have xterm opened, you can test any graphical application that you want, for example, you can try xclock.

Best practices to improve security

A) Using an SSH local tunnel to avoid opening extra ports and encrypt traffic. By tunneling the xRDP access through SSH you not only encrypt all traffic but you also avoid opening up extra ports on your Linux VM and Azure network topology and the xRDP daemon can be basically listening to the localhost interface in your Linux VM on Azure. Basic steps to configure a more secure xRDP environment are: 1) Edit the /etc/xrdp/xrdp.ini and change/add the following to force the daemon to listen to 127.0.0.1 and also use a higher encryption level: [globals] bitmap_cache=yes bitmap_compression=yes port=3389 crypt_level=high channel_code=1 max_bpp=24 #black=000000 #grey=d6d3ce #dark_grey=808080 #blue=08246b #dark_blue=08246b #white=ffffff #red=ff0000 #green=00ff00 #background=626c72 address=127.0.0.1 NOTE: Keep in mind that by changing the daemon to 127.0.0.1 you will only be able to access it by doing an SSH tunnel which can be done by following the steps below. For OSX and Linux users, you can run on a terminal window: ssh -N -p 22 USER@VM_PUBLIC_IP -L 3389:127.0.0.1:3389 In the example above you are basically using a local port 3389 (on your workstation/laptop) and tunnelling that traffic to your Linux VM on Azure to its localhost on port 3389, so to connect to the VM using xRDP you will basically connect locally on your workstation to port 3389 and that gets automatically tunnelled to the VM on Azure.

Generally GUI's are disabled by default on Linux servers due to security concerns so, if you really need to enable this kind of access it is highly recommended that you are using SSH Keys as well as local tunnels so you don't have to open extra ports to the Linux VM.