HP ILO VSP: CentOS 7/RHEL 7 Installation through Serial Console

Though it's feared or considered as outdated by some, command line tools (CLI) remain the right ones for many System Administration tasks. Among its countless advantages, there's tasks' automation through its Script-ability.

In this post, I'm willing to describe the Usage of HP ILO CLI utility for the Complete Installation of CentOS 7/RHEL 7 Distribution. Through this mean, one can complete a full System Installation from CLI. Such Installation are quite useful for many cases, like Remote Installation with low bandwidth, No need for  Advanced ILO license...

The Installation described below is a CentOS 7.1 System on Proliant G8 (using ILO4) and I'm leveraging on KickStart to automate its Install process. But, the same procedure can be easily adapted for other Linux Distributions.

Let's start by reviewing what is needed to complete that Installation:

1. Prerequisites:

• ILO Configured with Valid IP Addressing parameters and reachable through SSH

• Linux System (any distribution) with mkisofs installed

That Linux System will be used to build Custom CentOS/RHEL Image, in this case I'm using the System the same system to share the ISO mentioned below (Hostname: stivinstall; IP: 192.168.1.11)

• OS Distribution Media (ISO) available on the network and reachable from the ILO

As said above, I'm installing CentOS 7.1, so I shared the ISO on an http server (installed and configured on my Linux System) and checked that it's well available (http://192.168.1.11/mnt/CentOS-7-x86_64-DVD-1503-01-text.iso)

• (Optional) KickStart File for automated Installation

I'm leveraging on Kickstart to fully automate this Installation. But that is obviously optional. In this case, I made Kickstart file available on the same System where I hosted the OS ISO (http://192.168.1.11/dladc2-infpup01.ks)

2.  Modify CentOS 7 boot image to get output on Serial Console

By default, the boot ISO of RHEL 7/CentOS 7 will redirect its output to Graphical Console, so that needs to be modified to have Output directed to the Serial Console (VSP).

Mount the CentOS ISO on my elected Installation System (stivinstall)

[root@stivinstall ~]# mount -o loop /mnt/CentOS-7-x86_64-DVD-1503-01.iso /var/www/html/media_centos7u1/ mount: /dev/loop0 is write-protected, mounting read-only

The main directory we'll be modifying in this ISO is Isolinux, so Copy this directory under a temporarily writable directory.

[root@stivinstall ~]# cd /var/www/html/media_centos7u1/ [root@stivinstall media_centos7u1]# cp -a isolinux /tmp/ [root@stivinstall media_centos7u1]# cd /tmp/isolinux/

As we're aiming to have everything in Text (console) mode, we should first get rid of graphical feature, like image. So, in the boot.msg, remove the ^Xsplash.lss line and delete the boot.cat (will be re-created later)

[root@stivinstall isolinux]# cat boot.msg splash.lss - Press the 01<enter>07 key to begin the installation process. [root@stivinstall isolinux]# vim boot.msg [...] "boot.msg" 4L, 72C written [root@stivinstall isolinux]# cat boot.msg - Press the 01<enter>07 key to begin the installation process. [root@stivinstall isolinux]# rm boot.cat rm: remove regular file ‘boot.cat’? y

Now, the main and most important modifications are to add/edit two main configuration in the isolinux.cfg:

1. A new line with "serial 1 9600" which indicates to IsoLinux to redirect the output to the Serial Console
2. Append "console=ttyS1" to the Kernel Option (append initrd...). This is a kernel option that specifies what device to use as the primary console and its implies text as the Installation Mode.

Below, I've made use of diff between the modified and an unmodified isolinux.cfg to highlight that modifications. We can see that around Line 54 (this can obviously added as option in other place in the file), I added (the line starting with "#"is just a comment):

# Output to Serial Console ttyS1 - Stivesso
serial 1 9600


And for most of the append Kernel options (did this for Linux, Troubleshoot and Rescue Entries), I appended this at the end:
console=ttyS1

[root@stivinstall isolinux]# cp isolinux.cfg /tmp/ [root@stivinstall isolinux]# vim isolinux.cfg [...] "isolinux.cfg" 123L, 3135C written [root@stivinstall isolinux]# diff isolinux.cfg /tmp/isolinux.cfg 54,56d53 < # Output to Serial Console ttyS1 - Stivesso < serial 1 9600 < 67c64 < append initrd=initrd.img inst.stage2=hd:LABEL=CentOS\x207\x20x86_64 quiet console=ttyS1 --- > append initrd=initrd.img inst.stage2=hd:LABEL=CentOS\x207\x20x86_64 quiet 73c70 < append initrd=initrd.img inst.stage2=hd:LABEL=CentOS\x207\x20x86_64 rd.live.check quiet console=ttyS1 --- > append initrd=initrd.img inst.stage2=hd:LABEL=CentOS\x207\x20x86_64 rd.live.check quiet 99c96 < append initrd=initrd.img inst.stage2=hd:LABEL=CentOS\x207\x20x86_64 rescue quiet console=ttyS1 --- > append initrd=initrd.img inst.stage2=hd:LABEL=CentOS\x207\x20x86_64 rescue quiet

With that modification on isolinux.cfg completed, we can recreate the ISO. but first, we mount (with the --bind option) our modified directory under the isolinux directory of the mounted Image. Then, we can create the ISO using mkisofs.

Note that the volume ID (-V) of the ISO image must be the same as the inst.stage2= parameter in isolinux.cfg and replacing \x20 by a space (described in BZ#915563), Otherwise we'll be facing an Issue during the Installation and will be redirected to Dracut Emergency mode (... Warning: /dev/root does not exist, Entering emergency mode. Exit the shell to continue.)

[root@stivinstall isolinux]# mount --bind /tmp/isolinux /var/www/html/media_centos7u1/isolinux [root@stivinstall isolinux]# mkisofs -R -J -T -v -V "CentOS 7 x86_64" -no-emul-boot -boot-load-size 4 -boot-info-table -b isolinux/isolinux.bin -c isolinux/boot.cat -o /mnt/CentOS-7-x86_64-DVD-1503-01-text.iso /var/www/html/media_centos7u1 I: -input-charset not specified, using utf-8 (detected in locale settings) genisoimage 1.1.11 (Linux) [...] 99.78% done, estimate finish Sun Aug 30 09:34:35 2015 Total translation table size: 907346 Total rockridge attributes bytes: 400459 Total directory bytes: 606208 Path table size(bytes): 140 Done with: The File(s) Block(s) 2103975 Writing: Ending Padblock Start Block 2104510 Done with: Ending Padblock Block(s) 150 Max brk space used 3d2000 2104660 extents written (4110 MB) [root@stivinstall isolinux]# umount /var/www/html/media_centos7u1/isolinux

The following is just a way to check that the created ISO has the needed modification in isolinux.cfg.

[root@stivinstall isolinux]# mount -o loop /mnt/CentOS-7-x86_64-DVD-1503-01-text.iso /mnt/tst mount: /dev/loop1 is write-protected, mounting read-only [root@stivinstall isolinux]# grep -i console /mnt/tst/isolinux/isolinux.cfg # Output to Serial Console ttyS1 - Steve ESSO append initrd=initrd.img inst.stage2=hd:LABEL=CentOS\x207\x20x86_64 quiet console=ttyS1 append initrd=initrd.img inst.stage2=hd:LABEL=CentOS\x207\x20x86_64 rd.live.check quiet console=ttyS1 append initrd=initrd.img inst.stage2=hd:LABEL=CentOS\x207\x20x86_64 rescue quiet console=ttyS1 [root@stivinstall isolinux]# umount /mnt/tst

3.  Use ILO CLI to insert a virtual media

Once we've met the prerequisites listed above and created our modified ISO Image, we can go to the ILO (ssh) and insert the OS Distribution ISO in the ILO Virtual Media. This is done using the vm command, to see a full description of vm command options and syntax, one can use "help vm" as seen below

</>hpiLO-> help vm status=0 status_tag=COMMAND COMPLETED Sat Aug 29 07:53:09 2015 VM : Virtual media commands. Usage: vm <device> insert <path> -- Inserts an image CLP COMMAND: set /map1/oemhp_vm1/<device> oemhp_image=<path> vm <device> eject -- Ejects an image CLP COMMAND: set /map1/oemhp_vm1/<device> oemhp_image=none vm device get -- Gets the status of the virtual media CLP COMMAND: show /map1/oemhp_vm1/<device> vm <device> set <options> -- Sets the status of the virtual media CLP COMMAND: set /map1/oemhp_vm1/<device> <property=value> Command Options: Device is floppy or cdrom CLP device is floppydr or cddr Path is the url to the media image Access option is write_protect or write_allow Boot option is boot_always, boot_once, no_boot, connect or disconnect CLP properties are : oemhp_connect, oemhp_boot, oemhp_wp

Get the Status of the CDROM Virtual Media,

</>hpiLO-> vm cdrom get status=0 status_tag=COMMAND COMPLETED Sat Aug 29 07:53:31 2015 Image Connected = No Image URL = None VM Applet = No Boot Option = NO_BOOT Write Protect = No

Insert our OS Distribution ISO Image,

</>hpiLO-> vm cdrom insert http://192.168.1.11/mnt/CentOS-7-x86_64-DVD-1503-01-text.iso status=0 status_tag=COMMAND COMPLETED Sat Aug 29 07:54:25 2015 </>hpiLO-> vm cdrom get status=0 status_tag=COMMAND COMPLETED Sat Aug 29 07:54:28 2015 Image Connected = No Image URL = http://192.168.1.11/mnt/CentOS-7-x86_64-DVD-1503-01-text.iso VM Applet = No Boot Option = NO_BOOT Write Protect = Yes

Connect the Inserted OS Image,

</>hpiLO-> vm cdrom set connect status=0 status_tag=COMMAND COMPLETED Sat Aug 29 08:02:31 2015 </>hpiLO-> vm cdrom get status=0 status_tag=COMMAND COMPLETED Sat Aug 29 08:02:35 2015 Image Connected = Yes Image URL = http://192.168.1.11/mnt/CentOS-7-x86_64-DVD-1503-01-text.iso VM Applet = No Boot Option = BOOT_ALWAYS Write Protect = Yes

Set the system to boot on this image during the next reboot, for that we can either use boot_once to have it mounted and set as boot drive just during the next boot or boot_always to have it permanently mounted and set as boot drive.

</>hpiLO-> vm cdrom set boot_once status=0 status_tag=COMMAND COMPLETED Sat Aug 29 08:02:48 2015 </>hpiLO-> vm cdrom get status=0 status_tag=COMMAND COMPLETED Sat Aug 29 08:02:50 2015 Image Connected = Yes Image URL = http://192.168.1.11/mnt/CentOS-7-x86_64-DVD-1503-01-text.iso VM Applet = No Boot Option = BOOT_ONCE Write Protect = Yes

4.  OS Installation:

Now that we have the media Inserted, we can proceed to the OS Installation by powering on the Server (or resetting if it was already running) and getting to VSP (Virtual Serial Port) to complete OS Installation.
I strongly advise to have the ILO SSH Console opened in max window size to make sure that the console output fit the size of the SSH console.

</>hpiLO-> power on status=0 status_tag=COMMAND COMPLETED Sun Aug 30 08:43:47 2015 Server powering on ....... </>hpiLO-> vsp Virtual Serial Port Active: COM2 Starting virtual serial port. Press 'ESC (' to return to the CLI Session. ProLiant System BIOS - P71 (11/01/2014) Copyright 1982, 2014 Hewlett-Packard Development Company, L.P. 2 Processor(s) detected, 4 total cores enabled, Hyperthreading is enabled Proc 1: Intel(R) Xeon(R) CPU E5-2637 0 @ 3.00GHz Proc 2: Intel(R) Xeon(R) CPU E5-2637 0 @ 3.00GHz [...] ISOLINUX 4.05 0x54f93f16 Copyright (C) 1994-2011 H. Peter Anvin et al

We'll get to the following nice screen,

Press Up (to make sure it’s at “Install CentOS 7” entry) and Tab,

From here, you can either just press enter to proceed to an Interactive Install, or if you prefer to use KickStart (as I do :-) ), then just enter the appropriate options. In this case, I’m entering the following inst.ks=http://192.168.1.10/stix-1.ks ip=192.168.1.11::192.168.1.1:255.255.255.0::ens2f0:none (more details about RHEL7/CentOS7 Kickstart Install in this post)

My suggestion for entering the option, "don't copy-paste, better write..."

References:

http://blog.allot.fr/?p=85
https://rhinstaller.github.io/anaconda/boot-options.html (Anaconda Documentation)
http://hintshop.ludvig.co.nz/show/centos-text-cd/

HP ILO / RHEL7 Systemd Output to VSP console

During my last post, I described the redirection of Linux Output to Serial on an Upstart Distribution based (using RHEL6). The main aim of this post is to describe the same on Systemd Distribution based (RHEL7). Before I begin with the technical matters, I have to say that I've been quite impressed by how Systemd makes this configuration so easy (no pain at all! So cool!). This isn't to make a comparison between Systemd and Traditional Init or Upstart, but you can check these two previous posts to draw your own Conclusions:  VSP/Traditional Init, VSP/Upstart.

Let's now delve in the interesting matters. The whole procedure is just about setting Kernel Options and reboot, and if the reboot can't be performed right away, just start a systemd service.

1. Set the Kernel Options boot options: 

On RHEL7 with Grub2, add "console=ttyS1" to  GRUB_CMDLINE_LINUX in /etc/default/grub file (You might also remove rhgb quiet as rhgb is for RedHat Graphical Boot and quiet is meant to hide the majority of boot messages before rhgb starts)

[root@mynode ~]# grep "GRUB_CMDLINE_LINUX" /etc/default/grub GRUB_CMDLINE_LINUX="crashkernel=auto vconsole.font=latarcyrheb-sun16 rd.lvm.lv=el/swap rd.lvm.lv=el/root vconsole.keymap=us console=ttyS1"

Changes to /etc/default/grub require rebuilding the grub.cfg file. This file location's on BIOS based machine is /boot/grub2 and for UEFI based machine, it is /boot/efi/EFI/redhat/.

On BIOS Based Machine:

[root@mynode ~]# grub2-mkconfig -o /boot/grub2/grub.cfg

On UEFI Based Machine:

[root@mynode ~]# grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg

2. Reboot or start a serial-getty service on ttyS1

Now we can either reboot the System to have the Kernel Loaded with the new parameter during the reboot , or (especially if we can't afford a downtime :-)) run the following to have a getty service started right away on the ttyS1.

[root@mynode ~]# systemctl start serial-getty@ttyS1.service

That's it! So simple! Go on the console, run vsp and there's a nice prompt...

Red Hat Linux Server 7.0 Kernel 3.8.13-35.3.1.el7uek.x86_64 on an x86_64 mynode login:


Reference:
http://0pointer.de/blog/projects/serial-console.html

HP ILO / RHEL6 Upstart Output to VSP console

Following my post related to HP ILO VSP console redirection , I've received many comments related to the same Configuration on recent Linux Distribution. In this post, I'm willing to detail the same on Upstart Based Distribution (using RHEL6, but should be the same for others Upstart Based distribution ). In fact, the main configuration on old Distribution was mostly related to Init Daemon Configuration, with new distribution based on Upstart/Systemd, things are slightly different. 

Note that I won't detail the configuration of the Virtual Serial Port on BIOS/UEFI as it had already been discussed in this previous post.
If you're interested in the same configuration for Systemd Based Distribution, it's described in this post.

1. Create an init configuration file for ttyS1

[root@mynode ~]# vi /etc/init/ttyS1.conf start on runlevel [S345] stop on runlevel [016] respawn instance /dev/ttyS1 exec /sbin/agetty ttyS1 115200 vt100-nav

2. Check the init configuration and start running the agetty process

Upstart leverages the Linux inotify API to make itself aware of any changes that can happen within its configuration directory, so the creation of the ttyS1 file above is enough to have the service available and listed when using initctl. The only thing we have to do is to start the process.

[root@mynode ~]# initctl list | grep ttyS1 ttyS1 stop/waiting [root@mynode ~]# initctl start ttyS1 ttyS1 (/dev/ttyS1) start/running, process 38394

3. Test you have access to the System through vsp

</>hpiLO-> vsp Virtual Serial Port Active: COM2 Starting virtual serial port. Press 'ESC (' to return to the CLI Session. mynode login:

4. Add Serial Port to securetty to allow login as root

This is needed if we want root account to be able to log in through this serial console.

[root@mynode ~]# echo "ttyS1" >> /etc/securetty

5. Configure the Grub GRUB config file

Finally, we can configure the GRUB to have outputs of the boot process on the console, this is easily achieved by adding console=tty0 console=ttyS1,115200

[root@mynode ~]# cat /etc/grub.conf # grub.conf generated by anaconda # # Note that you do not have to rerun grub after making changes to this file # NOTICE: You have a /boot partition. This means that # all kernel and initrd paths are relative to /boot/, eg. # root (hd0,0) # kernel /vmlinuz-version ro root=/dev/sda5 # initrd /initrd-[generic-]version.img #boot=/dev/sda default=0 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu title Red Hat Enterprise Linux (2.6.32-220.el6.x86_64) root (hd0,0) kernel /vmlinuz-2.6.32-220.el6.x86_64 ro root=UUID=0084ea5e-39e2-4994-aaa5-5abe8bf7eeb0 rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=128M KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM console=tty0 console=ttyS1,115200 initrd /initramfs-2.6.32-220.el6.x86_64.img

HP ILO/Linux output to VSP

The aim here is to describe the configuration and usage of HP ILO VSP (Virtual Serial Port) in order to redirect a RHEL5 System Output on this Virtual Port.

Note that the procedure is different for RHEL6/Upstart or RHEL7/Systemd, these entries address the same for: RHEL6/Upstart based-System , RHEL7/Systemd based-system. 

The same procedure could be used with others Init-based Linux Distribution. We'll go through the following Steps:

BIOS Configuration

Grub Configuration

Init Configuration

Connect trough ILO

1. BIOS Configuration:

Go to ‘BIOS Serial Console & EMS’ menu

Set  ‘BIOS Serial Console Port’ to COM2

Set ‘BIOS Serial Console Baud Rate’ to 115200

2. Grub Configuration:

Before modifying the grub.conf, we should first confirm the name of COM2 device which was previously configured in the BIOS. Setserial is a program designed to set and/or report the configuration information associated with a serial port. This information includes what I/O port and IRQ a particular serial port is using.

[root@mypc.me ~]# setserial -g /dev/ttyS[01] /dev/ttyS0, UART: 16550A, Port: 0x03f8, IRQ: 4 /dev/ttyS1, UART: 16550A, Port: 0x02f8, IRQ: 3

Append "console=tty0 console=ttyS1,115200" to the kernel configuration of your boot entry in grub.conf.

#boot=/dev/sda default=0 timeout=5 splashimage=(hd1,0)/grub/splash.xpm.gz #hiddenmenu title myboot (2.6.18-128.1.10.el5)         root (hd0,0)         kernel /vmlinuz-2.6.18-128.1.10.el5 ro root=/dev/VolGroup00/LogVol00 console=tty0 console=ttyS1,115200         initrd /initrd-2.6.18-128.1.10.el5.img

3. Init Configuration:

Add the following lines to /etc/inittab

# Added for VSP of ILO S1:2345:respawn:/sbin/agetty 115200 ttyS1 vt100

getty / agetty - agetty opens a tty port, prompts for a login name and invokes the /bin/login command.

Ask Init Daemon to reread its configuration file

root@mypc.me # init q

4. Connect trough ILO:

Now, it's time to try to connect.

</>hpiLO-> vsp Virtual Serial Port Active: COM2 Starting virtual serial port. Press 'ESC (' to return to the CLI Session. Red Hat Enterprise Linux Server release 5.3 (Tikanga) Kernel 2.6.18-128.el5PAE on an i686 mypc.me login:

If the Virtual Port is already in use by someone else, you'll have the following error: "Virtual Serial Port is currently in use by another session." In this case, you can stop the actual connection and start yours.

</>hpiLO-> vsp status=2 status_tag=COMMAND PROCESSING FAILED error_tag=COMMAND ERROR-UNSPECIFIED Virtual Serial Port is currently in use by another session. </>hpiLO-> stop /system1/oemhp_vsp1