Oracle VM for Sparc Autosave / Practical case

Have ever faced a situation where you think you've lost LDOMs configuration (mostly after an unexpected downtime)? If that's the case, then you must know how much autosave feature on Oracle VM for SPARC is important. Indeed, the main aim of the autosave feature is to ensure that a copy of the current configuration is automatically saved on the control domain whenever the Logical Domains configuration is changed. More important, it occurs even when the new configuration is not explicitly saved on the SP.

Note that this doesn't replace the classic LDOMs configuration Backup which consists of saving the constraints information for each domain into an XML file (ldm list-constraints -x...), but may supplement it when what is needed is just to recover domain Configurations that weren't saved to the SP. 

Enough talk! Let's move forward with this practical case. We've a System with 05 configured LDOMs, and while building them, an unexpected Hardware Failure occurs (:-)). The main issue is that we haven't saved yet our LDOM Configuration on the SP. Meaning before the Outage, we had something similar to:

root@scdomain:~# ldm ls-spconfig factory-default initial [next poweron] root@scdomain:~# ldm ls NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- UART 16 32G 6.9% 12h 21m STEVELDOM01 active -n---- 5000 16 32G 0.0% 10h 17m STEVELDOM02 active -n---- 5001 16 32G 1.0% 10h 52m STEVELDOM03 active -n---- 5002 16 32G 0.1% 10h 52m STEVELDOM04 active -n---- 5003 16 32G 0.1% 10h 52m STEVELDOM05 active -n---- 5004 16 32G 9.9% 10h 51m

And after the outage, we've something like this (with maybe some other parameters not present for the primary domain...).

root@scdomain:~# ldm ls-spconfig factory-default initial [next poweron] root@scdomain:~# ldm ls NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- UART 16 32G 6.9% 10m

Let's move forward with the following 05 little steps to recover our LDOMs configuration.

1. As a matter of precaution, take a Backup of the actual configuration and another one of the /var/opt/SUNWldm

The /var/opt/SUNWldm folder contains the Autosave directories.

root@scdomain:~# ldm list-constraints -x > /export/home/admin/backup_faulty.xml root@scdomain:~# tar cvf /export/home/admin/sunwldom.tar /var/opt/SUNWldm


2.lists the available autosave configurations

At this stage, we should have our autosaved configuration shown as being newer than the current configuration.

root@scdomain:~# ldm ls-spconfig -r initial [newer]

3. Recover the Autosave Configuration

root@scdomain:~# ldm add-spconfig -r initial


4. Perform a full power Cycle

root@scdomain:~# shutdown -y -g0 -i5 -> start /SYS Are you sure you want to start /SYS (y/n)? y Starting /SYS

5. Check the LDOM Configuration 

root@scdomain:~# ldm ls-spconfig factory-default initial [next poweron] root@scdomain:~# ldm ls NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- UART 16 32G 6.9% 12h 21m STEVELDOM01 active -n---- 5000 16 32G 0.0% 10h 17m STEVELDOM02 active -n---- 5001 16 32G 1.0% 10h 52m STEVELDOM03 active -n---- 5002 16 32G 0.1% 10h 52m STEVELDOM04 active -n---- 5003 16 32G 0.1% 10h 52m STEVELDOM05 active -n---- 5004 16 32G 9.9% 10h 51m

References:

http://docs.oracle.com/cd/E19604-01/821-0406/configurationmanagement/index.html

HP ILO / RHEL7 Systemd Output to VSP console

During my last post, I described the redirection of Linux Output to Serial on an Upstart Distribution based (using RHEL6). The main aim of this post is to describe the same on Systemd Distribution based (RHEL7). Before I begin with the technical matters, I have to say that I've been quite impressed by how Systemd makes this configuration so easy (no pain at all! So cool!). This isn't to make a comparison between Systemd and Traditional Init or Upstart, but you can check these two previous posts to draw your own Conclusions:  VSP/Traditional Init, VSP/Upstart.

Let's now delve in the interesting matters. The whole procedure is just about setting Kernel Options and reboot, and if the reboot can't be performed right away, just start a systemd service.

1. Set the Kernel Options boot options: 

On RHEL7 with Grub2, add "console=ttyS1" to  GRUB_CMDLINE_LINUX in /etc/default/grub file (You might also remove rhgb quiet as rhgb is for RedHat Graphical Boot and quiet is meant to hide the majority of boot messages before rhgb starts)

[root@mynode ~]# grep "GRUB_CMDLINE_LINUX" /etc/default/grub GRUB_CMDLINE_LINUX="crashkernel=auto vconsole.font=latarcyrheb-sun16 rd.lvm.lv=el/swap rd.lvm.lv=el/root vconsole.keymap=us console=ttyS1"

Changes to /etc/default/grub require rebuilding the grub.cfg file. This file location's on BIOS based machine is /boot/grub2 and for UEFI based machine, it is /boot/efi/EFI/redhat/.

On BIOS Based Machine:

[root@mynode ~]# grub2-mkconfig -o /boot/grub2/grub.cfg

On UEFI Based Machine:

[root@mynode ~]# grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg

2. Reboot or start a serial-getty service on ttyS1

Now we can either reboot the System to have the Kernel Loaded with the new parameter during the reboot , or (especially if we can't afford a downtime :-)) run the following to have a getty service started right away on the ttyS1.

[root@mynode ~]# systemctl start serial-getty@ttyS1.service

That's it! So simple! Go on the console, run vsp and there's a nice prompt...

Red Hat Linux Server 7.0 Kernel 3.8.13-35.3.1.el7uek.x86_64 on an x86_64 mynode login:


Reference:
http://0pointer.de/blog/projects/serial-console.html

HP ILO / RHEL6 Upstart Output to VSP console

Following my post related to HP ILO VSP console redirection , I've received many comments related to the same Configuration on recent Linux Distribution. In this post, I'm willing to detail the same on Upstart Based Distribution (using RHEL6, but should be the same for others Upstart Based distribution ). In fact, the main configuration on old Distribution was mostly related to Init Daemon Configuration, with new distribution based on Upstart/Systemd, things are slightly different. 

Note that I won't detail the configuration of the Virtual Serial Port on BIOS/UEFI as it had already been discussed in this previous post.
If you're interested in the same configuration for Systemd Based Distribution, it's described in this post.

1. Create an init configuration file for ttyS1

[root@mynode ~]# vi /etc/init/ttyS1.conf start on runlevel [S345] stop on runlevel [016] respawn instance /dev/ttyS1 exec /sbin/agetty ttyS1 115200 vt100-nav

2. Check the init configuration and start running the agetty process

Upstart leverages the Linux inotify API to make itself aware of any changes that can happen within its configuration directory, so the creation of the ttyS1 file above is enough to have the service available and listed when using initctl. The only thing we have to do is to start the process.

[root@mynode ~]# initctl list | grep ttyS1 ttyS1 stop/waiting [root@mynode ~]# initctl start ttyS1 ttyS1 (/dev/ttyS1) start/running, process 38394

3. Test you have access to the System through vsp

</>hpiLO-> vsp Virtual Serial Port Active: COM2 Starting virtual serial port. Press 'ESC (' to return to the CLI Session. mynode login:

4. Add Serial Port to securetty to allow login as root

This is needed if we want root account to be able to log in through this serial console.

[root@mynode ~]# echo "ttyS1" >> /etc/securetty

5. Configure the Grub GRUB config file

Finally, we can configure the GRUB to have outputs of the boot process on the console, this is easily achieved by adding console=tty0 console=ttyS1,115200

[root@mynode ~]# cat /etc/grub.conf # grub.conf generated by anaconda # # Note that you do not have to rerun grub after making changes to this file # NOTICE: You have a /boot partition. This means that # all kernel and initrd paths are relative to /boot/, eg. # root (hd0,0) # kernel /vmlinuz-version ro root=/dev/sda5 # initrd /initrd-[generic-]version.img #boot=/dev/sda default=0 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu title Red Hat Enterprise Linux (2.6.32-220.el6.x86_64) root (hd0,0) kernel /vmlinuz-2.6.32-220.el6.x86_64 ro root=UUID=0084ea5e-39e2-4994-aaa5-5abe8bf7eeb0 rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=128M KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM console=tty0 console=ttyS1,115200 initrd /initramfs-2.6.32-220.el6.x86_64.img

Oracle/Redhat Enterprise Linux 7 Kickstart Installation / without DHCP

RHEL7/OEL7 is out for few months now, with a lot of new features (Revamped Anaconda, Systemd...). But before really starting to enjoy all these nice features, let's perform some basic Automated Kickstart Installation. In this short post, I'm willing to describe just that type of Installation without relying on a DHCP Server (using Static Network Parameters). I'm also using an Installation tree and Kickstart file located on httpd servers and reachable from the System I'm installing. Enough talk! let's detailed this Kickstart Installation in the following 4 steps:

1. Make Installation Tree available on an httpd server:

We've RHEL7/OEL7 ISO Files and an httpd server (192.168.0.10) configured (with DocumentRoot being the classic /var/www/html). Mounting the ISO as loop device in the DocumentRoot is enough to have the Installation Tree available over httpd.

# mkdir /var/www/html/media_oel7u0 # mount -o loop /home/stiv/Downloads/Oracle_Enterprise_Linux_7.0.iso /var/www/html/media_oel7u0

2. Create the Kickstart File and make it available on the httpd system:

For that, I used as template an anaconda-ks.cfg  from another installed node and created the following kickstart.

#version=RHEL7 # System authorization information auth --enableshadow --passalgo=sha512 # Use http tree for the installation url --url=http://192.168.0.10/media_oel7u0 # Run the Setup Agent on first boot firstboot --enable ignoredisk --only-use=sda # Keyboard layouts keyboard --vckeymap=us --xlayouts='us' # System language lang en_US.UTF-8 # Network information network --bootproto=static --device=eno49 --gateway=192.168.0.1 --hostname=olnode--ip=192.168.0.20 --nameserver=192.168.0.1 --netmask=255.255.255.0 --noipv6 --activate network --bootproto=dhcp --device=eno50 --onboot=off --ipv6=auto network --bootproto=static --device=eno51 --ip=192.168.50.22 --netmask=255.255.255.0 --onboot=off --ipv6=auto network --bootproto=dhcp --device=eno52 --onboot=off --ipv6=auto # Root password rootpw --iscrypted $6$v797wiKZv7W0MK58$4drw.ys34Z4cMoR7ECwkCIbhYuY7Aoc2A3dWs3wCTTyRTt1OovohA518vsdqpEfFCKwqwfU4weCcFK6VAmdv/1 # System timezone timezone Africa/Douala --isUtc --nontp # System bootloader configuration bootloader --location=mbr --boot-drive=sda # Partition clearing information clearpart --all --initlabel --drives=sda # Disk partitioning information part /boot --fstype="xfs" --ondisk=sda --size=500 part pv.26 --fstype="lvmpv" --ondisk=sda --size=254005 part /boot/efi --fstype="efi" --ondisk=sda --size=200 --fsoptions="umask=0077,shortname=winnt" volgroup ol --pesize=4096 pv.26 logvol /var/lib/mysql --fstype="xfs" --size=150000 --name=var_lib_mysql --vgname=ol logvol swap --fstype="swap" --size=64000 --name=swap --vgname=ol logvol /home --fstype="xfs" --size=20000 --name=home --vgname=ol logvol / --fstype="xfs" --size=10000 --name=root --vgname=ol logvol /var --fstype="xfs" --size=10000 --name=var --vgname=ol %packages @base @compat-libraries @core @development @file-server @hardware-monitoring @infiniband @large-systems @network-file-system-client @performance %end

Make this file available on http (copied under httpd DocumentRoot) and test to make sure it's reachable (i.e http://192.168.0.10/olnode.ks )

3. Start the Kickstart Installation

I'm using an UEFI system, so In order to provide the right Kickstart Parameters during the boot process, I'm selecting an installation option in the boot menu and then press either the E key ( For BIOS Systems, that'll be he Tab key). A prompt is displayed which enables to edit the boot options already defined and to add new options.  In this case, I'm adding the following: 

inst.ks=http://192.168.0.10/olnode.ks ip=192.168.0.20::192.168.0.1:255.255.255.0:olnode:eno49:none

inst.ks specifies the location of the kickstart file, and ip sets statics network parameters, it must be in this form: ip=ip::gateway:netmask:hostname:interface:none. Below are some screenshots taken to illustrate this process.
Note that parameters such as hostname and Interface could be empty

4. Enjoy the Automation:

Once that's done, the last step is to Press Ctrl-x (or enter on BIOS System) and enjoy the automation...

References:
http://docs.oracle.com/cd/E52668_01/E54695/html/ol7-install-boot-options.html

Oracle VM for X86 / RHEL PVM Kickstart Installation without DHCP

Just a quick and short post to share a tip related to installation of RHEL as ParaVirtual Machine using kickstart(or any distribution that can use kickstart) on Oracle VM 3 for X86 without having to rely on a DHCP for the Network Parameters. The only prerequisite that is needed is to have kickstart file and RHEL Installation Path available on the network and reachable from Oracle VM Manager Server.

Once that's done, we create our Virtual Server as usual, except that for Network Path, we're having to add more parameters. Indeed we can use the --args to add any parameters we're willing to pass to the kernel. So, we're having something like :

--args kernel_boot_paramaters http://install_server/rhel6_4

For example, let's say that my kickstart file is located on an http server and accessible via http://install_server/my_kickstart.ks, I also need to set 192.169.0.10 as IP of eth1 interface in order to get access to this kickstart file's URL and the RHEL installation tree is http://install_server/rhel6_4. Then, i'll have the following:

--args "ks=http://install_server/my_kickstart.ks ip=192.169.0.10 ksdevice=eth1 netmask=255.255.255.0 gateway=192.169.0.1" http://install_server/rhel6_4

Note that the --args section are in quote, this is needed to clearly draw the boundary between what is kernel arguments and the installation tree's URL. 

Ansible Hosts / Install alternate upgraded Python version

Following the highlighted error below which happened while trying to configure a RHEL4 node to be manageable by ansible (and that i rightly attributed to the old Python 2.3.4 which is the default version of Python for RHEL4), i've decided to install an alternate Python version on the same RHEL node without impacting the already existing environment. Below i'm sharing the 8 steps I followed to have that working.

Note that although i'm describing this Python Installation in  relation with Ansible Integration, Step 1 to Step 6 can be used for the installation of an alternate Python Environment on any other Linux Systems.

Ansible's Error:

[stivesso@ansible-server ~]$ ansible my_ansible_node -m ping  --ask-pass

SSH password:

my_ansible_node | FAILED >> {

    "failed": true,

    "msg": "\r\nSUDO-SUCCESS-josewbugtgyoijxrgkuxihoejbjsbiuq\r\n  File \"/home/esso_s/.ansible/tmp/ansible-tmp-1406122775.05-117998270117858/ping\", line 1177\r\n    clean_args = \" \".join(pipes.quote(arg) for arg in args)\r\n

     ^\r\nSyntaxError: invalid syntax\r\n", 

    "parsed": false

}

1. Install Prerequisites on the target host

# yum groupinstall "Development tools" # yum install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel

2. Get an updated python package

# wget http://python.org/ftp/python/2.7.5/Python-2.7.5.tgz --no-check-certificate

3. Untar/unzip the package on the target node

# tar zxvf Python-2.7.5.tgz

4. configure with the alternate path as option (here, i'm planning to install the alternate environment in /opt/python2.7) and compile/install

# ./configure --prefix=/opt/python2.7 --enable-shared # mkdir /opt/python2.7 # make # make altinstall

5. Export Shared Library and Bin Library in PATH and LD_LIBRARY_PATH on User profile 

# tail -3 /etc/profile ## Added for python 2.7 export PATH=$PATH:/opt/python2.7/bin export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/python2.7/lib

6. Create ld.so.conf configuration file and run ldconfig to have library loaded system wide

# cat /etc/ld.so.conf.d/python27.conf /opt/python2.7/lib # ldconfig

7. Add the following to /etc/ansible/hosts on the Ansible Control Machine for the target node

my_ansible_node ansible_python_interpreter=/opt/python2.7/bin/python2.7  

8. Test again the ping module from ansible-server...

[esso_s@ansible-server ~]$ ansible my_ansible_node -m ping --ask-pass SSH password: my_ansible_node | success >> { "changed": false, "ping": "pong" }

Linux KVM / Network Bridge over bonded Interface

This is a short post about KVM Network Bridge Configuration for Guest Domain over a bonded Interface. For this post, we're using a Redhat Enterprise Linux 6 as KVM Physical System, but the procedure can be easily adjusted for any others Linux System which supports KVM. Before delving in the Technical Configuration, let’s have a look on what we’re trying to achieve. We’re having a Physical Server with 02 Physical Interfaces (eth0 and eth1) and are planning to use these two physical Interfaces in a highly available configuration. The same interfaces are also used to support a bridge that is used by KVM DomU (Guest Hosts). Below, we can see a picture describing the details of this configuration.

The first thing to do is to complete the bonding configuration and that's pretty straightforward. We're just creating the bonding.conf file in /etc/modprobe.d (to dynamically load the bonding module, bonding's option  will be modified later on and must reflects ones' needs), then we're editing the ifcfg-bond0 network configuration files.

[root@my_dom0 ]# cat /etc/modprobe.d/bonding.conf alias bond0 bonding

Let us now configure the bond0 Interface by editing its configuration files, note the BRIDGE configuration mentioned below, this is explicitly added to link the bond interface to the bridge interface. We are also setting the bond Interface parameters here.

[root@my_dom0 ]# cat /etc/sysconfig/network-scripts/ifcfg-bond0 DEVICE="bond0" ONBOOT="yes" BRIDGE=br0 NM_CONTROLLED=no BONDING_OPTS="mode=active-backup miimon=100 downdelay=200 updelay=200"

Next thing to do is to configure the bridge Interface, as usual we’re creating the ifcfg-br0 file containing our network configuration (IP, DNS...). Note that as described above, this is the Interface that will have the IP settings used to administrate the Physical Server (Note that this is a matter of architecture/design, on a typical production server with four interfaces, we could have for example separated this management network from the Guest Domains Network...)

[root@my_dom0 ]# cat /etc/sysconfig/network-scripts/ifcfg-br0 DEVICE=br0 TYPE=Bridge BOOTPROTO=static ONBOOT=yes # IP Configuration HOSTNAME="my_dom0" IPADDR="192.168.1.10" NETMASK="255.255.255.0" DNS1="192.168.1.2" DNS2="192.168.1.3" GATEWAY="192.168.1.1" IPV6INIT="yes" MTU="1500" NM_CONTROLLED="no"

Finally, we can configure the physical interface and restart the network service (or reboot the node…)

[root@my_dom0 ]# cat ifcfg-eth0 DEVICE="eth0" HWADDR="XX:XX:XX:XX:XX:X8" NM_CONTROLLED="no" ONBOOT="yes" BOOTPROTO=none MASTER=bond0 SLAVE=yes USERCTL=no [root@my_dom0 ]# cat ifcfg-eth2 DEVICE="eth2" HWADDR="XX:XX:XX:XX:XX:X9" NM_CONTROLLED="no" ONBOOT="yes" BOOTPROTO=none MASTER=bond0 SLAVE=yes USERCTL=no [root@my_dom0 ]# shutdown –r now

Check the Bridge Configuration, should have output similar to the one below. The bridged interface is now ready to be used by the Guest Domains.

[root@my_dom0 ~]# brctl show bridge name bridge id STP enabled interfaces br0 8000.b4b52f6d1bc0 no bond0

P2V Migration From Solaris 10 Global Zone to Oracle Solaris 11 Local Zone (Solaris 10 Branded Zone)

The aim of this post is to describe a Physical to Virtual (P2V) migration of a Bare-Metal Solaris 10 (a Physical Node) on a Solaris 11 Global Zone (Solaris 10 Branded Zone on Oracle Solaris 11). The scenario describes below involves a Solaris 10 Physical node named oranode (hosts an Oracle Database 10G and another Oracle application) and a Solaris 11 Global Zone named sol11-gz (already hosting some other Local Zones). 

In order to make this migration easily understandable, It has been technically divided in in 03 parts, the first parts deals with Analysis of the Source System, the second is about System/Data Collection and the third is about Target Zone Configuration and Installation.

I. Analysis of the Source System

The Data Analysis phase aims to collect some initial analysis Data. Oracle has provided a very nice tool for that phase, namely zonep2vchk. zonep2vchk serves two functions. First, it can be used to report issues on the source which might prevent a successful p2v migration. Second, it can output a template zonecfg, which can be used to assist in configuring the non-global zone target (man page).

The /usr/sbin/zonep2vchk script is belonging to pkg:/system/zones on Solaris 11 and can be copied from an Oracle Solaris 11 system to an Oracle Solaris 10 system. It is not required to run this utility from a specific location on the system.

Below, we're copying this utility to our Solaris 10 (oranode) and then we're running it with the option -T S11 which specifies that the target node for our planned migration of this Solaris 10 System is running on Oracle Solaris 11.

root@s11-gz:~# scp /usr/sbin/zonep2vchk stivesso@oranode: Password: zonep2vchk 100% |*******************************| 142 KB 00:00 root@oranode # chmod ug+x /export/home/stivesso/zonep2vchk root@oranode # /export/home/stivesso/zonep2vchk -T S11 --Executing Version: 1.0.5-11-16135 - Source System: oranode Solaris Version: Oracle Solaris 10 8/11 s10s_u10wos_17b SPARC Solaris Kernel: 5.10 Generic_147440-01 Platform: sun4u SUNW,Sun-Fire-15000 - Target System: Solaris_Version: Solaris 11 Zone Brand: solaris10 (Solaris 10 Container) IP type: exclusive --Executing basic checks - The following /etc/system tunables exist. These tunables will not function inside a zone. The /etc/system tunable may be transfered to the target global zone, but it will affect the entire system, including all zones and the global zone. If there is an alternate tunable that can be configured from within the zone, this tunable is described: forceload: drv/emcpsf No alternate tunable exists. forceload: drv/sd No alternate tunable exists. forceload: drv/ssd No alternate tunable exists. forceload: drv/emcp No alternate tunable exists. forceload: misc/emcpgpx No alternate tunable exists. forceload: misc/emcpmpx No alternate tunable exists. forceload: misc/emcpvlumd No alternate tunable exists. forceload: misc/emcpxcrypt No alternate tunable exists. forceload: misc/emcpdm No alternate tunable exists. forceload: misc/emcpioc No alternate tunable exists. set emcp:bPxEnableInit=1 zonep2vchk has no information on tunable set shmsys:shminfo_shmmax=4294967295 zonep2vchk has no information on tunable project.max-shm-memory set shmsys:shminfo_shmmin=1 Tunable is obsolete on target host set shmsys:shminfo_shmmni=100 zonep2vchk has no information on tunable project.max-shm-memory set shmsys:shminfo_shmseg=10 Tunable is obsolete on target host set semsys:seminfo_semmns=1024 Tunable is obsolete on target host set semsys:seminfo_semmsl=256 Alternate tunable exists inside a non-global zone: rctl process.max-sem-nsems set semsys:seminfo_semmni=100 Alternate tunable exists inside a non-global zone: rctl project.max-sem-ids set semsys:seminfo_semvmx=32767 Tunable is obsolete on target host set noexec_user_stack=1 zonep2vchk has no information on tunable - The following SMF services will not work in a zone: svc:/network/iscsi/initiator:default svc:/system/iscsitgt:default - The system is configured with the following non-root ZFS pools. Pools cannot be configured inside a zone, but a zone can be configured to use a pool that was set up in the global zone: swappool zpool_oracle zpool_oracle2 Basic checks compete, 25 issue(s) detected --Total issue(s) detected: 25

Quite verbose! Indeed this initial analysis give us a lot of insights for the whole p2v procedure. For example, we can see that in this study case, we have much more than 01 ZFS Zpool (beside the initial rpool). These are File-system which host the Oracle Database 10G (mentioned above) Data Files.

Thus, we must decide how these  application Data's file-systems will be migrated. The strategy we're applying here is to migrate just the Operating System (excluding the Application Data File-System), and use SAN replication for the Application Data. Note that this decision is based on the source system architecture (we may for example decide to go for a zfs send/receive if these Data file-system are not located on an external storage…).

Once done with that, we complete this analysis phase by creating a template file for the Target Zone Creation (using the same zonep2vchk tool with the option -c). This is just a template , it is used to ease the creation of the target Zone and should be adjusted to suit with the requirement of the target environment.

root@oranode # /export/home/stivesso/zonep2vchk -T S11 -c create -b set zonepath=/zones/oranode set brand=solaris10 add attr set name="zonep2vchk-info" set type=string set value="p2v of host oranode" end set ip-type=exclusive # Uncomment the following to retain original host hostid: # set hostid=94a76031 # Max procs and lwps based on max_uproc/v_proc set max-processes=20000 set max-lwps=40000 add attr set name=num-cpus set type=string set value="original system had 4 cpus" end # Only one of dedicated or capped cpu can be used. # Uncomment the following to use cpu caps: # add capped-cpu # set ncpus=4.0 # end # Uncomment the following to use dedicated cpu: # add dedicated-cpu # set ncpus=4 # end # Uncomment the following to use memory caps. # Values based on physical memory plus swap devices: # add capped-memory # set physical=16384M # set swap=119823M # end # Original ce0 interface configuration: # Statically defined oranode (oranode) # Factory assigned MAC address 0:14:4f:1e:a2:90 add anet set linkname=ce0 set lower-link=change-me # Uncomment the following to retain original link configuration: # set mac-address=0:14:4f:1e:a2:90 end # Original dman0 interface configuration: # Statically defined 192.168.1.3/27 # Factory assigned MAC address 0:0:be:a9:66:71 add anet set linkname=dman0 set lower-link=change-me # Uncomment the following to retain original link configuration: # set mac-address=0:0:be:a9:66:71 end exit

II. System/Data Collection

This phase aims to collect Source System's Data (Solaris 10 System - oranode) and transferred them to the Target System (Solaris 11 Global Zone - s11-gz). As discussed during the analysis phase, I have decided to migrate System Data and Application Data in separate processes.

For the system, I’m making use of the classic flarcreate to generate system archive (due to the presence of Data File-System that i'd like to exclude, i'm using CPIO as method to archive the file). For the Application Data, they're being migrated using External Storage Cloning features (not described here!). 

root@oranode # flarcreate -S -n oranode -x /oracle -x /zpool_oracle -x /zpool_oracle2 -L cpio /zpool_oracle/$(hostname)-$(/usr/bin/date +"%Y-%m-%d").flar Archive format requested is cpio Full Flash Checking integrity... Integrity OK. Running precreation scripts... Precreation scripts done. Creating the archive... cpio: cpio: var/adm/lastlog: too large to archive in current mode 14429359 blocks 1 error(s) Archive creation complete. Running postcreation scripts... Postcreation scripts done. Running pre-exit scripts... Pre-exit scripts done.

The transfer of this archive to the target Global Zone can be done using any conventional method (scp/sftp/ftp, SAN Replication...). Once completed, we can move to the last part.

III. Target Zone Configuration and Installation

Now that we've the Data Collection phase completed, we’re going to create the target zone and start the restoration of the flararchive that was created. We’ll also add the Zpool Dataset which were migrated using the SAN replication features.

The zone will be created using the template that was generated by the zonep2vchk tool (we'll obviously customize it to fit the target environment). Before starting, we have to create the zonepath needed for the target zone and import the zpool that was migrated through the SAN replication.

root@sol11-gz:~# zpool create zpool_oranode <local_device> root@sol11-gz:~# zfs create zpool_oranode/oranode root@sol11-gz:~# chmod 700 /zpool_oranode/oranode root@sol11-gz:~# zpool import -f zpool_oracle zpool_oranode_data

Let’s now configure our new local zone by using the template generated by zonep2vchk tool during the first phase.

root@sol11-gz:~# cat /export/zonecfg/oranode.cfg create -b set zonepath=/zpool_oranode/oranode set brand=solaris10 add attr set name="zonep2vchk-info" set type=string set value="p2v of host oranode" end set ip-type=exclusive # Uncomment the following to retain original host hostid: # set hostid=94a76031 # Max procs and lwps based on max_uproc/v_proc set max-processes=20000 set max-lwps=40000 add attr set name=num-cpus set type=string set value="original system had 4 cpus" end # Only one of dedicated or capped cpu can be used. # Uncomment the following to use cpu caps: # add capped-cpu # set ncpus=8.0 # end # Uncomment the following to use dedicated cpu: add dedicated-cpu set ncpus=8 end # Uncomment the following to use memory caps. # Values based on physical memory plus swap devices: add capped-memory set physical=16384M set swap=119823M end # Added by Steve ESSO # for the new network configuration add anet set linkname=oranode_net0 set vlan-id=10 set lower-link=net0 end add anet set linkname=oranode_net1 set vlan-id=10 set lower-link=net1 end exit root@sol11-gz:~# zonecfg -z oranode -f /export/zonecfg/oranode.cfg root@sol11-gz:~# zonecfg -z oranode zonecfg:oranode> info zonename: oranode zonepath: /zpool_oranode/oranode brand: solaris10 autoboot: false bootargs: file-mac-profile: pool: limitpriv: scheduling-class: ip-type: exclusive hostid: fs-allowed: [max-lwps: 40000] [max-processes: 20000] anet: linkname: oranode_net0 lower-link: net0 allowed-address not specified configure-allowed-address: true defrouter not specified allowed-dhcp-cids not specified link-protection: mac-nospoof mac-address: random mac-prefix not specified mac-slot not specified vlan-id: 10 priority not specified rxrings not specified txrings not specified mtu not specified maxbw not specified rxfanout not specified anet: linkname: oranode_net1 lower-link: net1 allowed-address not specified configure-allowed-address: true defrouter not specified allowed-dhcp-cids not specified link-protection: mac-nospoof mac-address: random mac-prefix not specified mac-slot not specified vlan-id: 10 priority not specified rxrings not specified txrings not specified mtu not specified maxbw not specified rxfanout not specified dedicated-cpu: ncpus: 8 capped-memory: physical: 16G [swap: 117G] attr: name: zonep2vchk-info type: string value: "p2v of host oranode" attr: name: num-cpus type: string value: "original system had 4 cpus" rctl: name: zone.max-processes value: (priv=privileged,limit=20000,action=deny) rctl: name: zone.max-lwps value: (priv=privileged,limit=40000,action=deny) rctl: name: zone.max-swap value: (priv=privileged,limit=125643522048,action=deny)

Having the configuration done, we can start the zone Installation

root@sol11-gz:~# zoneadm -z oranode install -p -a /zpool_oranode_data/oranode-25nov2013.flar root@sol11-gz:~# zoneadm -z oranode install -p -a /zpool_oranode_data/oranode-25nov2013.flar Progress being logged to /var/log/zones/zoneadm.20131126T212904Z.oranode.install Installing: This may take several minutes... Postprocessing: This may take a while... Postprocess: Updating the image to run within a zone Postprocess: Migrating data from: zpool_oranode/oranode/rpool/ROOT/zbe-0 to: zpool_oranode/oranode/rpool/export Postprocess: A backup copy of /export is stored at /export.backup.20131126T213800Z. It can be deleted after verifying it was migrated correctly. Postprocess: Migrating data from: zpool_oranode/oranode/rpool/ROOT/zbe-0 to: zpool_oranode/oranode/rpool Postprocess: A backup copy of /rpool is stored at /rpool.backup.20131126T213856Z. It can be deleted after verifying it was migrated correctly. Result: Installation completed successfully. Log saved in non-global zone as /zpool_oranode/oranode/root/var/log/zones/zoneadm.20131126T212904Z.oranode.install

Finally, We are adding the ZFS dataset migrated through SAN Replication,

root@sol11-gz:~# zonecfg -z oranode zonecfg:oranode> add dataset zonecfg:oranode:dataset> set name=zpool_oranode_data zonecfg:oranode:dataset> set alias=zpool_oranode_data zonecfg:oranode:dataset> end zonecfg:oranode> commit zonecfg:oranode> exit

And the big moment! Boot the new zone...

root@sol11-gz:~# zoneadm -z oranode boot root@sol11-gz:~# zlogin -C oranode

Oracle Solaris 11: EMC PowerPath / powermt display dev ; Device(s) not found.

This is just a small post about Oracle Solaris 11 and EMC PowerPath Configuration. If you've been trying to get EMC PowerPath configured on Solaris 11 and facing error(s) similar to the one below, then follow steps described below (can also follow the same for initial PowerPath Configuration on Oracle Solaris 11)

root@stiv-sol11:~# powermt display Device(s) not found. root@stiv-sol11:~# powermt display dev=all Device(s) not found.


1. Make sure that Zoning/Lun Mapping from SAN/Storage is well configured.

For the zoning , run fcinfo remote-port -p <pwwn_connected_hba> (must list the PWWN and others details of the storage, otherwise must check that the zoning is well complete);

For the Storage Configuration, you can use fcinfo lu -v (list all fibre channel  logical  units, must have one or more "OS Device Name" listed here, otherwise must check that mapping is well done at the SAN Level). 

root@stiv-sol11:~# fcinfo remote-port -p <local_pwwn_port01> Remote Port WWN: <san_pwwn_spa_p1> Active FC4 Types: SCSI SCSI Target: yes Port Symbolic Name: DGC LUNZ 0430 Node WWN: <san_wwn_spa_p1> Remote Port WWN: <san_pwwn_spb_p1> Active FC4 Types: SCSI SCSI Target: yes Port Symbolic Name: DGC LUNZ 0430 Node WWN: <san_wwn_spb_p1> root@stiv-sol11:~# fcinfo remote-port -p <local_pwwn_port02> Remote Port WWN: <san_pwwn_spa_p2> Active FC4 Types: SCSI SCSI Target: yes Port Symbolic Name: DGC LUNZ 0430 Node WWN: <san_wwn_spa_p2> Remote Port WWN: <san_pwwn_spb_p2> Active FC4 Types: SCSI SCSI Target: yes Port Symbolic Name: DGC LUNZ 0430 Node WWN: <san_pwwn_spb_p2> root@stiv-sol11:~# fcinfo lu -v OS Device Name: /dev/rdsk/c0t60060160CD0E260000A8B1D4B789E311d0s2 HBA Port WWN: <local_pwwn_port02> Remote Port WWN: <san_pwwn_spa_p1> LUN: 0 Remote Port WWN: <san_pwwn_spb_p1> LUN: 0 HBA Port WWN: <local_pwwn_port01> Remote Port WWN: <san_pwwn_spa_p2> LUN: 0 Remote Port WWN: <san_pwwn_spb_p2> LUN: 0 Vendor: DGC Product: RAID 5 Device Type: Disk Device


2. Check that Oracle Solaris mpxio is disabled.
That can easilly be achieved by running stmsboot with -L option.  If Solaris  I/O  multipathing  is not enabled, then no mappings are displayed. If It's enabled, then disable it by running "stmsboot -d". For example, below we can see the output with Solaris mpxio enabled (and the output printed when disabling -and the reboot required-).

root@stiv-sol11:~# stmsboot -L non-STMS device name STMS device name ------------------------------------------------------------------ /dev/rdsk/c6t<san_pwwn_spa_p1>d0 /dev/rdsk/c0t60060160CD0E260000A8B1D4B789E311d0 /dev/rdsk/c6t<san_pwwn_spa_p2>d0 /dev/rdsk/c0t60060160CD0E260000A8B1D4B789E311d0 /dev/rdsk/c14t<san_pwwn_spb_p1>d0 /dev/rdsk/c0t60060160CD0E260000A8B1D4B789E311d0 /dev/rdsk/c14t<san_pwwn_spb_p2>d0 /dev/rdsk/c0t60060160CD0E260000A8B1D4B789E311d0 ........................................... root@stiv-sol11:~# stmsboot -d WARNING: stmsboot operates on each supported multipath-capable controller detected in a host. In your system, these controllers are /pci@400/pci@2/pci@0/pci@8/SUNW,qlc@0/fp@0,0 /pci@400/pci@2/pci@0/pci@8/SUNW,qlc@0,1/fp@0,0 /pci@500/pci@2/pci@0/pci@a/SUNW,qlc@0/fp@0,0 /pci@500/pci@2/pci@0/pci@a/SUNW,qlc@0,1/fp@0,0 /pci@400/pci@2/pci@0/pci@e/scsi@0/iport@1 ...... If you do NOT wish to operate on these controllers, please quit stmsboot and re-invoke with -D { fp | mpt | mpt_sas | pmcs} to specify which controllers you wish to modify your multipathing configuration for. Do you wish to continue? [y/n] (default: y) WARNING: This operation will require a reboot. Do you want to continue ? [y/n] (default: y) The changes will come into effect after rebooting the system. Reboot the system now ? [y/n] (default: y)

3. Check that your storage's type is managed by EMC PowerPath
For that , we're using powermt with options display options. If our storage's type is listed as unmanaged (like the clariion below), then simply mark it as managed (using powermt manage as seen below) and you're done!

root@stiv-sol11:~# powermt display options Default storage system class: all Show CLARiiON LUN names: true Path Latency Monitor: Off Path Latency Threshold: 0 milliseconds Performance Monitor: disabled Autostandby: Proximity (prox): enabled IOs per Failure (iopf): enabled iopf aging period : 7 d iopf limit : 6000 Storage System Class Attributes ------------ ---------- Symmetrix periodic autorestore = on reactive autorestore = on status = managed CLARiiON periodic autorestore = on reactive autorestore = on status = unmanaged Invista periodic autorestore = on reactive autorestore = on status = managed Hitachi periodic autorestore = on reactive autorestore = on status = managed HP xp periodic autorestore = on reactive autorestore = on status = managed Ess periodic autorestore = on reactive autorestore = on status = managed HP HSx periodic autorestore = on reactive autorestore = on status = unmanaged VPLEX periodic autorestore = on reactive autorestore = on status = managed VNX periodic autorestore = on reactive autorestore = on status = managed

root@stiv-sol11:~# powermt manage class=clariion root@stiv-sol11:~# powermt display CLARiiON logical device count=1 ============================================================================== ----- Host Bus Adapters --------- ------ I/O Paths ----- ------ Stats ------ ### HW Path Summary Total Dead IO/Sec Q-IOs Errors ============================================================================== 3077 pci@500/pci@2/pci@0/pci@a/SUNW,qlc@0,1/fp@0,0 optimal 2 0 - 0 0 3081 pci@400/pci@2/pci@0/pci@8/SUNW,qlc@0,1/fp@0,0 optimal 2 0 - 0 0

We should also pay attention to /kernel/drv/iscsi.conf. In fact, though Solaris  can distinguish between FC and iSCSI devices, some PowerPath versions don't make this distinction for manage and unmanage. So the mpxio-disable value must be set to yes in both the fp.conf (for fp.conf, it's also done automatically with stmsboot -d) and iscsi.conf files for PowerPath to manage EMC Clariion and VNX Storage Arrays.