The aim here is to describe the configuration of Sun LDOMs (also named Oracle VM For SPARC) as failover guest Domains. As the name suggests, failover guest domain is a Sun Logical Domain (often abbreviated LDOM) which could be hosted on one of the nodes of a Solaris Cluster at any one time. It implies that, in order to configure a failover guest domain we should already have installed and configured Solaris Cluster on a set of Control domains.
Oracle VM for SPARC provides the ability to split a single physical system into multiple, independent virtual systems. This is achieved by an additional software application in the firmware layer, interposed between the operating system and the hardware platform called as the hypervisor. It abstracts the hardware and can expose or hide various resources, allowing for the creation of resource partitions that can operate as discrete systems, complete with virtual CPU, memory and I/O devices."
While the benefit of platform virtualization encourages server consolidation, it introduces a problem that the physical platform now becomes a huge single point of failure. In this regard, the Solaris Cluster agent for LDoms guest domains aims to mitigate against a physical server failure by managing the guest domains. The agent performs the following:
- Manage the start/stop and restart of LDoms guest domains.
- Failover a Ldoms guest domain between Solaris Cluster nodes.
- Allow for strong positive and negative affinities between LDoms guest domains across Solaris Cluster nodes.
- Allow for different failover techniques, i.e. Stop/Failover/Start, Warm migration.
- Plugin probe facility for fault monitor.
At this point as the term “Migration” is being introduced, it is important to emphasize that migration on it's own does not represent high availability. The significant difference between the two [migration and high availability] is that migration, without Sun cluster and HA-LDOM agent, requires human intervention to initiate migration. whereas high availability is responsible for the automated failure detection in hardware and software so that services can be started on another cluster node without human intervention.
Below is a description of the architecture we're willing to deploy (click on the image for full size).
As you can see, we've 02 Coolthreads servers (we used Sun Fire T5440) connected to a shared Storage trough a SAN Infrastructure. A Service and Control Domain was configured as well (primary domain). Below we' won't covered neither the installation of Oracle VM For Sparc ( You can check Oracle VM Server for SPARC Administration Guide on oracle.com) nor the installation of Solaris Cluster (Again, Installation Guide for Solaris Cluster Available on oracle.com), we'll just describe the Failover guest domain Configuration.
Also, this configuration won't allow us to perform neither a Live nor a Warm Migration, only a cold migration will be possible in such configuration (the main reason here is that ZFS cannot be used as a Global FileSystem). If you're not familiar with the LDOM Migration, then i highly recommend you to read the Oracle VM Server for SPARC Administration Guide on oracle.com, the Migration process is very well described in this document.
1. Create the Zpool which will serve as Receptacle for the LDOM Domains Virtual Disks.
After Oracle VM for SPARC Installation/configuration and Solaris Cluster Installation/Configuration, A ZFS storage pool (Zpool) is created on Solaris Cluster did device (the shared LUN provided by the storage was identified as d14 on Solaris Cluster). Found better to partition the shared LUN and allocate all available space to slice 6.
2. Create Resource Group and add your Zpool as resource
Nodes List is provided during the creation in order to set a preference. In fact when the Resource group is brought online, Resource Group Manager will try to bring this RG up on the first node provided in this node list (in our case, ldom1-rg will be brought online on node1).
# clrs create -g ldom1-rg -t HAStoragePlus -p Zpools=zpool_ldom1 ldom1-fs-rs
# clrg online -M ldom1-rg
Check and verify that the Resource Group was brought online and Zpool imported and mounted on node1 (you can also verify that it wasn't mounted on Node2 :) )
3. Configure the Service and Control Domain (here it's the primary domain)
Keep in mind that both SC Domains should have same configurations in term of Services provided (same vswitch, same vdsdev...). So, the commands below should be run on Node1 and on Node2 and every time you perform a reconfiguration on one of the SC Domain, manually do the same on the other one.
#ldm add-vswitch net-dev=nxge2 primary-vsw1 primary
#ldm add-vdiskserver primary-vds0 primary
4. Create and configure your LDOM's Virtual Disks on Node1
Below, 02 Virtual Disks were created:
#zfs create zpool_ldom1/ldom1
#cd /zpool_ldom1/ldom1
#date && mkfile -v 100G disk0 && date
#date && mkfile -v 100G disk1 && date
After that, VDS-DEV corresponding to those 02 virtuals Disks were configured (again, on both primary domains). Doing so on the node where ldom1-rg is running wasn't a problem (was on Node1), but for the Node2, ldom1-rg should first be switched on the node (Otherwise, you'll have an error -the path must exist when adding the vds-dev)
#ldm add-vdsdev //zpool_ldom1//ldom1/disk0 ldom1-vol0@primary-vds0
#ldm add-vdsdev //zpool_ldom1//ldom1/disk1 ldom1-vol1@primary-vds0
#clrg switch -n node2 ldom1-rg
On Node2:
#ldm add-vdsdev //zpool_ldom1//ldom1/disk0 ldom1-vol0@primary-vds0
#ldm add-vdsdev //zpool_ldom1//ldom1/disk1 ldom1-vol1@primary-vds0
#clrg switch -n node1 ldom1-rg
Now, it's time to create and configure the LDOM.
Configuration of the LDOM itself must be performed only on One Node.
No need to perform the same on the others cluster nodes.
On Node 1:
#ldm add-domain ldom1
#ldm add-vcpu 8 ldom1
#ldm add-memory 16G ldom1
#ldm add-vdisk vdisk0 ldom1-vol0@primary-vds0 ldom1
#ldm add-vdisk vdisk1 ldom1-vol1@primary-vds0 ldom1
#ldm add-vnet vnet0 primary-vsw0 ldom1
#ldm add-vnet vnet1 primary-vsw1 ldom1
#ldm set-variable auto-boot\?=false ldom1
#ldm set-variable local-mac-address\?=true ldom1
#ldm set-variable boot-device=/virtual-devices@100/channel-devices@200/disk@0 ldom1
#ldm bind-domain ldom1
#ldm start ldom1
After that, go on the Console (#telnet localhost vconsport) and proceed to Classic Solaris 10 OS Installation.
6. Perform a test Migration (without LDOM-HA)
We choose to use a non-interactive migration (To perform this migration without being prompted for the target machine password). As explained in the introduction we've to perform a cold migration.
So we'll first stop and unbind the Logical Domain, then trigger a failover for the Zpool which contain the Vdisk of the LDOM and start the migration
On Node1:
#ldm stop ldom1
#ldm unbind ldom1
#clrg switch -n node2 ldom1-rg
#ldm migrate-domain -p /pfile ldom1 node2
The -p option takes a file name as an argument. The specified file contains the superuser
password for the target machine.
On Node2:
#ldm bind ldom1
#ldm start ldom1
Check that LDOM is booting without any errors.
Perform any other tests you want for LDOM1, then shut it down and bring it back to node1
#ldm stop ldom1
#ldm unbind ldom1
#clrg switch -n node1 ldom1-rg
#ldm migrate-domain -p /pfile ldom1 node1
The -p option takes a file name as an argument. The specified file contains the superuser
password for the target machine.
7. Create the SUNW.ldom resource for the guest Domain
Now, it's time to automate failure detection in hardware and software so that LDOM can be started on another cluster node without human intervention.
We'll achieve that by creating the following Resource in ldom1-rg
> -p password_file=/pfile -p Resource_dependencies=ldom1-fs-rs \
> –p Migration_type=NORMAL ldom1-ldom-rs
the first property (-p) here is simply the Domain Name, the Password File used here is the same used to test the migration (step 5), Resource dependencies is set to the Resource which is responsible of the Zpool which hosts the Vdisk, with this setting we're sure that ldom1 Domain won't be started if the zpool isn't imported and mounted on a node. The last property is just to precise the type of migration to use (Normal for cold Migration).
Test the migration of the Resource Group
# clrg switch -n node2 ldom1-rg
8. Maintenance of the Failover Guest Logical Domain
One thing to note is that SUNW.ldom will bring up the System even if you performed a clean shutdown from the Operating System (like an init 5). This behavior is normal because this agent is configured in a way that it'll always try to keep the Guest Domain up and running. So in order to perform LDOM's operations which request a downtime, the best thing to do is to suspend the resource group which own the domain. For example below, i decided to modify the hostid of a guest Domain which is under SUNW.ldom monitoring.
# ldm set-dom hostid=82a85641 ldom1
LDom ldom1 must be stopped before the hostid can be modified
# ldm ls
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 16 32992M 11% 19d 4h 4m
ldom1 active -t---- 5001 4 16G 25% 1m
# clrg suspend ldom1-rg
# ldm ls
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 16 32992M 3.2% 19d 4h 5m
ldom1 active -t---- 5001 4 16G 25% 1m
# ldm stop ldom1
LDom ldom1 stopped
# ldm ls
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 16 32992M 8.6% 19d 4h 5m
ldom1 bound ------ 5001 4 16G
# ldm set-dom hostid=82a85641 ldom1
# ldm ls
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 16 32992M 7.8% 19d 4h 5m
ldom1 bound ------ 5001 4 16G
# clrg resume ldom1-rg
# ldm ls
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 16 32992M 9.0% 19d 4h 7m
ldom1 active -n---- 5001 4 16G 28% 1m
DONE.
Hope that this will help many people to deploy Highly Available LDOMs.
Below are links for Documentations i used.
Feel free to post any comments.
References:
Oracle Solaris Cluster Essentials, 2010, Prentice Hall
http://hub.opensolaris.org/bin/view/Project+ha-xvm/WebHome
http://download.oracle.com/docs/cd/E23120_01/index.html
http://www.oracle.com/technetwork/server-storage/solaris-cluster/overview/index.html?ssSourceSiteId=ocomen
Hi Steve, it' s a great article with clear and easy to follow procedures:). My company is embarking on a huge project that will likely to use the LDOM with HA setup and your article gives me a very good foundation on what's going to happen.
ReplyDeleteHence, does it mean that if we want to use the live migration feature of LDOM, we cannot use ZFS as the Global FileSystem within the control domain. Do you know anyway we can workaround it?
Thanks.
LD
Hi, Thanks for the compliments. In fact, ZFS is not supported under Solaris Cluster as a Global Filesystem. If you want to use Live Migration, then the supported method is to use either UFS or VXFs (and put your ldoms backend disks devices on it).
DeleteHi Steve,
DeleteOk thanks for the clarification. I did some read up and here is my understanding on the concept but not sure if they are valid.
- Live Migration between guest domains will work on ZFS without any constraints if Solaris Cluster is not installed. But there are some ptr-requistes for Live Migration that must be met as defined in Chapter 9 at http://docs.oracle.com/cd/E23120_01/html/821-2854/migratingldoms.html#scrolltoc.
- If Sun Cluster is installed, then the backend disks devices for the guest domains had to be deposited on a Global file system(which is "accessible" by all nodes in the cluster concurrently) for Live Migration to work. The constraint with ZFS here is that currently it is supported as a HA Local File System only, hence Live Migration will not work cause the backend disk devices are not found on a shared filesystem.
But if we really need to use the Live Migration feature, then the backend disk devices had to be placed on a UFS or VXFS formated shared filesystem.
Please correct me if I am wrong.
Hi Steve,
ReplyDeleteOk thanks for the clarification. I did some read up and understood the following but not sure if they are correct. Really appreciate if you can cross check to see if we are on the same page.
- Solaris Cluster is not required for Live Migration to work. But there are certain
pre-requisites that must be met as indicated in Chapter 9 of the Oracle VM Server for SPARC 2.1 Administration Guide for Live migration to work.
- For Live Migration to work with Solaris Cluster, only image files can be used as backend disk devices and they must reside on a Global FileSystem (filesystem which is "accessible" via multiple nodes simultaeously). Since ZFS only supports HA on Local FileSystem, it cannot be used as a Global FileSystem hence we have to resort to a VXFS or UFS formated filesystem instead.
This also means other types of backend disk devices like VXFS/SVM volumes or raw devices cannot be used as a repository for the LDOM guest OS(s).
Lastly, I am also quite curious on how you backup and recover the guest and control LDOMs at your site. Care to share your backup and recovery strategy as well?
Millions of thanks.
Hi,
DeleteAbout your first point, you're correct. Live Migration is a LDOM feature, Solaris Cluster just helps to automate this Migration (No manual Intervention needed).
For the second one, i'll say that only image files are officially supported by Oracle as backend disk devices and they must reside on a Global FileSystem. In fact, setting raw LUN in shared storage as backend device will work (and often provides better Disk performance).But it's not documented.
Concerning Backup, it all depends on your design. IMHO, the easiest backup strategy is to use your actual Backup Infrastructure (like with physical servers). Now, you can look for Optimization by using for example ZFS Snapshot Feature (if your Backend Disk resides on ZFS) or Storage Advanced feature (like Snapview for EMC if your backend devices reside on a SAN LUN). Also, don't forget to backup your Guest Doamins Configuration by using "ldm list-constraints -x > .xml" from the Control Domain.
Hi Steve,
DeleteThanks for the advices and I really appreciate that. Seems like I still have have to rely on EMC Networker for backup and recovery. :-(
I've been told by Oracle backline that we can pass the Solaris Cluster DIDs directly to the guest domains and do live migration that way. So, we don't 'need' to use a PxFS/Global FS to be the backing store for the LDOM virtual disks. Simply add the /dev/did/dsk/dNs2 to the vds as vdsdev, set the vdsdev as the vdisk in the HA LDOM and then let the HA-LDOM data service do the migration.
ReplyDeleteI just tested this manually and it works like a charm. Having issues the the SUNW.ldom agent however (got a case open with Oracle for that).
Hi,
DeleteThanks for sharing the method to use Solaris Cluster DIDs instead of image files. May I know what are the benefits of using the Solaris Cluster DIDs over image files? Which will be easier to manage and maintain potentially?
IMHO, you'll get better performance by using DID devices (In term of Disk I/Os).
DeleteKeep in mind that with Image Files, we've guest domain's filesystems build on Image files which are sitting on a primary domain's filesystem , too many layers for Disk i/o.
While with devices (DID, dsk, emcpower...), you have your guest filesystem build directly on the Device.
I did the same but with emcpower devices instead of DID devices (i used it for an Oracle RDBMS which generated many IOs) and it also worked very well (but still i didn't use Live Migration because i wasn't aware that it's supported).
Using File Images you'll just get better flexibility (mostly for Backup and Cloning).
The procedure is still the same, the only differences are:
1. When adding your vdsdev, specify the raw device as backend device:
for example
ldm add-vdsdev /dev/dsk/emcpowerxc ldom1-vol0@primary-vds0 (for emcpower device)
or
ldm add-vdsdev /dev/did/dsk/dxs2 ldom1-vol0@primary-vds0 (for DID device, The first anonymous guy could confirm :) )
2. If you're going for Live Migration change the Property Migration_type of the SUNW.ldom resource to MIGRATE (instead of NORMAL)
The default Migration_type is MIGRATE in Cluster 4. I've just finished rolling out two clusters with Solaris 11 in the CD and am building out a bunch of ldoms (solaris 11 and solaris 10 u10), configured with the ldom data service.
DeleteThere was an issue with the stop_timeout value for the ldom resource in that the live migration takes more than 2 minutes and the cluster was triggering a restart of the ldom because the stop_timeout (def. 300) was not enough. Oracle support asked me to change that 1200 and then the live migrations were being done by the cluster with a clrg switch command.
Only one disk related issue I've encountered so far is with a second zpool in one of my ldoms which failed during live migration and had to be hard-started by cluster on another node. At that point, the second zpool didn't get imported into the guest OS. After manually importing the pool in, subsequent live migrations worked very nicely.
Ok Great Steve! Thanks for the detailed answer. Based on all the information that I gathered and understood, I would like to use the following setup but do you think it is feasible or does it differ from the industrial standard on how things are done?
ReplyDelete* Solaris Cluster 3.3 Update 5 will be installed on Oracle Solaris 10 with ZFS root pool on each cluster node
* OS image file will be stored on each guest domain ZFS pool but for the virtual disk(s) used to store data, a complete LUN in the form of /dev/did/dsk/dXXs2 will be exported to the guest domains.
As such two virtual disk servers will be created - the primary vds to store the OS zpools which contain the image files and a second vds to hold the data LUNs.
* For networking, each logical domain will have
- one primary virtual network switch with 2 network ports aggregated from 2 different physical switches
- one backup LAN virtual network switch that comprise of only one port which is shared across all logical domains for OS backup
* The rest of the configuration is the same as what was describe in Steve' s blog except that the mode of migration_type is set to MIGRATE
Hi,I assume that when you said "OS image file will be stored on each guest domain ZFS pool" you want to say "OS image file will be stored on each primary domain ZFS pool".
DeleteAbout your configuration, you won't be able to use MIGRATE as migration_type if the Guest OS image files are sitting on a ZFS Pool.ZFS isn't a global filesystem, so a ZPOOL will be active on one node at a time. If you're planning to use image files and MIGRATE as migration_type, then you should use either UFS or VXFS (as global filesystem).
Apart that, IMHO your design seems correct.
Hi Steve, Thanks for the excellent documentation. I have an query on HALdom setup. As we use HALdom agent how do we control Applications/Databases which installed on failover ldom? Should Applications/Databases be confgured with SMF?
ReplyDeleteHi, Thanks for the compliments.
DeleteI must admit that i don't understand your question. If it's about the relation between the services(Applications & Databases) which are running in the Failover Guest Domain and The Cluster Software, i'll say that there's no direct relation using HA-LDOM. In fact, using HA-LDOM, Solaris Cluster software controls the virtualized entity without having any presence within the virtualized entity (so with HA-LDOM, you can view LDOM as the "application" that the Solaris Cluster is managing).
In contrast, you can install Solaris Cluster on 02 guests LDOM and clustered these LDOMs (as you could with 02 physical systems, but this isn't HA-LDOM). Let me know if that's what you wanted to know.
Hi Steve,
ReplyDeleteThanks for your response & update. Yes It's about relation between the services(Applications & Databases) which are running in the Failover Guest Domain and The Cluster Software.
In Failover Guest Domain(HALdom) setup, is there any way to configure resource in cluster that it would trigger failover RG if Applications/Databases got some issue?
I assume that can't be done with Failover Guest Domain(HALdom) setup. That can be done in Solaris Cluster on 02 guests LDOM only.
Am I correct?
I'm using the smf and init scripts right now to control automated start-ups and shutdowns of the apps (including Oracle DB).
DeleteIn the HA Zone agent, there was an option to write an scsmf or sczsh probe to check for specific applications (or infact inspect anything in the non-global zone) via the cluster framework. I didn't find references to such things in the ldom data service. It is logical for that to be the case since the ldom (VM)is really a self-contained entity and cluster is merely managing the state of this entity (this is called a black-box approach, iinm)
Hi Anonymous :), you're correct. As Dwai said, HALDOM is a black box approach.
DeleteHi,
ReplyDeleteJust curious can the SUNW.ldom resource and resource group creation (I mean all Solaris cluster steps) be defered to a later stage instead of performing them at the beginning or middle? Thanks
Thanks Dwai & steve I agree with your updates :)
ReplyDeleteHi,
ReplyDeleteJust would like to c;larify can the cluster steps clrg/clrs be deferred to a later stage.. Meaning only after I have tested all the SAN/network connectivity and after the guest LDOMs OS are all setup, then the guest LDOMs be managed by the cluster service? Thanks.
What do you think to install SUN Cluster in each guest domain instead of Control Domain? Are there an disadvantages?
ReplyDeleteHi, Can we reboot the Control Domain without affecting guest domains? Why?
ReplyDeleteIn this case, rebooting the control domain will affect guest domains that are running on it due to the cluster mechanism. In fact, the cluster services (guests LDOMs and their Back-end Devices) will automatically migrate on the other cluster node.
DeleteHi Steve
ReplyDeleteGreat blog on fail-over, have u tried it for 4.1? and RAC CL ZONES?
Thanks Michael, i haven't yet tried exactly the same with SC 4.1 .But i 'm actually configuring fail-over zones with sol11 11/11 and SC4.0. Planning to post about that very soon.
Deletehi Steve,
ReplyDeletethanks for this helpful document.
I'm not familiar with OVM and i have some question:
- cluster configuration must be done before LDOM configuration or after?
-how I can migrate my servers Solaris 8 and Solaris 9 into guest domain?
-do you have a procedure how to install and configure cluster within LDOM?
thanks for cooperation.
best regards...
Hello Kamal,
Delete- cluster configuration is done every time u need to configure LDOM in HA mode, i guess that your question is more related to the installation and first configuration. You can choose to install Cluster Framework before or after the LDOMs configuration (i mentioned in the post the fact that Cluster is responsible for the automated failure detection in hardware and software so that services can be started on another cluster node without human intervention), meaning that you can configure OVM/LDOM Migration without Cluster (the HA cluster function is just helping to automate the process).
-Solaris 8/9 aren't supported on Sun4v Server (OVM for Sparc is working on Sun4v Server). IMHO, you can install a Solaris 10 Box on OVM and then create Zone/Containers for Solaris 8/9 on top of it.
-For procedure about how to install and configure cluster within LDOM,i don't have a steps by steps procedure, but you can check this one (there isn't a big difference with doing the same with physical nodes): http://docs.oracle.com/cd/E19575-01/821-0259/ggvtu/index.html
dear Steve,
ReplyDeleteThank for your prompt reply.
can we use P2V utility to have our Solaris 8,9 machine on the OVM or it better to create a guest Solaris 10 under the LDOM and make a zone Solaris 8 and 9 ?
best regards...
IMHO, the best way is to first create your guest Solaris 10 under LDOM and then start a P2v (from your Solaris 8/9 physical systems to Zones on the Solaris 10 LDOM that you built).
DeleteWe have two T3-1 blades and have 4 ldoms in each.The LUNS are shared.
ReplyDeleteWe are able to migrate the guest ldoms manually from one blade to another.
We want to make sure that incase one blade goes down the guest ldoms in that blade migrate automatically to the other blade.
This did not happen once one of our blade got down. How do we accomplish this feature ?
Hi,
DeleteAll you need is to install Solaris Cluster on your System & Control Domains and then configure it to automate the migration of ur LDOMs (basically, you'll create Resource Group for each LDOM and as resources for this RG , at least 01 HAStoragePlus and 01 SUNW.ldom).
Hi
ReplyDeleteI have 4 gust domin with zfs file sytem can i create cluster using this zfs file system os-solaris11 & sun cluster is 4.1
Please help on this
i saw some new zfs will not support global file sytem then how can i care cluster in b/w two gest domains
Thanks
Kesava
Hello,
DeleteSorry, could you explain what you mean by "4 guest domain with zfs fs"?
Also, keep in mind that the cluster resource is based on zpool (not zfs dataset - i.e : clrs create -g ldom1-rg -t HAStoragePlus -p Zpools=zpool_ldom1 ldom1-fs-rs-).
Meaning that your failover strategy is mostly based on the zpool import/export (not individual zfs datasets...)
Hi Steve
ReplyDeleteThanks for your quick responce,
i have 2 SparcT5 servers, each servers i have created (4 ldoms), i have to configure cluster how can i do this ?
Example:
primary1 primary2
ldom1 ldom1
i have to configure cluster form primary1 ldom to primary 2 ldom, please help on this
I have installed cluster s/w on each ldom is it corrrect ?
Thanks
Kesava
As per your update so, can i create zpool's inside the ldoms ? is it correct.or i have to create zpools in primary domain .
ReplyDeleteHi Steve
ReplyDeletePlease share the steps how to configure sun cluster on 2 gust domain(ldoms),now i am working on this could you provide ASAP all ere in (solaris11 -os),
Thanks
Kesava
Hi Steve
ReplyDeleteI am waiting for for your update ??
Thanks
Kesava
Hello Kesava, sorry i'm just reading your questions. Anyway, configuring the cluster on the guest domains isnt really different from what you can do with 02 physical servers. You can easilly do that by following this from Oracle : http://www.oracle.com/technetwork/articles/servers-storage-admin/o11-147-install-2node-cluster-1395587.html
DeleteThe main challenge for you in this configuration is to make sure that the shared disks are well assigned among the node (especially if they're multipath devices)
Dear Steve,
ReplyDeleteThanks For sharing the Document it's very easy to understand , i will fellow same.
Thanks
Kesava
hi Steve,
ReplyDeleteI got one problem setting the stack vale of Oracle user ,how to do it permanent,
Remaining all vales i am able to set but stack value not able to set(min,max)
Thanks
Kesava
Hi Steve,
ReplyDeleteCould you please help me to configure the quorum device on 2ldoms from different hosts, i have assigned share lun to 2 physical server but i am not able to add this lun to gust domain.
so,Please help me how add a shared lun to gust domains in cluster environment.
Thanks
Kesava
Hi Steve,
ReplyDeleteHow can map shared luns to directly to gust domain. please provide the info if u have any idea abount above 2 requests.
Thanks
Kesava
Hello Kesava,
DeleteTell me, what multipath software are you using? (solaris mpxio, emc powerpath...)
Steve,
ReplyDeleteI have a question on the LDOM and ZFS filesystem.
If I share a LUN between the source and target T4 servers and then assign this LUN to the LDOM created on the source system as a vdisk and create a ZFS pool on that vdisk.
Then will I be able to migrate this ldom from the source server to the target server on-need basis and will the ZFS pool be seen on the target system as the below underlying physical disk/LUN is the same and seen on both the nodes?
I am asking this question because ZFS is more a volume manager and not a clustered file system so will a vdisk with ZFS be allowed to migrate and be made available on the target server.
The doc below does show a neat example but I am still looking for clarifications and further insights if this is something that would happen or not.
http://www.oracle.com/technetwork/server-storage/vm/ovm-sparc-livemigration-1522412.pdf
Appreciate your inputs
Thanks in advance,
Rohan
Hello Rohan, IMO, as long as the same vds-dev is created on both node, you should be able. You can see in step 4 of this post that at a certain time, i'm performing a manual migration of the ldom (Using Solaris Cluster) to have it running on the second node (the one you're referring as target node) and then create the required vds-dev on this node (see below).
DeleteYou can do the same and add the vdisk afterward (only on one node). Just to try to summarize, The live migration will migrate your guest ldom conf, you normally have to put in place configuration related to the primary domains on both nodes. I haven't read the whole doc, but check the picture on p.17. Hope i've been able to provide you some clarifications...
On Node1:
#ldm add-vdsdev //zpool_ldom1//ldom1/disk0 ldom1-vol0@primary-vds0
#ldm add-vdsdev //zpool_ldom1//ldom1/disk1 ldom1-vol1@primary-vds0
#clrg switch -n node2 ldom1-rg
On Node2:
#ldm add-vdsdev //zpool_ldom1//ldom1/disk0 ldom1-vol0@primary-vds0
#ldm add-vdsdev //zpool_ldom1//ldom1/disk1 ldom1-vol1@primary-vds0
#clrg switch -n node1 ldom1-rg
Hi Steve,
ReplyDeleteWe are using Solaris 10 and cluster version 3.3u2 and ldm version 3.1.1.2.2.
We are using RAW disks (LUNs) for guest domains.
So while configuring failover LDOM, what we have to take care with respect to RAW disks ?
Thanks,
Harish
Hi Harish,
DeleteThe main thing you should take care of is that the raw disk to use is available on both Primary Domains.
Then you must make sure of the name consistency across the Primary Domains Configuration and IMO, the best way to do that is to use the cldev (cluster device name) representing your raw disk as vdsdev.
So, that'll be during the step 4 of this post, and you should have something like (can use "cldev list -v" for cluster device name details):
On Node1:
#ldm add-vdsdev /dev/did/dsk/dxxs2 ldom1-vol0@primary-vds0
#clrg switch -n node2 ldom1-rg
On Node2:
#ldm add-vdsdev /dev/did/dsk/dxxs2 ldom1-vol0@primary-vds0
#clrg switch -n node2 ldom1-rg
thanks Steve, let me try this.
Deletewill update you within 2-3 days
Hi Steve,
ReplyDeleteI am able to do the live-migration with migrate-domain option.
However I am trying to do the same under sun-cluster 3.3 and when I am trying to create the resource getting error like invalid password file.
Even I have tried with point#5 as you specified then getting below message .Please suggest on this.
"Failed to establish connection with ldmd(1m) on target: node2
Check that the 'ldmd' service is enabled on the target machine and
that the version supports Domain Migration. Check that the 'xmpp_enabled'
and 'incoming_migration_enabled' properties of the 'ldmd' service on
the target machine are set to 'true' using svccfg(1M)."
hey Steve,
ReplyDeleteIts done , created that file on both the nodes :)
Thanks
Harish
You're welcome Harish! Congrats!
DeleteThe error is also seen in case incorrect destination root password is supplied
ReplyDeleteIndeed Gurpreet,
DeleteMore appreciable. Thank you sharing
ReplyDeleteweb development company in Bangalore | Website Design Company in Bangalore | user experience design firms india| ux design firms in bangalore
You are welcome!
DeleteReviving this thread! Comparing this to vSphere HA/DRS, what would happen if you had a host failure and the target server didn't have enough resources? Would LDOM's be created on the target and just not booted?
ReplyDeleteCan you have a cluster with more than 2 nodes? Lets say I had a cluster with 4 T7-2's and one host failed. What is the logic behind where the LDOMS are booted up on the surviving hosts?