Benoît Créau
Aix System Engineer
@chmod666
chmod666org
http://chmod666.org
benoit.creau@chmod666.org
PowerVC for PowerVM Deep Dive
Tips & Tricks
Technical University/Symposia materials may not be reproduced in whole or in part without the prior written permission of IBM.
9.0
whoami
• AIX/PowerVM System Engineer working in Large french banks
• Passionate about AIX and Power Systems
• Blogger since Feb 2010
– http://chmod666.org
• Creating tools for AIX an Power
– Github : chmod666org
• Sharing my daily life as a sysadmin on Twitter (tips and tricks,
blog post)
– @chmod666
• Recently Redbook author: PowerVC 1.2.3 Introduction and
configuration
(www.redbooks.ibm.com/redpieces/abstracts/sg248199.html?
Open)
© Copyright IBM Corporation 2015
2
Agenda
• Part 1 - Network management
• Part 2 - Storage management
• Part 3 - Datasource/activation
• Part 4 - Disks PVID
• Part 5 - cloud-init
• Part 6 - sys0 attributes ghostdev/clouddev
• Part 7 - Using openstack commands
• Part 8 - Rest api
• Part 9 - Linked clones
• Part 10 - Dynamic Resource Optimizer
© Copyright IBM Corporation 2015
3
PowerVC network management
• Shared Ethernet Adapters:
– PowerVC add and remove vlans from the Shared Ethernet Adapters.
– If a VLAN is needed on a host PowerVC add the VLAN on the Shared
Ethernet Adapter:
• At deploy time
• At migrate time (Live Partition Mobility)
• At restart time (Remote Restart Operation)
– If a VLAN is not needed anymore PowerVC removes the VLAN from
the Shared Ethernet Adapter:
• Same conditions as above (deploy, migrate, restart)
• PowerVC can create Virtual Ethernet Adapters and add it in the
Shared Ethernet adapters if needed
• PowerVC can remove Virtual Ethernet Adapter from the
Shared Ethernet Adapter
© Copyright IBM Corporation 2015
4
PowerVC Network Management : Forget your
conventions
• Don’t prepare a slot numbering convention as PowerVC can
add or remove the Virtual Ethernet Adapters
• Advice: Do not create SEA with more than two VEA.
Automatically added by
PowerVC (PVID 4094)
No custom buffers on
this interface
Output of lsseas korn shell script (https://github.com/chmod666org/lsseas)
© Copyright IBM Corporation 2015
5
PowerVC network management : SEA buffers
• Using the chdef way:
# chdef -a min_buf_tiny=4096 -c adapter –s vdevice –t IBM,l-lan
# chdef -a max_buf_tiny=4096 -c adapter –s vdevice –t IBM,l-lan
Same for min_buf_small=4096 max_buf_small=4096
min_buf_medium=512 max_buf_medium=512
min_buf_large=128 max_buf_large=128
min_buf_huge=128 max_buf_huge=128
• Using the Virtual I/O servers rules ( >= 2.2.4):
# rules -o modify -t adapter/vdevice/IBM,l-lan -a min_buf_tiny=4096
# rules -o modify -t adapter/vdevice/IBM,l-lan -a max_buf_tiny=4096
# rules –o list
[..]
adapter/vdevice/IBM,l-lan
max_buf_tiny
4096
adapter/vdevice/IBM,l-lan
min_buf_tiny
4096
# rules –o diff –s
# rules –o deploy
[1] Warning #7 (0590-206 A manual post-operation is required for the
changes to take effect): Please reboot the system
# shutdown -force -restart
© Copyright IBM Corporation 2015
6
Disabling vlan addition/removal
(automated_powervm_vlan_cleanup)
• PowerVC 1.2.3.1/2 bug: PowerVC forget to keep at least two
VEA for Shared Ethernet Adapter in sharing mode
• Some customers wants to disable this feature:
– Keep their numbering convention
– Avoiding removal of the next to last VEA
• Using openstack-config:
# openstack-config --set /etc/nova/nova.conf DEFAULT
automated_powervm_vlan_cleanup False
# openstack-config --set /etc/nova/nova-828642A_10D6D5T.conf DEFAULT
automated_powervm_vlan_cleanup False
• Modifying nova.conf :
# grep automated_powervm_vlan_cleanup /etc/nova/nova.conf
automated_powervm_vlan_cleanup = False
# grep automated_powervm_vlan_cleanup /etc/nova/nova9117MMD_659C5FF.conf
automated_powervm_vlan_cleanup = False
© Copyright IBM Corporation 2015
7
Disabling vlan addition/removal
(automated_powervm_vlan_cleanup)
• Any configuration files modification = RESTART POWERVC
# powervc-services restart
© Copyright IBM Corporation 2015
8
VLANs: be careful when adding new hosts!
• Adding a host on PowerVC does not automatically set which
SEA will be utilized for a particular VLAN.
• When adding a host, be carefull modifying all your existing
VLAN to match the SEA configuration !
• PowerVC will use the first founded one !
• This can result in propagating a VLAN in the wrong network
vswitch (deploy, LPM errors)
© Copyright IBM Corporation 2015
9
Zoning: How many zones will be created
• Number of zones created depends of:
– Storage connectivity group (easy to understand)
• NPIV Fabric Access Requirement.
• VIOS Redundency for volume Connectivity
– Storage Templates:
• Use all available WWPNs for attachement (very very important option) (
next slide)
This will decide how
many ports will be
zoned on your
storage array
© Copyright IBM Corporation 2015
10
Zoning: « Use all available WWPNs » (SVC)
• Option not checked
–
–
–
–
–
Both nodes will be zoned but not for each WWPNs
One zone per WWPN
For instance 4 NPIV adapters = 4 zones
No spof / Multi fabric
Warning on the SVC
Best practice on AIX: Use 4 or 8 paths, exceptionally 16 (too much paths can result in huge performance issues)
© Copyright IBM Corporation 2015
11
Zoning: « Use all available WWPNs » (SVC)
• Option checked
–
–
–
–
–
Both nodes will be zoned for each WWPNs
Many zones per WWPN
All SVC available ports will be zoned
No spof / Multi fabric
Too many zones ?
© Copyright IBM Corporation 2015
12
Zoning: FE Ports spreading
• By using a algorithm based on:
– The number of ports that can be zoned
– The name of the Virtual Machines
• PowerVC will spread the zoning on each available ports:
Virtual Machine 2
Virtual Machine 1
• In version <= 1.2.3.2 PowerVC just use the four first ports
available on the SVC.
• Ask for patch, open a PMR to get it.
© Copyright IBM Corporation 2015
13
Storage: Good to know
• Machines with no storage templates (adopt using « manage
existing » button):
– Use the default storage template (depending on the storage provider)
– When you’ll add a new LUN (from a different storage template) it adds
new zones to an existing VM
• Be very careful when creating VMs:
– PowerVC can create new zones on a existing VM but can not remove
a part of this zones (ie. all zones will be deleted at the VM deletion,
but you can’t remove zones if you made a mistake when choosing a
storage connectivity group or a storage template).
– If you notice that you choose the wrong storage template for a VM you
can add a new lun in the new storage template and all the existing lun
will use the new zones (just cfgmgr).
© Copyright IBM Corporation 2015
14
Openstack data-source used by PowerVC 1/4
1. The user enter ip address, the hostname, … in the GUI
2. A iso image is created on the PowerVC host (glance is doing that)
3. This iso is transfered on one of the Virtual I/O Server (POST rest api)
4. By using Virtual Optic devices the iso is attached to the virtual machine
5. The machine is created
6. cloud-init or activation engine is mounting and reading the virtual cdrom and can
use the information entered in the PowerVC GUI to setup the Virtual Machine
IP, hostname, activation input ….
1
VM creation
User
2
- IP
- hostname
- ae
- ….
PowerVC
server
mount
3
Loadbalanced VIOS1
(to be verified)
4
5
6
Cloud-init/ae
Created VM
VIOS2
© Copyright IBM Corporation 2015
15
Openstack data-source used by PowerVC 2/4
• For each Virtual Machines created PowerVC create a vscsi adapter
used for the virtual optic data source
• This adapter can be removed by the user if he wants to. This one will
not be utilized anymore after the VM creation
• Be careful with VIOS rootvg size (be sure to have enough space in /var
to copy the iso, had issue when moving on P8 boot on san VIOS)
• What’s inside this iso ?
# lsrep
Name
File Size Optical
Access
vopt_754df319e3f14cbc979011e2360fc723
1 None
ro
# ls –l /var/vio/VMLibrary | grep vopt
-r--r----1 root
system
425984 Aug 3 12:36
vopt_754df319e3f14cbc979011e2360fc723
# loopmount -i vopt_754df319e3f14cbc979011e2360fc723 -l loop0 -o "-V cdrfs -o ro" -m /mnt
© Copyright IBM Corporation 2015
16
Openstack data-source used by PowerVC 3/4
• Network configuration (cloud-init format/debian style):
# cat /mnt/openstack/content/0000
# Injected by Nova on instance boot
[..]
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
address 10.10.10.11
hwaddress ether fa:34:9b:4c:fb:20
netmask 255.255.255.0
broadcast 10.10.10.254
pre-up [ $(ifconfig eth0 | grep -o -E
'([[:xdigit:]]{1,2}:){5}[[:xdigit:]]{1,2}') =
"fa:34:9b:4c:fb:20" ]
gateway 10.10.10.254
dns-nameservers 8.8.8.8 8.8.8.9
dns-search lab.chmod666.org
© Copyright IBM Corporation 2015
Yes ! Cloud-init use
the debian style
interface format.
Don’t worry it’s his
job to translate it for
AIX (mktcpip)
17
Openstack data-source used by PowerVC 4/4
• User data (user-data):
Adding an ssh key
# cat /mnt/openstack/latest/user_data
runcmd:
- root_home=$(grep ^root /etc/passwd | awk -F ':' '{print
$6}') ; echo 'ssh-dss AAAABBBBCCCC[…]'>>
${root_home}/.ssh/authorized_keys
- mount deckard:/export/nim/postinstall /mnt
- /mnt/postintall.ksh
- /mnt/install_was.ksh
- /mnt/install_jre.ksh
Running post-install
scripts
This was a Websphere VM
© Copyright IBM Corporation 2015
18
The PVID topic 1/4
• To capture the Virtual Machines PowerVC is using:
– Snapshot (Shared Storage Pools)
– Flashcopies (SVC, Storwise)
• To create a new Virtual Machines PowerVC is using these
Snapshots/Flashcopies
– After the deployments your VM will have the same PVIDs for both
rootvg and datavg
– This is not an issue! But some customers may want to change this (for
instance storage billing can be based on PVID … or stuff like that)
# for i in $(cat myvms) ; do echo "==$i==" ; ssh -q $i "lspv | grep -E
\"rootvg|appsvg\"" ; done
==rachael==
hdisk0
00c30e8e42c879b9
rootvg
active
hdisk1
00c30e8e42d0aae5
appsvg
active
==tyrell==
hdisk0
00c30e8e42c879b9
rootvg
active
hdisk1
00c30e8e42d0aae5
appsvg
active
© Copyright IBM Corporation 2015
19
The PVID topic 2/4
• Solution for the rootvg
• Did you know changing the sys0 « ghostdev » to 2 and
rebooting the VM change rootvg PVID !
# lsattr -El sys0 -a ghostdev
ghostdev 0 Recreate ODM devices on system change / modify PVID True
# chdev -l sys0 -a ghostdev=2
sys0 changed
# lsattr -El sys0 -a ghostdev
ghostdev 2 Recreate ODM devices on system change / modify PVID True
• After rebooting the lpar ghostdev attribute will automatically be
reset to 0, and rootvg PVID will be changed
• Documentation about ghostdev : https://www01.ibm.com/support/knowledgecenter/#!/ssw_aix_61/com.ibm.
aix.osdevice/change_pvid_vgid.htm
• AIX 6.1/7.1/7.2
© Copyright IBM Corporation 2015
20
The PVID topic 3/4
• Solution for other VG, use the « recreatevg » command to
change all the PVIDs of a volume group
• cloud-init shell solution
© Copyright IBM Corporation 2015
21
The PVID topic 4/4
• cloud-init cloud-config solution :
© Copyright IBM Corporation 2015
22
cloud-init 1/5
• Cloud-init needs a lot of prerequistes. There is a script doing
the job for you (https://github.com/transt/cloud-init0.7.5/blob/master/cloudinst.sh)
# cloudinst.sh
• You need an access to internet to run the script. If you don’t
have any internet access, install it by hand. (rpm)
# rpm -ivh --nodeps gettext-0.17-8.aix6.1.ppc.rpm
[..]
gettext
##################################################
# for rpm in bzip2-1.0.6-2.aix6.1.ppc.rpm db-4.8.24-4.aix6.1.ppc.rpm expat-2.1.0-1.aix6.1.ppc.rpm gmp5.1.3-1.aix6.1.ppc.rpm libffi-3.0.11-1.aix6.1.ppc.rpm openssl-1.0.1g-1.aix6.1.ppc.rpm zlib-1.2.56.aix6.1.ppc.rpm gdbm-1.10-1.aix6.1.ppc.rpm libiconv-1.14-1.aix6.1.ppc.rpm bash-4.2-9.aix6.1.ppc.rpm info5.0-2.aix6.1.ppc.rpm readline-6.2-3.aix6.1.ppc.rpm ncurses-5.9-3.aix6.1.ppc.rpm sqlite-3.7.15.22.aix6.1.ppc.rpm python-2.7.6-1.aix6.1.ppc.rpm python-2.7.6-1.aix6.1.ppc.rpm python-devel-2.7.61.aix6.1.ppc.rpm python-xml-0.8.4-1.aix6.1.ppc.rpm python-boto-2.34.0-1.aix6.1.noarch.rpm python-argparse1.2.1-1.aix6.1.noarch.rpm python-cheetah-2.4.4-2.aix6.1.ppc.rpm python-configobj-5.0.5-1.aix6.1.noarch.rpm
python-jsonpointer-1.0.c1ec3df-1.aix6.1.noarch.rpm python-jsonpatch-1.8-1.aix6.1.noarch.rpm python-oauth1.0.1-1.aix6.1.noarch.rpm python-pyserial-2.7-1.aix6.1.ppc.rpm python-prettytable-0.7.2-1.aix6.1.noarch.rpm
python-requests-2.4.3-1.aix6.1.noarch.rpm libyaml-0.1.4-1.aix6.1.ppc.rpm python-setuptools-0.9.82.aix6.1.noarch.rpm fdupes-1.51-1.aix5.1.ppc.rpm ; do rpm -ivh $rpm ;done
[..]
python-oauth
##################################################
python-pyserial
##################################################
python-prettytable
##################################################
python-requests
##################################################
libyaml
##################################################
© Copyright IBM Corporation 2015
23
cloud-init 2/5
• The documentation is aweful
(https://cloudinit.readthedocs.org/en/latest/)
• Configuration file is located in /opt/freeware/etc/cloud.cfg
• For debugging purposes you may want to re-run cloudinit by
hand, delete somes files before re-running cloud-init:
# rm –rf /opt/freeware/var/lib/cloud/instances/*
• Default logs file located in:
– /var/log/cloud-init-output.log
– /var/log/cloud-init.log
• User data file located in
/opt/freeware/var/lib/cloud/instances/<XXX>/user-data.txt
#cloud-config
runcmd:
- mount nim:/export/nim/postinstall /mnt
- /mnt/install_oracle.ksh
- export LIBPATH=/usr/lib ; /mnt/script_install_was.ksh
© Copyright IBM Corporation 2015
24
Cloud-init 3/5
• Step 1 (local): cd is
loaded, ip, hostname
are set.
• Step 2 (init): All init
modules are launched
(rmc, bootlist, vg)
• Step 3 (config): All
config modules are
launched (mounts,
chef run, runcmd)
• Step 4 (final) : All final
modules are launched
(phone_home,
messages, shutdown)
Come join me in my
session about Chef
on AIX
(intermediate/expert)
© Copyright IBM Corporation 2015
25
cloud-init 4/5
• Files:
write_files:
- path: /tmp/cloud-init-started
content: |
cloud-init was started on this server
permissions: '0755'
- path: /var/log/cloud-init-sub.log
content: |
starting chef logging
permissions: '0755'
final_message: "The system is up, cloud-init is
finished"
© Copyright IBM Corporation 2015
26
cloud-init 5/5
• Power_state:
power_state:
delay: "+5"
mode: poweroff
message: cloud-init mandatory reboot for sddpcm
timeout: 5
• chef:
chef:
force_install: false
server_url:
"https://cserver.chmod666.org/organizations/chmod666"
validation_name: "chmod666-validator"
validation_key: |
-----BEGIN RSA PRIVATE KEY---------END RSA PRIVATE KEY----run_list:
- "role[aix7]"
© Copyright IBM Corporation 2015
27
Ghostdev ? Clouddev ? 1/2
• The problem:
– When a new machine is created, its rootvg was cloned. To remove the old devices
and wipe the ODM the activation is taking care of the ghostdev/clouddev attribute
(it’s a sys0 attribute)
• If you are using the activation engine:
– ghostdev is set to 1, after activation, ghostdev is automatically set back to 0
– It works on all versions of AIX.
• If you are using cloud-init:
– If you are using AIX >= 7100-03-05 or >=6100-09-05
• Clouddev is set to 1, after rebooting, clouddev is automatically set back to 0
– If you are using another version of AIX
• Ghostdev is set to 1, but after rebooting, ghostdev is not automatically set back
to 0
• Set manually (or using the activation input) ghostdev to 1
• If you forget to do that any remote restart operation will result in an ODM wipe.
• All AIX versions (6.1/7.1) will be supported in the next release of PowerVC
© Copyright IBM Corporation 2015
28
Ghostdev / Clouddev 2/2
• clouddev is using a flag is the nvram to check if the ODM needs to be
wiped or not
# errpt | more
IDENTIFIER TIMESTAMP T C RESOURCE_NAME DESCRIPTION
CB4A951F 0803171015 I S SRC SOFTWARE PROGRAM ERROR
D872C399 0803171015 I O sys0 Partition ID changed and devices recreat
• cloud-init on unsupported os:
# cloud-config
runcmd:
- chdev –l sys0 –a ghostdev=0
© Copyright IBM Corporation 2015
29
Using openstack command on PowerVC 1/2
• As PowerVC is based on Openstack, there is a way to use the
openstack commands on the PowerVC host.
• Create a powervcrc file:
powervc root
password
# cat powervcrc
export OS_USERNAME=root
export OS_PASSWORD=mypasswd
export OS_TENANT_NAME=ibm-default
export OS_AUTH_URL=https://powervc.lab.chmod666.org:5000/v3/
export OS_IDENTITY_API_VERSION=3
export OS_CACERT=/etc/pki/tls/certs/powervc.crt
export OS_REGION_NAME=RegionOne
export OS_USER_DOMAIN_NAME=Default export OS_PROJECT_DOMAIN_NAME=Default
powervc url
• Source it:
# source powervcrc
• You can now use openstack commands:
# nova list
+--------------------------------------+-----------------------+--------+------------+-------------+------------------------+
| ID
| Name
| Status | Task State | Power State | Networks
|
+--------------------------------------+-----------------------+--------+------------+-------------+------------------------+
| 394ab4d4-729e-44c7-a4d0-57bf2c121902 | deckard
| ACTIVE | | Running
| vlan666=10.16.66.60
|
| cd53fb69-0530-451b-88de-557e86a2e238 | priss
| ACTIVE | | Running
| vlan666=10.16.66.61
|
| 64a3b1f8-8120-4388-9d64-6243d237aa44 | rachael
| ACTIVE | | Running
|
|
| 2679e3bd-a2fb-4a43-b817-b56ead26852d | batty
| ACTIVE | | Running
|
|
| 5fdfff7c-fea0-431a-b99b-fe20c49e6cfd | tyrel
| ACTIVE | | Running
|
|
+--------------------------------------+-----------------------+--------+------------+-------------+------------------------+
© Copyright IBM Corporation 2015
30
Using openstack commands on PowerVC
• Reboot a machine:
# nova reboot deckard
• Migrate a machine (lpm):
# nova live-migration deckard 828641A_YYYYYY
• List hosts:
# nova hypervisor list
+----+---------------------+-------+---------+
| ID | Hypervisor hostname | State | Status |
+----+---------------------+-------+---------+
| 21 | 828641A_XXXXXXX
| up
| enabled |
| 23 | 828641A_YYYYYYY
| up
| enabled |
+----+---------------------+-------+---------+
• Maintenance mode:
# nova maintenance-enable --migrate active-only --target-host 828641A_XXXXXX 828641A_YYYYYYY
• Create a machine:
# nova create --image 7100-03-04-cic2-chef --flavor powervm.tiny --nic net-id=5fae84a7-b463-4a1a-b4dd9ab24cdb66b5,v4-fixed-ip=10.16.66.51 novacreated
• WARNING : No PowerVC extension with openstack cmd !
© Copyright IBM Corporation 2015
31
PowerVC Lun order
• To remove a rootvg lun from a Virtual Machine this one needs
to be in « position » 2. (not 1)
• For instance when I’m moving a new machine to PowerVC
(from an old host) I’m creating a VM with PowerVC (for
profiles,zoning, …). Then I’m mapping « old » luns to this
machine
© Copyright IBM Corporation 2015
32
Rest API 1/2
• The best way to scripts things on PowerVC is to use the rest API
• I warn you: for a system administrator using a rest API can be (at the
begining) difficult
• My advise is to use Python (json trees can be treated as dict)
• I’ve created a few example of how to use the PowerVC rest API :
– pvcgrowlun: growing a existing lun
(https://github.com/chmod666org/pvcmd/blob/master/pvcgrowlun)
– pvcmkvm: creating a Virtual machine with the rest API
(https://github.com/chmod666org/pvcmd/blob/master/pvcmkvm)
# pvcgrowlun -v multi-vol-bf697dfa-0000003a-828641A_21AFF8V-data-1 -s 84 -p
powervc.lab.chmod666.org -u root -P mypassword
# cat myvm
name:myvm
ip_address:10.10.10.2
vlan:vlan666
image:aix7100-03-05-chef
storage_connectivity_group:npiv_4ports
virtual_processor:1
entitled_capacity:0.1
memory:8192 storage_template:svc
# pvcmkvm -f myvm -p powervc.lab.chmod666.org -u root -P mypassword
© Copyright IBM Corporation 2015
33
Rest API 2/2
• The most difficult part with rest API is to find the json body you need to create
• My advice is to test every calls to the API with RESTClient (firefox plugin)
• Check json body with firefox firebug (example with network creation):
© Copyright IBM Corporation 2015
34
Copying images between storage providers
• When you have a lot of different storage providers (in my case I have three different
SVC) you have to synchronize your images between each one.
• Step 1: Zone all your storage provider on a host
• Step 2: Map your OS Image lun on this host (from the source Storage Provider)
• Step 3: Map a new lun (from the desitnation Storage Provider)
• Step 4: Use dd to copy the data from on lun to one another. (note the block size)
– /dev/mapper/mpathe -> lun of the OS Image (source Storage Provider)
– /dev/mapper/mpathc -> new lun (destination Storage Provider)
# dd if=/dev/mapper/mpathe of=/data/download/aix7100-03-04-cloudinit-chef bs=4M
# dd if=/data/download/aix7100-03-04-cloudinit-chef of=/dev/mapper/mpathc bs=4M
• Step 5: Import the new image on PowerVC
– Manage the new lun with « Manage Existing »
# powervc-volume-image-import --name rhel64 --os rhel --volume volume_capture2 --activationtype ae Password: Image creation complete for image id: e3a4ece1-c0cd-4d44-b197-4bbbc2984a34
© Copyright IBM Corporation 2015
35
Linked clones 1/2
• PowerVC is the only way to created « linked clone » VM
• You need to have a Shared Storage Pool as a storage
provider. How does it work:
– Step 1: The captured rootvg underlying disk is a Shared Storage Pool Logical Unit.
– Step 2: When the image is captured the rootvg Logical Unit is copied and is known as a “Logical
(Client Image) Unit”.
– Step 3: When deploying a new machine a snapshot is created from the Logical (Client Image) Unit.
– Step 4: A “special Logical Unit” is created from the snapshot. This Logical Unit seems to be a
pointer to the snapshot. We call it a clone.
– Step 5:The machine is booted and the activation engine is running and reconfiguring the network
– Step 6:When a block is modified on the new machine this one is duplicated and modified on one
new block on the Shared Storage Pool.
– Step7: This said if no blocks are modified all the machines created from this capture are sharing
the same blocks on the Shared Storage Pool.
– Step 8: Only modified blocks are not shared between Linked Clones. The more things you will
change on your rootvg the more space you will use on the Shared Storage Pool.
– Step 9:That’s why these machines are called Linked Clones : they all are connected by the same
source Logical Unit.
– Step 10:You will save TIME (just a snapshot creation for the storage side)
and SPACE (all rootvg will be shared by all the deployed machines) by using Linked Clones.
© Copyright IBM Corporation 2015
36
Linked clones 2/2
© Copyright IBM Corporation 2015
37
Debugging PowerVC
• Best things to do is to check logs:
– All openstack logs are located in /var/log
• Nova
• Cinder
• …
• Things I’ve learned by experience:
– Check VIOS rootvg /var space (PowerVC need space in /var)
– PowerVC needs responsive VIOS. By this I mean if some commands
are too long to answer, PowerVC will consider this VIOS as busy and
will try a new one. (And indicate it may be an RMC problem in the log)
– In the Virtual I/O Server check the logs located in /home/ios/logs
• viosvc.log
• viod_CM.log (if you have .FATAL files you can check them to find the
issue)
© Copyright IBM Corporation 2015
38
Backup/Restore strategy
• My advice:
–
–
–
–
Create two Linux hosts (on different sites)
Install PowerVC at the same level on each hosts
If you update one PowerVC update the backup too
On the production one run weekly backups:
# crontab –l
0 0 * * 6 rm -rf /var/opt/ibm/powervc/backups/* ; /opt/ibm/powervc/bin/powervcbackup --targetDir /var/opt/ibm/powervc/backups --noPrompt
– Copy this backup on a third host or backup it with TSM
– If you have an outtage of the first PowerVC restore the configuration
on the second one
# powervc-restore --targetDir /var/opt/ibm/powervc/backups/
Continuing with this operation will stop all PowerVC services and overwrite
critical PowerVC data in both the database and the file system. Do you want to
continue? (y/N):y
PowerVC services stopped.
[..]
© Copyright IBM Corporation 2015
39
Dynamic Resource Optimizer 1/2
• Had the chance to participate in the beta test program
• DRO is checking host utilization and optimize (rebalance) the
placement of Virtual Machine, or of CPU:
– By using Live Partition Mobility
– By using Power Entreprise Pool (mobiles cores)
• DRO can by run in:
– Active mode: VM will be moved, cores will be moved.
– Advise mode: DRO will advise you in the logs
• Specific machines, host, or host group can be excluded from
DRO.
• For the moment DRO is the checking the CPU usage:
– Criteria : Threshold, stabilization, run interval
© Copyright IBM Corporation 2015
40
Dynamic resource Optimizer 2/2
© Copyright IBM Corporation 2015
41
Redbook & resources
• My blog posts about PowerVC :
http://chmod666.org/index.php/tag/powervc/
• Redbook:
http://www.redbooks.ibm.com/redpieces/pdfs/sg248199.pdf
• Linked-in PowerVC group:
https://www.linkedin.com/grp/home?gid=6610502
• PowerVC DW Wiki (best place for documentation):
https://www.ibm.com/developerworks/community/wikis/home?l
ang=en_us
• Twitter:
– @IBMPowerVC
© Copyright IBM Corporation 2015
42
Thank you !
Special thanks to (for their support or help)
- All the PowerVC team in Austin (Joe, Christine, Sam, Gerald)
- Jay Kruemcke @chromeaix @cloudrancher
- Philippe Hermes @phhermes
- Nigel Griffiths @mr_nmon / Gareth Coates @power_gaz
- Chris Gibson @cgibbo
- Rosa Davidson @radavids1
- Eddie Shvartsman
© Copyright IBM Corporation 2015
43
Continue growing your IBM skills
ibm.com/training provides a
comprehensive portfolio of skills and
career
accelerators that are designed to
meet all
your training needs.
• Training in cities local to you - where and
when you need it, and in the format you want
– Use IBM Training Search to locate public training classes
near to you with our five Global Training Providers
– Private training is also available with our Global Training Providers
• Demanding a high standard of quality –
view the paths to success
– Browse Training Paths and Certifications to find the
course that is right for you
• If you can’t find the training that is right for you with
our Global Training Providers, we can help.
– Contact IBM Training at dpmc@us.ibm.com
Global Skills
Initiative
© Copyright IBM Corporation 2015
44
© Copyright IBM Corporation
2015