New Features in AIX 7.1

Cluster Aware AIX (CAA)

CAA simplifies cluster configuration and improves the performance/reliability of cluster services on AIX. Having built several HACMP (PowerHA) clusters over the years, I can tell you that this new feature will make an AIX PowerHA administrator's life a lot easier. This new set of cluster services and tools is going to reduce cluster administration effort. It will also provide third-party cluster software with a more efficient method of integrating with AIX and forming clusters across multiple AIX nodes. 

Some of the major changes to managing, monitoring and configuring clusters on AIX 7.1 are centered around things like a new multicast address for cluster communications between the nodes in the cluster to improve node communication, a new event infrastructure architecture for monitoring and responding to cluster events (like a node down for example) using a new pseudo filesystem interface, known as Autonomic Health Advisory File System (AHAFS), and a shared central cluster configuration repository that is accessible by all nodes in a cluster, which improves cluster configuration consistency.

Once you have configured a basic cluster, using the new mkcluster command, the cluster exports cluster messaging and cluster socket services to other applications, such as Reliable Scalable Cluster Technology (RSCT) and PowerHA SystemMirror. PowerHA SystemMirror 7.1 will be the first release of PowerHA that takes advantage of CAA 

Terabyte segment support

AIX 7 extends the capabilities of the AIX OS to expand the vertical scalability of AIX to partitions with 256 processor cores and 1024 threads to handle the largest workloads. To support higher performance for large workloads, AIX 7 also includes new Terabyte segment support which leverages memory management capabilities of POWER7 processors designed to improve memory performance. This Terabyte segment capability is also included in AIX 6 at Technology Level 6 but is not automatically enabled on AIX 6.

AIX5.3 on AIX7.1

AIX 7 also includes new virtualization capabilities designed to simplify the consolidation of older, AIX V5.3 environments. This new capability, which requires the purchase of the “AIX 5.3 Workload Partitions for AIX 7” product, is designed to allow administrators to simply back up an existing LPAR running AIX 5.3 and restore it into an AIX 7 Workload Partition.

Domain RBAC

This enhancement to RBAC -- which was first introduced in AIX 6.1 with Enhanced RBAC -- is an interesting new feature. It will allow administrators even greater control over the delegation of certain system administration tasks. With RBAC in 6.1 you could allow unprivileged users to execute privileged commands. With Domain RBAC we can further isolate these users to certain resources on a system. For example, I can delegate the use of the ifconfig command to a non-root user so that they can manage network interfaces. Without Domain RBAC this user could use ifconfig to manipulate any network interface on the system. Domain RBAC allows me to isolate the user to a specific interface only, for example, en1. So now this non-root user can only use ifconfig against the en1 interface. They can bring en1 up and/or down as required, but if they attempt to do the same with any other interface, such as en0, they will be prevented from accessing the interface.

DSM/NIM integration

Distributed Systems Management (DSM) actually replaces the Cluster System Management (CSM) filesets that have been part of AIX for many years. DSM integrates with IBM Systems Director. It will help Director automate the installation of AIX LPARS via NIM. However, you can also take advantage of this functionality by using NIM and DSM together. There are several new commands that can help you streamline AIX installation via NIM. If you are experienced with using NIM, you may be familiar with the process of enabling a NIM client BOS installation and then accessing the console and initiating a network boot. With DSM, you can now automate the network boot and avoid having to manually access the SMS menu. It will make your NIM AIX installations more efficient. You can also open remote console sessions with your LPARs, directly from a NIM master.

Renaming devices

It is now possible to rename devices in AIX. You can rename most devices, such as disks and adapters. For example, you could rename hdisk5 to hdisk0, or rename vscsi3 to vscsi8. This is really useful, especially in clustered environments. It is often the case that hdisks in an AIX cluster may end up with the different names across cluster nodes. As an example, in a 2 node cluster, you could have a shared disk known to one node as hdisk1 and hdisk4 to the other. The PVID/LUN id is identical but the hdisk name is different. This can be confusing for the administrator and could lead to mistakes. With the rendev command, you can now rename the devices so that the hdisk names are identical across cluster nodes. Cluster Aware AIX takes advantage of this feature and automatically keeps cluster device names (hdisk and network interfaces) in sync across all nodes in a cluster. 

Outside of clustered environment, the rendev command could be used to rename a bunch of disks on a system so that there a no gaps in the names. For example, you may have a system with hdisk0, hdisk2, hdisk4, and hdisk5, which looks like someone removed some disks from this system over time. It would be nice if the hdisk numbers were all in sequence: hdisk0, hdisk1, hdisk2, and hdisk3. The rendev command will allow you to rename hdisk2 to hdisk1, hdisk4 to hdisk3, and hdisk5 to hdisk4. This is a lot simpler than having to rmdev the device, run cfgmgr, and hope the disks are discovered again in the correct order. 

NIM not Bound to rsh

The second announcement -- buried in a single line in the AIX 7 Data Sheet -- has been a long time coming. The Network Installation Manager -- NIM -- has (finally!) had its dependency on rsh removed. Up till now, if you wanted to use NIM to migrate to a new release of AIX (for example, from AIX 5.3 or 6.1 to 7.1), you had to enable remote commands on the NIM client using the rshd protocol. For many organizations, this was an unacceptable risk, even if the rsh ports were only opened for the short time of the migration itself. For such security-sensitive sites, an AIX migration via NIM was simply not an option. Now the limitation of having to use rsh has been removed.

Encrypted File System (EFS) on AIX

The Encrypted File System (EFS) is a J2 filesystem-level encryption through individual key stores. This allows for file encryption in order to protect confidential data from attackers with physical access to the computer. User authentication and access control lists can protect files from unauthorized access while the operating system is running; however, it’s easy to circumvent the control lists if an attacker gains physical access to the computer

One solution is to store the encrypted files on the disks of the computer. In EFS, a key is associated to each user. These keys are stored in a cryptographically protected key store and upon successful login, and the user's keys are loaded into the kernel and associated with the process credentials. When the process needs to open an EFS-protected file, the system tests the credentials. If the system finds a key matching the file protection, the process is able to decrypt the file key and file content. The cryptographic information is kept in the extended attributes for each file. EFS uses extended attribute Version 2, and each file is encrypted before being written on the disk. The files are decrypted when they are read from the disk into memory so that the file data kept in memory is in clear format. The data is decrypted only once, which is a major advantage. When another user requires access to the file, their security credentials are verified before being granted access to the data even though the file data is already in memory and in clear format. If the user is not entitled to access the file, the access is refused. File encryption does not eliminate the role of traditional access permissions, but it does add more granularity and flexibility.

In order to be able to create and use the EFS-enabled file system on a system, the following prerequisites must be met:

• Install the CryptoLite in C (CliC) cryptographic library.
• Enable the RBAC.
• Enable the system to use the EFS file system.

How is AIX EFS different from others available in the market?

AIX® EFS encryption is at the file system level. Each file is protected with a unique file key, and protection is created against malicious root.

Frequently used commands

efsenable

The efsenable command activates the EFS capability on a system. It creates the EFS administration keystore, the user keystore, and the security group keystore. Keystore is a key repository that contains EFS security information. The access key to the EFS administration keystore is stored in the user keystore and the security group keystore. The efsenable command creates the /var/efs directory. The /etc/security/user and /etc/security/group files are updated with new EFS attributes on execution of this command.

efskeymgr

The efskeymgr command is dedicated to all key management operations needed by an EFS. The initial password of a user keystore is the user login password. Group keystores and admin keystores are not protected by a password but by an access key. Access keys are stored inside all user keystores that belong to this group.

When you open a keystore (at login or explicitly with the efskeymgr command), the private keys contained in this keystore are pushed to the kernel and associated with the process. If access keys are found in the keystore, the corresponding keystores are also opened and the keys are automatically pushed into their kernel.

efsmgr

The efsmgr command is dedicated to the files encryption and decryption management inside EFS. Encrypted files can only be created on the EFS-enabled JFS2 file systems. Inheritance is set on the file system or the directory where the file is being created using this command. When inheritance is set on a directory, all new files created in this directory are encrypted by default. The cipher used to encrypt files is the inherited cipher. New directories also inherit the same cipher. If inheritance is disabled on a subdirectory, the new files created in this subdirectory will not be encrypted.

Setting or removing inheritance on a directory or a file system has no effect on the existing files. The efsmgr command must be used explicitly to encrypt or decrypt files.

Sample scenario

Let's take a scenario of a company that has three departments, namely sales, marketing, and finance. These three departments share the same AIX machine to store their confidential content. If EFS is not enabled, the potential of having the data exposed between the three departments is extremely high. See Listing 1 below to learn how to make this threat-prone machine become a safe location to store data.

Enabling EFS

To enable EFS on AIX, type the following:

Listing 1. EFS enablement in AIX

# efsenable -a Enter password to protect your initial keystore: Enter the same password again:

Enter the following to see the directories created to facilitate EFS:

# cd /var/efs # ls efs_admin efsenabled groups users

All of the EFS capabilities should now be enabled.

You are now going to create a separate file system for all the three departments. The creation of an EFS is similar to the creation of a normal file system. The only difference is that you have to enable the EA2 efs = yes attribute.

Listing 2 illustrates how to create an encrypted file system through the System Management Interface Tool (SMIT):

Listing 2. EFS creation through SMIT

Add an Enhanced Journaled File System Type or select values in entry fields. Press Enter AFTER making all desired changes. [Entry Fields] Volume group name rootvg SIZE of file system Unit Size Megabytes + * Number of units [100] # * MOUNT POINT [/sales] Mount AUTOMATICALLY at system restart? no + PERMISSIONS read/write + Mount OPTIONS [] + Block Size (bytes) 4096 + Logical Volume for Log + Inline Log size (MBytes) [] # Extended Attribute Format + ENABLE Quota Management? no + Enable EFS? yes + Allow internal snapshots? no +

You can also create the same file system through the command line, as shown here in Listing 3:

Listing 3. EFS creation through the command line

#crfs -v jfs2 -g rootvg -m /sales -a size=100M -a efs=yes #crfs -v jfs2 -g rootvg -m /marketing -a size=100M -a efs=yes #crfs -v jfs2 -g rootvg -m /finance -a size=100M -a efs=yes

You have now successfully created three separate file systems for these three departments.

Creating keystores for users and groups

In order to handle or maintain these individual file systems, you need create three different users and create a keystore for them (see Listing 4). (A key store for an user is created when a password is set for that user).

Listing 4. Creation of users

#mkuser salesman #passwd salesman #mkuser marketingman #passwd marketingman #mkuser financeman #passwd financeman

This creates three separate keystores for these three users in the /var/efs/users directory (see Listing 5).

Listing 5. Keystore location for users

# pwd /var/efs/users # ls .lock salesman marketingman financeman root

You can also create keystores for the groups with EFS (see Listing 6).

Listing 6. Keystore creation for groups

#efskeymgr -C finance # pwd /var/efs/groups # ls .lock finance security

Creation of a keystore for a group requires at least one user under it.

Creating EFS directories and setting properties

This section shows how you can create encrypted files and directories in the EFS file system and manipulate their properties. In order to create EFS directories, you need the EFS file system to be mounted (see Listing 7).

Listing 7. Creating the EFS directory

# mount /finance # cd /finance #mkdir yearlyreport # efsmgr -E yearlyreport # efsmgr -L yearlyreport EFS inheritance is set with algorithm: AES_128_CBC

The yearlyreport directory is now set for inheritance. It indicates that a file or directory inherits both the property of encryption and all encryption parameters from its parent directory.

There are various options with efsmgr, which facilitates you to set the type of cipher to be used on this directory, enable and disable inheritance, and add or remove users and groups from the EFS access list of this directory.

Encrypting individual files

In order to carry out any EFS-related activity, you need to load the keystore. If you try to create a file inside this encrypted directory without having access to the keystore that protects it, the following will result:

# cd yearlyreport # ls # touch apr_report touch: 0652-046 Cannot create apr_report.

This happens when you don't have the keystore loaded to perform the EFS activity (see Listing 8).

Listing 8. Loading EFS keystore to the shell

# efskeymgr -o ksh financeman's EFS password: # touch apr_report

Now that you have loaded the keystore, any information that is added to this file is encrypted at the file system level (see Listing 9).

Listing 9. Encrypted file in EFS

# ls -U apr_report -rw-r--r--e 1 financeman system 0 Nov 28 06:14 apr_report

The "e" set for this file means that it's encrypted and no one other than the owner who possesses the key store can access and read its content (see Listing 10).

Listing 10. Listing encrypted file attributes

# efsmgr -l apr_report EFS File information: Algorithm: AES_128_CBC List of keys that can open the file: Key #1: Algorithm : RSA_1024 Who : uid 0 Key fingerprint : 4b6c5f5f:63cb8c6f:752b37c3:6bc818e1:7b4961f9

With the different flags available with the efsmgr command, you can change the cipher and other attributes of the file. If you want to create a file that does not come under any encrypted directory, then you need to use the following option to encrypt such standalone files (see Listing 11):

Listing 11. Encrypting a single file

#cd /finance #touch companylist # ls -U total 16 -rw-r--r--- 1 root system 8 Nov 28 06:21 companylist drwxr-xr-x- 2 root system 256 Nov 28 05:52 lost+found drwxr-xr-xe 2 root system 256 Nov 28 06:14 yearlyreport # efsmgr -c AES_192_ECB -e companylist # ls -U companylist -rw-r--r--e 1 root system 8 Nov 28 06:24 companylist

Facilitating the access of other users for your files

Now you have seen that each department has created a separate file system and has a keystore to guard them. If the scenario requests that a person from finance wants to access the encrypted files from sales, then you need to be able to grant him or her permission to do so (see Listing 12 and Listing 13).

Listing 12. vi output when the file is encrypted

#vi sales_report ~ ~ ~ ~ ~ ~ ~ ~ "sales_report" Security authentication is denied.

Listing 13. Passing keystore access to another user

# efskeymgr -k user/salesman -s user/financeman

This command now sends the access key of the "salesman" user to the "financeman" user.

If you try to edit a file owned by salesman, you can read and access the content in its plain format, as you now possess the keystore of the user who created the file (see Listing 14).

Listing 14. vi output after receiving keystore access

#vi sales_report Sales report for this financial year ~ ~ ~ ~ ~ ~ ~ ~ "sales_report" [Read only] 1 line, 36 characters

Granting and revoking access to individual files

Instead of sending the complete access key to another user, you can also set access permissions on individual files residing on EFS.

Let's now suppose you have a file in the /marketing filesystem directory and you wish to give access to a particular /marketing/strategy.txt file to the "salesman" user and to the "finance" group. In order to accomplish this task, you need to reviewListing 15 and Listing 16.

Listing 15. Granting access to an user

# efsmgr -l strategy.txt EFS File information: Algorithm: AES_128_CBC List of keys that can open the file: Key #1: Algorithm : RSA_1024 Who : uid 0 Key fingerprint : 4b6c5f5f:63cb8c6f:752b37c3:6bc818e1:7b4961f9 # efsmgr -a strategy.txt -u salesman # efsmgr -l strategy.txt EFS File information: Algorithm: AES_128_CBC List of keys that can open the file: Key #1: Algorithm : RSA_1024 Who : uid 0 Key fingerprint : 4b6c5f5f:63cb8c6f:752b37c3:6bc818e1:7b4961f9 Key #2: Algorithm : RSA_1024 Who : uid 204 Key fingerprint : f91b5a79:53bdd7f1:58987a33:f5701a38:99145b24

Listing 16. Granting access to a group

# efsmgr -a strategy.txt -g finance # efsmgr -l strategy.txt EFS File information: Algorithm: AES_128_CBC List of keys that can open the file: Key #1: Algorithm : RSA_1024 Who : uid 0 Key fingerprint : 4b6c5f5f:63cb8c6f:752b37c3:6bc818e1:7b4961f9 Key #2: Algorithm : RSA_1024 Who : uid 204 Key fingerprint : f91b5a79:53bdd7f1:58987a33:f5701a38:99145b24 Key #3: Algorithm : RSA_1024 Who : gid 201 Key fingerprint : 8cb65011:2a42e9f0:91f7b712:20e36bb7:5eb0db0a

If you need to revoke the access that was provided to the "finance" group, then use the "-r" flag with the efsmgr command, as shown in Listing 17 below.

Listing 17. Revoking access to a group

# efsmgr -r strategy.txt -g finance # efsmgr -l strategy.txt EFS File information: Algorithm: AES_128_CBC List of keys that can open the file: Key #1: Algorithm : RSA_1024 Who : uid 0 Key fingerprint : 4b6c5f5f:63cb8c6f:752b37c3:6bc818e1:7b4961f9 Key #2: Algorithm : RSA_1024 Who : uid 204 Key fingerprint : f91b5a79:53bdd7f1:58987a33:f5701a38:99145b24

Renaming Devices in AIX7.1

It is now possible to rename devices in AIX. You can rename most devices, such as disks and adapters. For example, you could rename hdisk5 to hdisk0, or rename vscsi3 to vscsi8. This is really useful, especially in clustered environments. It is often the case that hdisks in an AIX cluster may end up with the different names across cluster nodes. As an example, in a 2 node cluster, you could have a shared disk known to one node as hdisk1 and hdisk4 to the other. The PVID/LUN id is identical but the hdisk name is different. This can be confusing for the administrator and could lead to mistakes. With the rendev command, you can now rename the devices so that the hdisk names are identical across cluster nodes. Cluster Aware AIX takes advantage of this feature and automatically keeps cluster device names (hdisk and network interfaces) in sync across all nodes in a cluster. 

Outside of clustered environment, the rendev command could be used to rename a bunch of disks on a system so that there a no gaps in the names. For example, you may have a system with hdisk0, hdisk2, hdisk4, and hdisk5, which looks like someone removed some disks from this system over time. It would be nice if the hdisk numbers were all in sequence: hdisk0, hdisk1, hdisk2, and hdisk3. The rendev command will allow you to rename hdisk2 to hdisk1, hdisk4 to hdisk3, and hdisk5 to hdisk4. This is a lot simpler than having to rmdev the device, run cfgmgr, and hope the disks are discovered again in the correct order.

Shared Ethernet Adapter failover with Load Sharing

The Virtual I/O Server Version 2.2.1.0, or later, provides a load sharing function to enable to use the bandwidth of the backup Shared Ethernet Adapter (SEA).

The hitherto known SEA failover configuration provides redundancy by configuring a primary and backup SEA pairon Virtual I/O Servers (VIOS). The backup SEA is in standby mode, and is used when the primary SEA fails. The bandwidth of the backup SEA is not used in normal operation.

Shows a basic SEA failover configuration. All network packets of all Virtual I/O clients are bridged by the primary VIOS.

On the other hand, SEA failover with Load Sharing makes effective use of the backup SEA bandwidth, as shown in Figure. In this example, network packets of LPAR1and LPAR2are bridged by VIOS2, and LPAR3is bridged by VIOS1

Prerequisites and requirements for SEA failover with Load Sharing are as follows:

• ·         Both primary and backup Virtual I/O Servers are at Version 2.2.1.0, or later.
• ·         Two or more trunk adapters are configured for the primary and backup SEA pair.
• ·         Load sharing mode must be enabled on both primary and backup SEA pair.
• ·         The virtual local area network (VLAN) definitions of the trunk adapters are identical between the primary and backup SEA pair.

Important: You need to set the same priority to all trunk adapters under one SEA. The primary and backup priority definitions are set at the SEA level, not at trunk adapters level.

You can check these prerequisites and requirements in this sample, SEA failover with Load Sharing configuration shown as in Figure.

·         Both VIOS1and VIOS2should be at Version 2.2.1.0, or later.

·         Two trunk adapters, Adapter Aand B, are configured on the primary SEA on VIOS1, and

Adapter C and D are configured on the backup SEA on VIOS2.

·         All of the VLAN definitions of trunk adapters match. The primary SEA on VIOS1has Adapter A with VLANs 10and 20, and the backup SEA on VIOS2 has Adapter C with VLANs 10and 20. The Adapter B and D is the same.

Configuring SEA failover with Load Sharing mode is the same as configuring SEA in failover mode. You set  ha_mode to sharing insted of auto when you create a SEA.

Creating an SEA (ent7) with Load Sharing mode

$ mkvdev -sea ent1 -vadapter ent4,ent5 -default ent4 -defaultid 10 -attr ha_mode=sharing ctl_chan=ent6

ent7 Available

$ lsdev -dev ent7 -attr

attribute value description user_settable

accounting disabled Enable per-client accounting of network statistics True

ctl_chan ent6 Control Channel adapter for SEA failover True

gvrp no Enable GARP VLAN Registration Protocol (GVRP) True

ha_mode sharing High Availability Mode True

jumbo_frames no Enable Gigabit Ethernet Jumbo Frames True

large_receive no Enable receive TCP segment aggregation True

largesend 1 Enable Hardware Transmit TCP Resegmentation True

lldpsvc no Enable IEEE 802.1qbg services True

netaddr 0 Address to ping True

pvid 10 PVID to use for the SEA device True

pvid_adapter ent4 Default virtual adapter to use for non-VLAN-tagged packets True

qos_mode disabled N/A True

real_adapter ent1 Physical adapter associated with the SEA

True

thread 1 Thread mode enabled (1) or disabled (0) True

virt_adapters ent4,ent5 List of virtual adapters associated with the SEA (comma separated) True

You have already configured an SEA failover environment in failover mode, you can change the ha_modeattribute from autoto sharingby using the chdev command dynamically. You can also add a new trunk adapter to the existing SEA if you need, as shown in Example.

If you need to disable it while it is running in load sharing mode, set the ha_mode to any value other than sharing, for example standby, auto, or disable.

Important:To create or enable the SEA failover with Load Sharing, you have to enable the load sharing mode on the primary SEA first before enabling load sharing mode on the backup SEA.

To change the ha_mode from sharing to auto, disable the load sharing mode, and set ha_modeto autoon the primary SEA first. Then set it on the backup to minimize the chance of a broadcast storm of the SEA.

Adding a trunk adapter and changing SEA (ent6) failover mode

Adding a trunk adaper to the SEA.

ent6: SEA

ent4: trunk adapter which is already part of the SEA

ent7: new trunk adapter adding to the SEA

$ chdev -dev ent6 -attr virt_adapters=ent4,ent7

ent6 changed

Changing the SEA to Load Sharing mode.

$ chdev -dev ent6 -attr ha_mode=sharing

ent6 changed

$ lsdev -dev ent6 -attr

attribute value description user_settable

accounting disabled Enable per-client accounting of network statistics True

ctl_chan ent5 Control Channel adapter for SEA failover True

gvrp no Enable GARP VLAN Registration Protocol (GVRP) True

ha_mode sharingHigh Availability Mode True

jumbo_frames no Enable Gigabit Ethernet Jumbo Frames True

large_receive no Enable receive TCP segment aggregation True

largesend 1 Enable Hardware Transmit TCP Resegmentation True

lldpsvc no Enable IEEE 802.1qbg services True

netaddr 0 Address to ping True

pvid 10 PVID to use for the SEA device True

pvid_adapter ent4 Default virtual adapter to use for non-VLAN-tagged packets True

qos_mode disabled N/A True

real_adapter ent1 Physical adapter associated with the SEA True

thread 1 Thread mode enabled (1) or disabled (0) True

virt_adapters ent4,ent7List of virtual adapters associated with the SEA (comma separated) True

The entstatcommand provides detailed information for the current SEA status such as State, Trunk Adapter Priority, and VLAN IDs. The output of entstat consists of some statistics for physical and virtual adapters in the SEA, as shown in Example.

Statistics for adapters in the Shared Ethernet Adapter

$ entstat -all ent6

ETHERNET STATISTICS (ent6) :

Device Type: Shared Ethernet Adapter

Hardware Address: 00:1a:64:bb:69:49

Elapsed Time: 0 days 10 hours 44 minutes 30 seconds

Transmit Statistics: Receive Statistics:

-------------------- -------------------...

Statistics for adapters in the Shared Ethernet Adapter ent6

--------------------------------------------------------------...

VLAN Ids :

ent4: 10

ent7: 30 130

Real Side Statistics:

Packets received: 34275

...

Type of Packets Received:

...

Limbo Packets: 0

State: PRIMARY_SH

Bridge Mode: Partial

VID shared: 10

Number of Times Server became Backup: 0

Number of Times Server became Primary: 1

High Availability Mode: Sharing

Priority: 1

Real Adapter: ent1

ETHERNET STATISTICS (ent1) :

Device Type: 4-Port 10/100/1000 Base-TX PCI-X Adapter (14101103)

...

Virtual Adapter: ent4

ETHERNET STATISTICS (ent4) :

Device Type: Virtual I/O Ethernet Adapter (l-lan)

...

Virtual I/O Ethernet Adapter (l-lan) Specific Statistics:

---------------------------------------------------------RQ Length: 4481

Trunk Adapter: True

Priority: 1 Active: True

Filter MCast Mode: False

...

Port VLAN ID: 10

VLAN Tag IDs: None

...

--------------------------------------------------------------Virtual Adapter: ent7

ETHERNET STATISTICS (ent7) :

Device Type: Virtual I/O Ethernet Adapter (l-lan)

...

Virtual I/O Ethernet Adapter (l-lan) Specific Statistics:

RQ Length: 4481

Trunk Adapter: True

Priority: 1 Active: False

Filter MCast Mode: False

...

Port VLAN ID:130

VLAN Tag IDs: 30

...

Control Channel Adapter: ent5

...

This example shows that the SEA consistsof one physical adapter, two trunk adapters with VLAN ID 10and 30, and one control channel adapter. The details are as follows:

·         State: PRIMARY_SH

The “_SH“ means that the SEA is running in load sharing mode. You can also see the status as High Availability Mode: Sharing.

·         Priority:1 Active: True

This shows that the trunk adapter is configured as a partof the primary SEA and the adapter is activated.

·         Priority:1 Active: False

This shows that the trunk adapter is configured as a partof the primary SEA and the adapter is deactivated.

Replace failed disk in VIOS in Virtualization

This article describes two common scenarios for replacing failing local disk in VIOS.

Failed disk in VIO server which is used by VIO client(s)

In this scenario the failing disk contains LVs used for rootvg of VIO clients. The rootvg is mirrored to another disk presented by a second VIO server. This scenario is illustrated below:

Follow the procedure to replace failing disk on VIO server:

On VIOS, as padmin user:

Get and record information which will be needed for later operations and recreation of devices and configurations:
$ lsdev -virtual

To get volume group name in which failed disk participate:
$ lspv

To get list of logical volumes on disk:
$ lspv -lv <hdisk#>

To get info about logical volumes, e.g. size (number of LPs):
$ lsvg -lv <VGname>
$ lsvg <VGname>

To get info about LVs, VTD names, vhost numbers and virtual clients:
$ lsmap -all

On client(s):Identify affected disk(s)(LVs on bad disk on VIOS):
# lscfg -vl <hdisk#>  (for all virtual SCSI disks)
hdisk1           U9117.MMA.999999-V2-C12-T1-L8200000000000000  Virtual SCSI Disk Drive


Take note of the following:
V# - LPAR ID (this should be the LPAR ID of the affected VIOS)
C# - slot number
L# - LUN ID


# lspv


The affected disk may be listed as removed or missing depending on the failure.
# lsvg -p rootvg

Remove the bad disk from the mirror:
# unmirrorvg rootvg <hdisk#>
# reducevg rootvg <hdisk#>
# rmdev -dl hdisk#

On VIOS:

Remove all VTDs and LVs that reside on the failed disk:
$ rmvdev -vtd <VTDname> -rmlv
or
$ rmdev -dev <VTDname>
$ rmdev -dev <LVname>


Check if all logical volumes are removed form bad disk:
$ lspv -lv <hdisk#>


Remove the disk from the respective volume group:
$ reducevg <VGname> <hdisk#>
Note: If the volume group consists of only one disk then the whole VG will need to be removed from ODM. In that case use the following commands:

$ deactivatevg <VGname>
$ exportvg <VGname>
Replace the failed disk:
$ diagmenu
--> select “Task Selection”
               --> select “Hot Plug Task”
                         --> select “SCSI and SCSI RAID Hot Plug Manager”
                                    --> Replace/Remove a Device Attached to an SCSI Hot Swap Enclosure


Configure the new disk:
$ cfgdev

Add the new disk to the volume group or recreate the VG in case it was removed:
$ extendvg <VGname> <new hdisk#>
or
$ mkvg -vg <VGname>  <new hdisk#>


Recreate the LVs with the same names and size which we got in the beginning.
$ mklv -lv <LVname> <VGname> <size>


Recreate the VTDs:
$ mkvdev -vdev <LVname> -vadapter <vhost#> -dev <VTDname>

On client(s):


Discover new disk(s) and rebuid mirror:
# cfgmgr
# extendvg rootvg <new hdisk>
# mirrorvg rootvg <new hdisk#>


Build boot image on both mirrored disks (just in case):
# bosboot -ad /dev/<hdisk0>
# bosboot -ad /dev/<hdisk1>

Set bootlist:
# bootlist -m normal <list names of both hdisks>
# bootlist -m normal -o

Bad disk in rootvg of VIO server

Usually rootvg utilize some kind of disk protection. Most often rootvg consists of disks which are LVM mirrored. To replace a mirrored hdisk in rootvg of VIO server you can use VIO commands or root AIX commands (to become root, use oem_setup_env command). In this example we will use VIO commands since this is the recommended way of managing VIOS.

Break the mirror:
$ unmirrorios <hdisk#>  , where <hdisk#> is the bad disk
 Check if any LV remained on the bad disk:
$ lspv -lv <hdisk#>
If there are any (e.g. lg_dumplv - dump device) migrate them to the other disk or remove them (dump device can be recreated later):

$ migratepv -lv <LV> <bad_hdisk> <good_hdisk>
or
$ rmlv -f <LV>


 Take out failed disk from rootvg:
$ reducevg rootvg <bad_hdisk>
 Use ”Hot Plug” procedure to replace the failed disk:


$ diagmenu
--> select “Task Selection”
               --> select “Hot Plug Task”
                         --> select “SCSI and SCSI RAID Hot Plug Manager”
                                    --> Replace/Remove a Device Attached to an SCSI Hot Swap Enclosure


Configure the new disk:
$ cfgdev


Verify that the new disk came back with the same number as the previous one:
$ lspv
$ extendvg rootvg <hdisk#>
$ mirrorios -defer <hdisk#>  (Note that if you do not use -defer option, your VIO server will be rebooted after mirroring completes)
Check bootlist to ensure that both disks are included as boot devices:

$ bootlist -mode normal -ls
hdisk0 blv=hd5
hdisk1 blv=hd5


Use the command below to include both disks if they do not show up in the bootlist:

$ bootlist -mode normal hdisk0 hdisk1