Bonding the Management Interface

Scenario #2, Tutorial #2, Part #5

Per Citrix’s recommendation, we’ll be configuring an active-passive bond for the PMI. The IP address that was configured for the PMI during installation will be inherited by the newly-created bond.

This process is nearly identical to…

  1. the process for creating the bonded Storage interface except that, when we create the bond for the PMI; we’ll be using the UUID of the interface identified by XAPI as the PMI (instead of merely specifying a physical interface).
  2. the process for creating the bonded Guest interface except that
    1. When we create the bond for the PMI; we’ll specify that the bond mode as ‘active-passive‘ instead of ‘active-active‘.
    2. The newly-created, non-physical interface will inherit an IP address from the physical interface that makes up one-half of the bond – the current PMI, eth0.

On the pool-master:

  1. Create a network specifically for the management traffic and store the network’s UUID in a shell variable for use later in this process:


    [glossary_exclude]

    Legend: GET | SEE | USE

    [root@xs-1 ~]# MGT_NET_UUID=$(xe network-create name-label=MGT)
    [root@xs-1 ~]# echo -e “Management Network UUID: $MGT_NET_UUID\n”
    Management Network UUID: 01ec20b5-1c91-d311-5d28-df7b6745b910

    [/glossary_exclude]

  2. Identify the UUID’s of each of the NIC’s that we’ll be bonding and store them in shell variables for use later in this process:


    [glossary_exclude]

    Legend: GET | SEE | USE

    [root@xs-1 ~]# PMI_PIF_UUID=$(xe pif-list host-uuid=$INSTALLATION_UUID management=true params=uuid | awk ‘{ print $5 }’)
    [root@xs-1 ~]# MGT_PIF0_UUID=$(xe pif-list host-uuid=$INSTALLATION_UUID device=eth0 params=uuid | awk ‘{ print $5 }’)
    [root@xs-1 ~]# MGT_PIF3_UUID=$(xe pif-list host-uuid=$INSTALLATION_UUID device=eth3 params=uuid | awk ‘{ print $5 }’)
    [root@xs-1 ~]# echo -e “Management PIF UUID’s:\n PMI: $PMI_PIF_UUID\n PIF0: $MGT_PIF0_UUID\n PIF3: $MGT_PIF3_UUID\n”
    Management PIF UUID’s:
    PMI: 8445b796-01b0-e2cf-ba17-9e7af7856925
    PIF0: 8445b796-01b0-e2cf-ba17-9e7af7856925
    PIF3: a9ffe301-bec3-807f-c825-702517196ead

    [/glossary_exclude]

  3. Create the bond and store the bond’s UUID in a shell variable for use later in this process:


    [glossary_exclude]

    Legend: GET | SEE | USE

    [root@xs-1 ~]# MGT_BOND_UUID=$(xe bond-create network-uuid=$MGT_NET_UUID pif-uuids=$PMI_PIF_UUID,$MGT_PIF3_UUID mode=active-backup)
    [root@xs-1 ~]# MGT_MSTR_UUID=$(xe pif-list host-uuid=$INSTALLATION_UUID network-uuid=$MGT_NET_UUID params=uuid | awk ‘{ print $5 }’)
    [root@xs-1 ~]# echo -e “Management Bond UUID’s:\n BOND INTERFACE UUID: $MGT_BOND_UUID\n BOND MASTER UUID: $MGT_MSTR_UUID\n”
    Management Bond UUID’s:
    BOND INTERFACE UUID: feb191c1-5faa-ca88-3a19-60604dc63ddd
    BOND MASTER UUID: 8f937d86-10a5-3bf5-132a-07f7a6bc9b46

    [/glossary_exclude]

  4. Finish by reviewing the effects of the configuration changes that were just performed:
    1. View the list of newly-created networks that are available in the resource pool:


      [glossary_exclude]

      Legend: GET | SEE | USE

      [root@xs-1 ~]# xe network-list params=uuid,name-label,bridge | grep -v Pool | grep -v management | grep -B1 -A2 name-label
      uuid ( RO) : 929521ce-0cd6-db0a-0501-4a6829192378
      name-label ( RW): NAS
      bridge ( RO): xapi0

      uuid ( RO) : 01ec20b5-1c91-d311-5d28-df7b6745b910
      name-label ( RW): MGT
      bridge ( RO): xapi2

      uuid ( RO) : 01abebfb-9b44-4a99-305e-b0cc94f068df
      name-label ( RW): PUB
      bridge ( RO): xapi1

      [/glossary_exclude]
      Notice that, besides the default networks that were created during installation, only the two networks that were created in Part #3 and Part #4 and the network that was just created in Step #1 are available in the resource pool .

    2. View the list of physical and non-physical interfaces available on the PM from the perspective of XAPI:


      [glossary_exclude]

      Legend: GET | SEE | USE

      [root@xs-1 ~]# xe pif-list host-uuid=$INSTALLATION_UUID physical=true IP-configuration-mode=Static params=uuid,device,IP,network-name-label

      [root@xs-1 ~]# xe pif-list host-uuid=$INSTALLATION_UUID physical=false IP-configuration-mode=Static params=uuid,device,IP,network-name-label
      uuid ( RO) : 501c099a-7215-fc4f-97c6-324b7661f827
      device ( RO): bond0
      network-name-label ( RO): NAS
      IP ( RO): 172.16.0.35

      uuid ( RO) : 8f937d86-10a5-3bf5-132a-07f7a6bc9b46
      device ( RO): bond2
      network-name-label ( RO): MGT
      IP ( RO): 172.16.0.10

      [root@xs-1 ~]# xe pif-list host-uuid=$INSTALLATION_UUID physical=false IP-configuration-mode=None params=uuid,device,IP,network-name-label
      uuid ( RO) : 41e8de7d-b7a5-6f2d-9134-b86009a56ebf
      device ( RO): bond1
      network-name-label ( RO): PUB
      IP ( RO):

      [/glossary_exclude]
      Notice that, from the perspective of XAPI, the IP address of the Primary Management Interace (172.16.0.10) appears to have been moved from the physical interface eth0 to the new, non-physical interface, bond2. In Step #4d, below, we’ll see that the IP address of the PMI has actually been assigned to the newly-created virtual switch xapi2.

    3. View the list of bonded interfaces available on the PM:


      [glossary_exclude]

      Legend: GET | SEE | USE

      [root@xs-1 ~]# xe bond-list params=uuid,master,mode
      uuid ( RO) : c7b1a5ec-34eb-7fd6-bfe2-a8ede5129a4c
      master ( RO): 41e8de7d-b7a5-6f2d-9134-b86009a56ebf
      mode ( RO): balance-slb

      uuid ( RO) : feb191c1-5faa-ca88-3a19-60604dc63ddd
      master ( RO): 8f937d86-10a5-3bf5-132a-07f7a6bc9b46
      mode ( RO): active-backup

      uuid ( RO) : b4da6a53-c233-3730-c98f-b90835eaf9ca
      master ( RO): effda92b-4a34-ad8b-582e-3c1de7cfd864
      mode ( RO): active-backup

      uuid ( RO) : 2c22c629-b1de-7921-dae3-6c36141f1bf3
      master ( RO): 501c099a-7215-fc4f-97c6-324b7661f827
      mode ( RO): active-backup

      uuid ( RO) : bb703a1f-eb27-0ac2-75fa-a37383ee6804
      master ( RO): f1a21f27-8191-f91a-51de-224ed52a2687
      mode ( RO): balance-slb

      [/glossary_exclude]
      Notice that…

      the master UUID of the newly-created bond is identical to the UUID of the new, non-physical interface, bond2.

    4. View the list of physical and non-physical interfaces available on the PM from the perspective of the control domain (Dom0):


      [glossary_exclude]

      Legend: GET | SEE | USE

      [root@xs-1 ~]# ifconfig | grep -v ‘127\.0\.0\.1’ | grep -B1 inet
      xapi0 Link encap:Ethernet HWaddr 00:26:55:D1:E8:CC
      inet addr:172.16.0.35 Bcast:172.16.0.47 Mask:255.255.255.240

      xapi2 Link encap:Ethernet HWaddr 00:1F:29:C4:9F:5C
      inet addr:172.16.0.10 Bcast:172.16.0.31 Mask:255.255.255.224

      [root@xs-1 ~]# ifconfig -a | grep -e xen -e xapi -e eth
      eth0 Link encap:Ethernet HWaddr 00:1F:29:C4:9F:5C
      eth1 Link encap:Ethernet HWaddr 00:1F:29:C4:9F:3C
      eth2 Link encap:Ethernet HWaddr 00:26:55:D1:E8:CC
      eth3 Link encap:Ethernet HWaddr 00:26:55:D1:E8:CD
      eth4 Link encap:Ethernet HWaddr 00:1B:78:5A:21:8D
      eth5 Link encap:Ethernet HWaddr 00:1B:78:5A:21:8C
      xapi0 Link encap:Ethernet HWaddr 00:26:55:D1:E8:CC
      xapi1 Link encap:Ethernet HWaddr 00:1F:29:C4:9F:3C
      xapi2 Link encap:Ethernet HWaddr 00:1F:29:C4:9F:5C

      [/glossary_exclude]
      Notice that…

      1. The IP address of the Primary Management Interace (172.16.0.10) has actually been moved from the physical interface eth0 to the newly-created virtual switch xapi2.
      2. From the perspective of the control domain (Dom0) the newly-created virtual switch xapi2 functions as a non-physical interface.
      3. The newly-created virtual switch xapi2 shares the MAC address of the physical interface eth0.
    5. View the list of bonds and virtual switches on the PM from the perspective of the control domain (Dom0):


      [glossary_exclude]

      Legend: GET | SEE | USE

      [root@xs-1 ~]# ovs-vsctl list-br
      xapi0
      xapi1
      xapi2

      [root@xs-1 ~]# ovs-appctl bond/list
      bond type slaves
      bond0 active-backup eth2, eth4
      bond1 balance-slb eth5, eth1
      bond2 active-backup eth0, eth3

      [/glossary_exclude]
      Notice that…

      1. The network bond bond2 has been formed by combinging the physical interfaces eth0 & eth3.
      2. The new network network bond is configured as an active-backup bond (i.e., an active-passive bond).
      3. The virtual network switch xapi2 has replaced the two default virtual switches xenbr0 & xenbr3.

On the other hosts in the resource pool:

On each of the other hosts in the resource pool: Restart the XAPI service to inherit the newly-created bonds and the newly-created networks from the PM…

  1. Begin by reviewing the current state of networking on the other members of the resource pool:
    1. View the list of networks that are available in the resource pool:


      [glossary_exclude]

      Legend: GET | SEE | USE

      [root@xs-2 ~]# xe network-list params=uuid,name-label,bridge | grep -v Pool | grep -v management | grep -B1 -A2 name-label
      uuid ( RO) : 929521ce-0cd6-db0a-0501-4a6829192378
      name-label ( RW): NAS
      bridge ( RO): xapi0

      uuid ( RO) : 01ec20b5-1c91-d311-5d28-df7b6745b910
      name-label ( RW): MGT
      bridge ( RO): xapi2

      uuid ( RO) : 01abebfb-9b44-4a99-305e-b0cc94f068df
      name-label ( RW): PUB
      bridge ( RO): xapi1

      [/glossary_exclude]
      Notice that, the network that was just created on the PM in Step #1 is already visible from the other pool members.

    2. View the list of physical and non-physical interfaces available on this member of the pool from the perspective of XAPI:


      [glossary_exclude]

      Legend: GET | SEE | USE

      [root@xs-2 ~]# xe pif-list host-uuid=$INSTALLATION_UUID physical=true IP-configuration-mode=Static params=uuid,device,IP,network-name-label
      uuid ( RO) : 7e171fca-dbbf-fb2b-dc27-b17dce481c02
      device ( RO): eth0
      network-name-label ( RO): Pool-wide network associated with eth0
      IP ( RO): 172.16.0.12

      [root@xs-2 ~]# xe pif-list host-uuid=$INSTALLATION_UUID physical=false IP-configuration-mode=Static params=uuid,device,IP,network-name-label
      uuid ( RO) : effda92b-4a34-ad8b-582e-3c1de7cfd864
      device ( RO): bond0
      network-name-label ( RO): NAS
      IP ( RO): 172.16.0.36

      [root@xs-2 ~]# xe pif-list host-uuid=$INSTALLATION_UUID physical=false IP-configuration-mode=None params=uuid,device,IP,network-name-label
      uuid ( RO) : f1a21f27-8191-f91a-51de-224ed52a2687
      device ( RO): bond1
      network-name-label ( RO): PUB
      IP ( RO):

      [/glossary_exclude]
      Notice that the IP address of the Primary Management Interace on this member of the resource pool (172.16.0.12) is still assigned to the physical interface eth0.

    3. View the list of bonded interfaces available in the resource pool:


      [glossary_exclude]

      Legend: GET | SEE | USE

      [root@xs-2 ~]# xe bond-list params=uuid,master,mode
      uuid ( RO) : c7b1a5ec-34eb-7fd6-bfe2-a8ede5129a4c
      master ( RO): 41e8de7d-b7a5-6f2d-9134-b86009a56ebf
      mode ( RO): balance-slb

      uuid ( RO) : feb191c1-5faa-ca88-3a19-60604dc63ddd
      master ( RO): 8f937d86-10a5-3bf5-132a-07f7a6bc9b46
      mode ( RO): active-backup

      uuid ( RO) : b4da6a53-c233-3730-c98f-b90835eaf9ca
      master ( RO): effda92b-4a34-ad8b-582e-3c1de7cfd864
      mode ( RO): active-backup

      uuid ( RO) : 2c22c629-b1de-7921-dae3-6c36141f1bf3
      master ( RO): 501c099a-7215-fc4f-97c6-324b7661f827
      mode ( RO): active-backup

      uuid ( RO) : bb703a1f-eb27-0ac2-75fa-a37383ee6804
      master ( RO): f1a21f27-8191-f91a-51de-224ed52a2687
      mode ( RO): balance-slb

      [/glossary_exclude]
      Notice that, even though the bond has not been created on this member of the resource pool yet, the bond that was created on the PM in Step #1 is visible from this member of the resource pool.

    4. View the list of physical and non-physical interfaces available on this member of the resource pool from the perspective of the control domain (Dom0):


      [glossary_exclude]

      Legend: GET | SEE | USE

      [root@xs-2 ~]# ifconfig | grep -v ‘127\.0\.0\.1’ | grep -B1 inet
      xapi0 Link encap:Ethernet HWaddr 00:26:55:D2:EE:50
      inet addr:172.16.0.36 Bcast:172.16.0.47 Mask:255.255.255.240

      xenbr0 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:BA
      inet addr:172.16.0.12 Bcast:172.16.0.31 Mask:255.255.255.224

      [root@xs-2 ~]# ifconfig -a | grep -e xen -e xapi -e eth
      eth0 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:BA
      eth1 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:C2
      eth2 Link encap:Ethernet HWaddr 00:26:55:D2:EE:50
      eth3 Link encap:Ethernet HWaddr 00:26:55:D2:EE:51
      eth4 Link encap:Ethernet HWaddr 00:1B:78:5C:74:35
      eth5 Link encap:Ethernet HWaddr 00:1B:78:5C:74:34
      xapi0 Link encap:Ethernet HWaddr 00:26:55:D2:EE:50
      xapi1 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:C2
      xenbr0 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:BA
      xenbr3 Link encap:Ethernet HWaddr 00:26:55:D2:EE:51

      [/glossary_exclude]
      Notice that…

      1. From the perspective of the control domain (Dom0) the IP address of the Primary Management Interace (172.16.0.12) is still assigned to xenbr0 (i.e., the default network that is connected to the physical interface eth0).
      2. The default network xenbr0 shares the MAC address of the physical interface the physical interface eth0.
    5. View the list of bonds and virtual switches on this member of the resource pool from the perspective of the control domain (Dom0):


      [glossary_exclude]

      Legend: GET | SEE | USE

      [root@xs-2 ~]# ovs-vsctl list-br
      xapi0
      xapi1
      xenbr0
      xenbr3

      [root@xs-2 ~]# ovs-appctl bond/list
      bond type slaves
      bond0 active-backup eth4, eth2
      bond1 balance-slb eth1, eth5

      [/glossary_exclude]
      Notice that…

      1. The virtual network switch xapi2 has not been created on this member of the resource pool yet and has not replaced the two default virtual switches xenbr0 & xenbr3 yet.
      2. The network bond bond2 has not been formed on this member of the resource pool yet.
    6. Restart the XAPI service to inherit the newly-created bonds and the newly-created networks from the PM:


      [glossary_exclude]

      [root@xs-2 ~]# service xapi restart && sleep 15
      Stopping xapi: .. [ OK ]
      Starting xapi: OK [ OK ]

      [/glossary_exclude]

    7. Finish by reviewing the effects of restarting XAPI in order to inherit the configuration changes that have been performed on the PM:
      1. View the list of networks that are available in the resource pool:


        [glossary_exclude]

        Legend: GET | SEE | USE

        [root@xs-2 ~]# xe network-list params=uuid,name-label,bridge | grep -v Pool | grep -v management | grep -B1 -A2 name-label
        uuid ( RO) : 929521ce-0cd6-db0a-0501-4a6829192378
        name-label ( RW): NAS
        bridge ( RO): xapi0

        uuid ( RO) : 01ec20b5-1c91-d311-5d28-df7b6745b910
        name-label ( RW): MGT
        bridge ( RO): xapi2

        uuid ( RO) : 01abebfb-9b44-4a99-305e-b0cc94f068df
        name-label ( RW): PUB
        bridge ( RO): xapi1

        [/glossary_exclude]

      2. View the list of physical and non-physical interfaces available on this member of the resource pool from the perspective of XAPI:


        [glossary_exclude]

        Legend: GET | SEE | USE

        [root@xs-2 ~]# xe pif-list host-uuid=$INSTALLATION_UUID physical=true IP-configuration-mode=Static params=uuid,device,IP,network-name-label

        [root@xs-2 ~]# xe pif-list host-uuid=$INSTALLATION_UUID physical=false IP-configuration-mode=Static params=uuid,device,IP,network-name-label
        uuid ( RO) : ce750d79-4770-b6c4-6b79-4813b2cacc26
        device ( RO): bond2
        network-name-label ( RO): MGT
        IP ( RO): 172.16.0.12

        uuid ( RO) : effda92b-4a34-ad8b-582e-3c1de7cfd864
        device ( RO): bond0
        network-name-label ( RO): NAS
        IP ( RO): 172.16.0.36

        [root@xs-2 ~]# xe pif-list host-uuid=$INSTALLATION_UUID physical=false IP-configuration-mode=None params=uuid,device,IP,network-name-label
        uuid ( RO) : f1a21f27-8191-f91a-51de-224ed52a2687
        device ( RO): bond1
        network-name-label ( RO): PUB
        IP ( RO):

        [/glossary_exclude]
        Notice that the IP address of the Primary Management Interace of this member of the resource pool (172.16.0.12) is still assigned to the physical interface eth2.

      3. View the list of bonded interfaces available in the resource pool:


        [glossary_exclude]

        Legend: GET | SEE | USE

        [root@xs-2 ~]# xe bond-list params=uuid,master,mode
        uuid ( RO) : c7b1a5ec-34eb-7fd6-bfe2-a8ede5129a4c
        master ( RO): 41e8de7d-b7a5-6f2d-9134-b86009a56ebf
        mode ( RO): balance-slb

        uuid ( RO) : feb191c1-5faa-ca88-3a19-60604dc63ddd
        master ( RO): 8f937d86-10a5-3bf5-132a-07f7a6bc9b46
        mode ( RO): active-backup

        uuid ( RO) : ab9c3ae8-775e-743e-847d-c990296513f2
        master ( RO): ce750d79-4770-b6c4-6b79-4813b2cacc26
        mode ( RO): active-backup

        uuid ( RO) : b4da6a53-c233-3730-c98f-b90835eaf9ca
        master ( RO): effda92b-4a34-ad8b-582e-3c1de7cfd864
        mode ( RO): active-backup

        uuid ( RO) : 2c22c629-b1de-7921-dae3-6c36141f1bf3
        master ( RO): 501c099a-7215-fc4f-97c6-324b7661f827
        mode ( RO): active-backup

        uuid ( RO) : bb703a1f-eb27-0ac2-75fa-a37383ee6804
        master ( RO): f1a21f27-8191-f91a-51de-224ed52a2687
        mode ( RO): balance-slb

        [/glossary_exclude]
        Notice that…

        1. a new network bond has been formed on this member of the resource pool.
        2. The new network network bond is configured as an active-backup bond (i.e., an active-passive bond).
        3. The master UUID of the newly-created bond is identical to the UUID of the new, non-physical interface, bond2.
      4. View the list of physical and non-physical interfaces available on this member of the resource pool from the perspective of the control domain (Dom0):


        [glossary_exclude]

        Legend: GET | SEE | USE

        [root@xs-2 ~]# ifconfig | grep -v ‘127\.0\.0\.1’ | grep -B1 inet
        xapi0 Link encap:Ethernet HWaddr 00:26:55:D2:EE:50
        inet addr:172.16.0.36 Bcast:172.16.0.47 Mask:255.255.255.240

        xapi2 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:BA
        inet addr:172.16.0.12 Bcast:172.16.0.31 Mask:255.255.255.224

        [root@xs-2 ~]# ifconfig -a | grep -e xen -e xapi -e eth
        eth0 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:BA
        eth1 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:C2
        eth2 Link encap:Ethernet HWaddr 00:26:55:D2:EE:50
        eth3 Link encap:Ethernet HWaddr 00:26:55:D2:EE:51
        eth4 Link encap:Ethernet HWaddr 00:1B:78:5C:74:35
        eth5 Link encap:Ethernet HWaddr 00:1B:78:5C:74:34
        xapi0 Link encap:Ethernet HWaddr 00:26:55:D2:EE:50
        xapi1 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:C2
        xapi2 Link encap:Ethernet HWaddr 00:1F:29:C4:9D:BA

        [/glossary_exclude]
        Just as we saw in Step #4d: Notice that, from the perspective of the control domain (Dom0), the newly-created virtual switch xapi2 shares the MAC address of the physical interface the physical interface eth0.

      5. View the list of bonds and virtual switches on this member of the resource pool from the perspective of the control domain (Dom0):


        [glossary_exclude]

        Legend: GET | SEE | USE

        [root@xs-2 ~]# ovs-vsctl list-br
        xapi0
        xapi1
        xapi2

        [root@xs-2 ~]# ovs-appctl bond/list
        bond type slaves
        bond0 active-backup eth4, eth2
        bond1 balance-slb eth1, eth5
        bond2 active-backup eth0, eth3

        [/glossary_exclude]
        Notice that…

        1. The virtual network switch xapi2 has been formed on this member of the resource pool and has replaced the two default virtual switches xenbr0 & xenbr3.
        2. The network bond bond2 has also been formed on this member of the resource pool.