Configure Network Devices with Virtualbox after cloning/copying for RAC setup

Typical RAC network device configuration used for RAC setup

eth0  - Virtualbox NAT  - Device used by connecting to VPN network for running yum and other stuff 
        CHECK    automatically obtain IP address with DHCP
        UNCHECK  Automatically obtain DNS information from provider
eth1  - Public cluster interface for VIPS,....
eht2  - Private cluster Interconnect

Methode I –  Fixing the network devices after cloning by using Network manager

  • Disable and delete all bak network devices
  • Edit scritpts located in  /etc/sysconfig/network-scripts to configure devices ( see sample scripts below )
  • Restart network with : # service network restart
  • Sample scripts for a typical RAC setup
Configuration for Network devices 
eth0  - Virtualbox NAT  - Device used by connecting to VPN network for running yum and other stuff 
        CHECK    automatically obtain IP address with DHCP
        UNCHECK  Automatically obtain DNS information from provider

Network script location:
/etc/sysconfig/network-scripts

ifcfg-eth0
DEVICE=eth0
BOOTPROTO=dhcp
HWADDR=08:00:27:F8:BA:74
ONBOOT=yes
DHCP_HOSTNAME=ract1.example.com
TYPE=Ethernet
DNS1=192.168.1.50
DNS2=192.135.82.44
DNS3=192.168.1.1
DOMAIN="example.com de.oracle.com"

eth1 : IP  192.168.1.131 : Virtualbox Internal Network : Public RAC Interface  
ifcfg-eth1  
# Intel Corporation 82540EM Gigabit Ethernet Controller
DEVICE=eth1
BOOTPROTO=none
BROADCAST=192.168.1.255
HWADDR=08:00:27:B5:5A:54
IPADDR=192.168.1.131
NETMASK=255.255.255.0
NETWORK=192.168.1.0
ONBOOT=yes
TYPE=Ethernet

eth2 :  IP  192.168.2.131 : Virtualbox Internal Network:  Private Cluster Interconnect
ifcfg-eth2 
DEVICE=eth2
BOOTPROTO=none
BROADCAST=192.168.2.255
HWADDR=08:00:27:26:67:20
IPADDR=192.168.2.131
NETMASK=255.255.255.0
NETWORK=192.168.2.0
ONBOOT=yes
TYPE=Ethernet

Methode II – delete  /etc/udev/rules.d/70-persistent-net.rules

Current Network device naming  by using VirtualBox Bridged Network Device

# ifconfig
eth4      Link encap:Ethernet  HWaddr 08:00:27:D5:CC:47  

# dmesg | grep eth
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
udev: renamed network interface eth0 to eth4
ADDRCONF(NETDEV_UP): eth4: link is not ready
e1000: eth4: e1000_watchdog_task: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
ADDRCONF(NETDEV_CHANGE): eth4: link becomes ready

# cat /etc/udev/rules.d/70-persistent-net.rules

# PCI device 0x8086:0x100e (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:ec:21:88", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
...
# PCI device 0x8086:0x100e (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:d5:cc:47", ATTR{type}=="1", KERNEL=="eth*", NAME="eth4"

Solution

# rm  /etc/udev/rules.d/70-persistent-net.rules
# reboot 
# ifconfig
eth0      Link encap:Ethernet  HWaddr 08:00:27:D5:CC:47 

 

Check that content of /etc/sysconfig/network-scripts/ifcfg-eth0 is correct:

Change HWADDR=08:00:27:EC:21:88 and  IPADDR=10.165.112.150 if needed
# cat ifcfg-eth0
HWADDR=08:00:27:D5:CC:47
TYPE=Ethernet
BOOTPROTO=none
IPADDR=10.165.112.150
PREFIX=23
GATEWAY=10.165.112.1
DNS1=10.165.246.33
DNS2=192.135.82.60
DOMAIN=de.oracle.com
DEFROUTE=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no

Verify that  scripts  located in /etc/sysconfig/network-scripts are working

# pwd
/etc/sysconfig/network-scripts
# ifdown eth0
Device state: 3 (disconnected)
# ifup eth0
Active connection state: activating
Active connection path: /org/freedesktop/NetworkManager/ActiveConnection/2
state: activating
state: activated
Connection activated

Vbox Network adapter fails with:  Device not managed by NetworkManager or unavailable

Runnung ifup fails with : 
$ ifup eth0 
Error: Connection activation failed: Device not managed by NetworkManager or unavailable

Device status 
[root@gract3 Desktop]# ifconfig
eth0      Link encap:Ethernet  HWaddr 08:00:27:A9:76:E3  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
--> Device is up but IP address was missing
    Workaround by adding NM_CONTROLLED=yes to ifcfg-eth0 doesn't solve the problem

Checking device status more in detail 
[root@gract3 ~]# dmesg | grep eth | grep becomes
ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready
ADDRCONF(NETDEV_CHANGE): eth3: link becomes ready
--> eth0 never reached ready state

Checking Ethernet addresses
[root@gract3 network-scripts]#  dmesg | grep probe
Fixed MDIO Bus: probed
e1000: 0000:00:03.0: e1000_probe: (PCI:33MHz:32-bit) 08:00:27:a9:76:e3
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
e1000: 0000:00:08.0: e1000_probe: (PCI:33MHz:32-bit) 08:00:27:2f:73:1f
e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection
e1000: 0000:00:09.0: e1000_probe: (PCI:33MHz:32-bit) 08:00:27:d9:bc:7e
e1000: eth2: e1000_probe: Intel(R) PRO/1000 Network Connection
e1000: 0000:00:0a.0: e1000_probe: (PCI:33MHz:32-bit) 08:00:27:cf:c0:9b
e1000: eth3: e1000_probe: Intel(R) PRO/1000 Network Connection
--> Looks fine with configured address in  ifcfg-eth0: HWADDR=08:00:27:A9:76:E3

Testing 
[root@gract3 Desktop]#  ifup eth0 up
Error: Connection activation failed: Device not managed by NetworkManager or unavailable

Trying to restart Network and Network Manager 
[root@gract3 Desktop]# service network restart
Shutting down interface eth0:                              [  OK  ]
Bringing up interface eth0:  
Determining IP information for eth0... failed; no link present.  Check cable?
                                                           [FAILED]
Restart  NetworkManager  
$ service NetworkManager stop 
$ service NetworkManager start 

VBOX: Reconfigure device eth0 to using  Internal Network  
[root@gract3 network-scripts]# ifup eth0
Active connection state: activating
Active connection path: /org/freedesktop/NetworkManager/ActiveConnection/5
state: activated
Connection activated
[root@gract3 network-scripts]# ifconfig -a eth0
eth0      Link encap:Ethernet  HWaddr 08:00:27:A9:76:E3  
          inet addr:192.168.1.178  Bcast:192.168.1.255  Mask:255.255.255.0
--> Device is up 

Now changing back to NAT 
[root@gract3 network-scripts]#  ifup eth0
Active connection state: activating
Active connection path: /org/freedesktop/NetworkManager/ActiveConnection/7
state: activated
Connection activated
[root@gract3 network-scripts]# ifconfig -a eth0
eth0      Link encap:Ethernet  HWaddr 08:00:27:A9:76:E3  
          inet addr:10.0.2.15  Bcast:10.0.2.255  Mask:255.255.255.0
--> Now device eth0 is up using NAT with a valid IP addresss 

Reference 
What is the difference between 'ifconfig up eth0' and 'ifup eth0'?

Change hostname in the “/etc/sysconfig/network” file for cloend VMs:

NETWORKING=yes
HOSTNAME=hhrac2.de.oracle.com

Problem /etc/resolv.conf gets overwritten on reboot or network restart

  • Choose “Automatic (DHCP) addresses only” method, so that your DHCP server cannot override your DNS setting. See following link for details.
  • Also double check /etc/sysconfig/network-scripts/ifcfg-eth0  and related network config files no having  out-dated  DOMAIN entries  like de.oracle.com in case your current domain  should be example.com. Always take care that your scripts ifcfg-eth0, ifcfg-eth1 and ifcfg-eth2 are up to date and in sync.

 


Reference

 

 

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *