Table of Contents
Key Facts
- Older Cluvfy doesn’t detect multicast issues – it only test broadcast issues
- Newer Cluvfy ( > 11.2.0.3 ) tests only MULITCAST address 230.0.1.0 : $ cluvfy stage -post hwos -n grac42
- ./mcasttest.pl tests Multicast addresses 230.0.1.0 and 224.0.0.251
- Clusterware 11.2.0.3 was enhanced to utilize broadcast or multicast to bootstrap
- Clusterware on 2.nd node may not start after upgrade to 11.2.0.3.5 – 11.2.0.3.7 as bootstrap code was not working ( if Mutlicast option is not available )
Overview
Multicast can be vaildated using the mcasttest.pl tool attached to this note to validate whether or not multicast on the 230.0.1.0 or 224.0.0.251 address can be used on the private interconnect interface(s). CLUFFY in 11.2.0.3 and above does contain similar checks but will report pass if the broadcast check passes, with mentioned issue which breaks broadcast in 11.2.0.3.5 - 11.2.0.3.7 GI PSU and 12.1.0.1 this pass is a false positive. For this reason it is recommended to use mcasttest.pl program to validate multicast functionality before patching/upgrading/installing the mentioned releases. Note: Multicast based communication only needs to be successful on either the 230.0.1.0 address or the 224.0.0.251 address. A successful multicast communication on both addresses is not required.
OS tools to check whether multicast is in use
Run netstat -g to get the multicast addresses this host subscribes to. [grid@grac42 grac42]$ netstat -g IPv6/IPv4 Group Memberships Interface RefCnt Group --------------- ------ --------------------- lo 1 224.0.0.1 eth0 1 224.0.0.251 eth0 1 224.0.0.1 eth1 1 224.0.0.251 eth1 1 224.0.0.1 eth2 3 230.0.1.0 eth2 1 224.0.0.1 eth2 4 224.0.0.251 lo 1 ff02::1 eth0 1 ff02::202 eth0 1 ff02::1:ff6c:8927 eth0 1 ff02::1 eth1 1 ff02::1:ff63:807 eth1 1 ff02::202 eth1 1 ff02::1 eth2 1 ff02::1:ffdf:79b9 eth2 1 ff02::202 Check that network devices used by cluster are enabled for MULTICAST [root@grac42 Desktop]# ifconfig eth2 eth2 Link encap:Ethernet HWaddr 08:00:27:DF:79:B9 inet addr:192.168.2.102 Bcast:192.168.2.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fedf:79b9/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1567103 errors:0 dropped:0 overruns:0 frame:0 TX packets:1420244 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:737015972 (702.8 MiB) TX bytes:571312727 (544.8 MiB)
Testing MULTICAST feature using mcasttest.pl
For downloading mcasttest.pl utility please read node 1212703.1. Running mcasttest.pl : [root@grac42 mcasttest]# ./mcasttest.pl -n grac41,grac42,grac43 -i eth2 ########### Setup for node grac41 ########## Checking node access 'grac41' Checking node login 'grac41' Checking/Creating Directory /tmp/mcasttest for binary on node 'grac41' Distributing mcast2 binary to node 'grac41' ########### Setup for node grac42 ########## Checking node access 'grac42' Checking node login 'grac42' Checking/Creating Directory /tmp/mcasttest for binary on node 'grac42' Distributing mcast2 binary to node 'grac42' ########### Setup for node grac43 ########## Checking node access 'grac43' Checking node login 'grac43' Checking/Creating Directory /tmp/mcasttest for binary on node 'grac43' Distributing mcast2 binary to node 'grac43' ########### testing Multicast on all nodes ########## Test for Multicast address 230.0.1.0 Jun 1 13:27:50 | Multicast Succeeded for eth2 using address 230.0.1.0:42000 Test for Multicast address 224.0.0.251 Jun 1 13:27:51 | Multicast Succeeded for eth2 using address 224.0.0.251:42001
Testing Multicast feature with cluvfy
Cluvfy option hwos will test multicast communication for multicast group "230.0.1.0" $ cluvfy stage -post hwos -n grac42 ... Checking multicast communication... Checking subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0"... Check of subnet "192.168.1.0" for multicast communication with multicast group "230.0.1.0" passed. Checking subnet "192.168.2.0" for multicast communication with multicast group "230.0.1.0"... Check of subnet "192.168.2.0" for multicast communication with multicast group "230.0.1.0" passed. Check of multicast communication passed. ..
Disabling Multicasting may not work with Virtualbox
Note I can't disable Multicasting in my environment ( Virtualbox 4.1 , OEL 6.2 , RAC 11.2.0.4.2 , internal network ): # ifconfig eth2 -multicast # ifconfig eth2 eth2 Link encap:Ethernet HWaddr 08:00:27:DF:79:B9 inet addr:192.168.2.102 Bcast:192.168.2.255 Mask:255.255.255.0 inet6 addr: fe80::a00:27ff:fedf:79b9/64 Scope:Link UP BROADCAST RUNNING MTU:1500 Metric:1 RX packets:1590101 errors:0 dropped:0 overruns:0 frame:0 TX packets:1444893 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:745346779 (710.8 MiB) TX bytes:581746160 (554.7 MiB) Running ./mcasttest.pl - expect that script fails # ./mcasttest.pl -n grac41,grac42,grac43 -i eth2 Capture traffic for MULTICAST IP address 224.0.0.251 ( start before running ./mcasttest.pl ) [root@grac42 mcasttest]# tcpdump -i eth2 host 224.0.0.251 -n tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes 09:47:09.324625 IP 192.168.2.102.42001 > 224.0.0.251.42001: UDP, length 128 09:47:09.544860 IP 192.168.2.103.42001 > 224.0.0.251.42001: UDP, length 128 09:47:09.642670 IP 192.168.2.101.42001 > 224.0.0.251.42001: UDP, length 128 09:47:09.824895 IP 192.168.2.102.42001 > 224.0.0.251.42001: UDP, length 128 09:47:10.044734 IP 192.168.2.103.42001 > 224.0.0.251.42001: UDP, length 128 --> Sending MULITCAST packets still works even MULTICAST feature is disabled via ifconfig Maybe need to test with iptables to suppress MULITCAST packets to be send
Reference
- Grid Infrastructure Startup During Patching, Install or Upgrade May Fail Due to Multicasting Requirement (Doc ID 1212703.1)
- Bug 9974223 : GRID INFRASTRUCTURE NEEDS MULTICAST COMMUNICATION ON 230.0.1.0 ADDRESSES WORKING
- Bug 16547309 : GIPC SHOWS RANK 0 OR -1 AFTER APPLIED PSU 11.2.0.3.5