Guenadi N Jilevski's Oracle BLOG

Oracle RAC, DG, EBS, DR and HA DBA BLOG

Oracle GNS and converting Grid Infrastructure from static DNS SCAN/VIP to dynamic GNS provided SCAN/VIP

Oracle GNS and converting Grid Infrastructure from static DNS SCAN/VIP to dynamic GNS provided SCAN/VIP

Note: As I did not see official Oracle documentation on the subject conduct thorough testing.

In the article you will have a look at the steps to convert already installed Oracle Grid Infrastructure based on static DNS SCAN and VIPs to dynamic GNS based where SCAN and node VIPs are provided by DHCP. You will refresh your understanding of GNS by briefly reviewing GNS concepts and benefits. You will look at the steps to set up and verify GNS configuration as a prerequisite for GNS implementation in existing Oracle 11.2.0.X GI/RAC installation.

Overview of GNS

Starting with release 11.2 Oracle introduced Grid Naming Service (GNS) functionality aiming at facilitating management of SCAN and nodes VIPs. GNS provides dynamic DHCP allocated addresses for the SCAN and nodes VIPs. For GNS to operate only one static address is required registered in the DNS that is for the GNS VIP. A sub-domain in DNS needs to be configured to perform delegation to the GNS, that is, queries for the defined sub-domain will be forwarded to GNS at the specified GNS VIP. DHCP is used to provide IP addresses. Due to the fact that DHCP does not provide a name to IP address mapping, mDNS is used to provide a DHCP IP address mapping to names. Within the cluster, nodes are using mDNS for name resolution while for servers outside the cluster DNS is used, which in turn forwards request to GNS. GNS acts as a gateway between DNS and mDNS and assists with name resolution of dynamically provided DHCP based IP addresses. The benefits of GNS are apparent in multi-node clusters where SCANs and node VIPs management is simplified and transparent for the administrator. In GNS configuration adding and deleting nodes is even simpler than with static DNS entries for the node VIPs.

If you have implemented RAC with static DNS initially, it makes sense to migrate to GNS if number of nodes increase.

Sample GNS domain delegation definition

There are two ways for sub-domain delegation implementation. For examples click here and here. In the article both methods were tested. The DNS IP is 192.168.2.11 and GNS VIP is 192.168.2.61.

Method A

Specify domain delegation in /etc/named.conf

zone “grid.bg.gj.com” IN {

    type forward;

    forward only;

    forwarders { 192.168.2.61 ;};

};

zone “gj.com.” IN {

type master;

file “gj1.zone”;

 notify no;

};

Specify a static GNS VIP in gj1.zone.

gns.grid.bg.gj.com. IN A 192.168.2.61

Method B

zone “gj.com.” IN {

type master;

file “gj1.zone”;

 notify no;

};

Specify a static GNS VIP and a sub-domain delegation in gj1.zone.

$ORIGIN grid.bg.gj.com.

@     IN NS    gns.grid.bg.gj.com.

;;

gns.grid.bg.gj.com. IN A 192.168.2.61

oel61.grid.bg.gj.com.     IN     A 192.168.2.11

Sample /etc/resolv.conf

[root@oel61 named]# cat /etc/resolv.conf

# Generated by NetworkManager

search gj.com

nameserver 192.168.2.11

[root@oel61 named]#

Sample DHCP configuration

[root@oel61 named]# cat /etc/dhcp/dhcpd.conf

#

# DHCP Server Configuration file.

# see /usr/share/doc/dhcp*/dhcpd.conf.sample

# see ‘man 5 dhcpd.conf’

#

ddns-update-style interim;

ignore client-updates;

subnet 192.168.2.0 netmask 255.255.255.0 {

option routers 192.168.2.1;

option subnet-mask 255.255.255.0;

option domain-name “grid.gj.com”;

option domain-name-servers 192.168.2.11;

option time-offset -18000; # Eastern Standard Time

range 192.168.2.100 192.168.2.130;

default-lease-time 86400;

}

[root@oel61 named]#

GNS prerequisites checks and verification

The cluvfy offers new functionality to verify the prerequisites for GNS setup.

To verify the DHCP setup use the following command.

[root@oel61 bin]# ./cluvfy comp dhcp -clustername oel-cluster

Verifying DHCP Check

Checking if any DHCP server exists on the network…

At least one DHCP server exists on the network and is listening on port 67

Checking if DHCP server has sufficient free IP addresses for all VIPs…

DHCP server was able to provide sufficient number of IP addresses

The DHCP server response time is within acceptable limits

Verification of DHCP Check was successful.

[root@oel61 bin]#

To verify the DNS sub-domain delegation from two separate terminal sessions issue the following commands for client and server and monitor whether a handshake will happen.

[root@oel61 bin]# ./cluvfy comp dns -server -domain grid.bg.gj.com -vipaddress 192.168.2.61/255.255.255.0/eth1 -verbose

Verifying DNS Check

Starting the test DNS server on IP “192.168.2.61/255.255.255.0/eth1″ listening on port 53

Started the IP address “192.168.2.61/255.255.255.0/eth1″ on node “oel61″

Received IP address lookup query for name “oel61.grid.bg.gj.com”

Received IP address lookup query for name “oel61.grid.bg.gj.com”

Successfully stopped the IP address “192.168.2.61″ on node “oel61″

The test DNS server successfully terminated

Verification of DNS Check was successful.

You have new mail in /var/spool/mail/root

[root@oel61 bin]#

[root@oel61 bin]# ./cluvfy comp dns -client -domain grid.bg.gj.com -vip 192.168.2.61 -last

Verifying DNS Check

Checking if the IP address “192.168.2.61″ is reachable

The IP address “192.168.2.61″ is reachable from local node

Successfully connected to test DNS server

Checking if the test DNS server started on address “192.168.2.61″, listening on port 53 can be queried

Check output of command “cluvfy comp dns -server” to see if it received IP address lookup for name “oel61.grid.bg.gj.com”

Successfully connected to the test DNS server started on address “192.168.2.61″, listening on port 53

Checking DNS delegation for the GNS subdomain “grid.bg.gj.com”…

Check output of command “cluvfy comp dns -server” to see if it received IP address lookup for name “oel61.grid.bg.gj.com”

Successfully verified DNS delegation of the GNS subdomain “grid.bg.gj.com”

Verification of DNS Check was successful.

[root@oel61 bin]#

Finally verify the GNS VIP DNS lookup and reverse lookup.

[root@oel61 ~]# nslookup gns.grid.bg.gj.com

Server:        192.168.2.11

Address:    192.168.2.11#53

Name:    gns.grid.bg.gj.com

Address: 192.168.2.61

[root@oel61 ~]# nslookup 192.168.2.61

Server:        192.168.2.11

Address:    192.168.2.11#53

61.2.168.192.in-addr.arpa    name = gns.grid.bg.gj.com.

[root@oel61 ~]#

[root@oel61 ~]# dig @192.168.2.11 gns.grid.bg.gj.com

; <<>> DiG 9.7.3-RedHat-9.7.3-2.el6 <<>> @192.168.2.11 gns.grid.bg.gj.com

; (1 server found)

;; global options: +cmd

;; Got answer:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 40615

;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1

;; QUESTION SECTION:

;gns.grid.bg.gj.com.        IN    A

;; ANSWER SECTION:

gns.grid.bg.gj.com.    86400    IN    A    192.168.2.61

;; AUTHORITY SECTION:

gj.com.            86400    IN    NS    oel61.gj.com.

;; ADDITIONAL SECTION:

oel61.gj.com.        86400    IN    A    192.168.2.11

;; Query time: 1 msec

;; SERVER: 192.168.2.11#53(192.168.2.11)

;; WHEN: Wed Nov 9 18:58:40 2011

;; MSG SIZE rcvd: 88

[root@oel61 ~]#

Adding GNS and reconfiguration for DHCP network

Configuring GNS involves few configuration processes that need to be executed in proper order. The idea is to create and start GNS. The network also needs to be enabled to obtain IPs automatically using DHCP that is, SCAN and VIPs need to be reconfigured to have IPs dynamically obtained from DHCP.

  1. Add and start GNS
  2. Modify network to have DHCP type addresses
  3. Recreate SCAN to be obtained IP from GNS. Modify SCAN listeners.
  4. Recreate nodeapps to have DHCP addresses
  5. Verify GNS and presence of dynamically obtained IP addresses.
  6. Change SCAN entries in tnsnames.ora
  7. Verify GNS lookup
  1. Add and start GNS

    In the first configuration process I will add and start GNS

[root@oel61 bin]# ./srvctl add gns -i 192.168.2.61 -d grid.bg.gj.com

[root@oel61 bin]# ./srvctl status gns

GNS is not running.

GNS is enabled.

[root@oel61 bin]# ./srvctl config gns

GNS is enabled.

[root@oel61 bin]#

[grid@oel61 ~]$ srvctl start gns

[grid@oel61 ~]$ srvctl config gns

GNS is enabled.

[grid@oel61 ~]$

[grid@oel61 ~]$ srvctl status gns

GNS is running on node oel61.

GNS is enabled on node oel61.

[grid@oel61 ~]$

Verification of the configuration process

[grid@oel61 ~]$ cluvfy comp gns -postcrsinst

Verifying GNS integrity

Checking GNS integrity…

The GNS subdomain name “grid.bg.gj.com” is a valid domain name

Checking if the GNS VIP belongs to same subnet as the public network…

Public network subnets “192.168.2.0″ match with the GNS VIP “192.168.2.0″

GNS VIP “gns.grid.bg.gj.com” resolves to a valid IP address

GNS resolved IP addresses are reachable

WARNING:

PRVF-5217 : An error occurred while trying to look up IP address for “oel-cluster.gj.com.grid.bg.gj.com”

WARNING:

PRVF-5217 : An error occurred while trying to look up IP address for “oel61-vip.grid.bg.gj.com”

GNS resource configuration check passed

GNS VIP resource configuration check passed.

GNS integrity check passed

Verification of GNS integrity was successful.

[grid@oel61 ~]$

  1. Modify network to have DHCP type addresses

    [root@oel61 bin]# ./srvctl config network

    Network exists: 1/192.168.2.0/255.255.255.0/eth1, type static

    [root@oel61 bin]# ./srvctl modify network -k 1 -w dhcp

    [root@oel61 bin]# ./srvctl config network

    Network exists: 1/192.168.2.0/255.255.255.0/eth1, type dhcp

    [root@oel61 bin]#

  2. Recreate SCAN to be obtained from GNS. Modify SCAN listeners.

    [grid@oel61 ~]$ srvctl stop scan_listener

    [grid@oel61 ~]$ srvctl stop scan

     [root@oel61 bin]# ./srvctl remove scan -f

    [root@oel61 bin]# ./srvctl add scan -n oel-cluster-scan.grid.bg.gj.com

    [root@oel61 bin]#

    [grid@oel61 ~]$ srvctl config scan

    SCAN name: oel-cluster-scan.grid.bg.gj.com, Network: 1/192.168.2.0/255.255.255.0/eth1

    SCAN VIP name: scan1, IP:

    SCAN VIP name: scan2, IP:

    SCAN VIP name: scan3, IP:

    [grid@oel61 ~]$ srvctl start scan

    [grid@oel61 ~]$ srvctl status scan

    SCAN VIP scan1 is enabled

    SCAN VIP scan1 is running on node oel61

    SCAN VIP scan2 is enabled

    SCAN VIP scan2 is running on node oel61

    SCAN VIP scan3 is enabled

    SCAN VIP scan3 is running on node oel61

    [grid@oel61 ~]$

    [grid@oel61 ~]$ srvctl stop scan_listener

    [grid@oel61 ~]$ srvctl stop scan

    [grid@oel61 ~]$ srvctl modify scan_listener -u

    [grid@oel61 ~]$ srvctl start scan

    [grid@oel61 ~]$ srvctl start scan_listener

    [grid@oel61 ~]$ srvctl config scan

    SCAN name: oel-cluster-scan.grid.bg.gj.com, Network: 1/192.168.2.0/255.255.255.0/eth1

    SCAN VIP name: scan1, IP: /oel-cluster-scan.grid.bg.gj.com/192.168.2.115

    SCAN VIP name: scan2, IP: /oel-cluster-scan.grid.bg.gj.com/192.168.2.104

    SCAN VIP name: scan3, IP: /oel-cluster-scan.grid.bg.gj.com/192.168.2.103

    [grid@oel61 ~]$srvctl config scan_listener

    SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521

    SCAN Listener LISTENER_SCAN2 exists. Port: TCP:1521

    SCAN Listener LISTENER_SCAN3 exists. Port: TCP:1521

    [grid@oel61 ~]$ srvctl status scan_listener

    SCAN Listener LISTENER_SCAN1 is enabled

    SCAN listener LISTENER_SCAN1 is running on node oel61

    SCAN Listener LISTENER_SCAN2 is enabled

    SCAN listener LISTENER_SCAN2 is running on node oel61

    SCAN Listener LISTENER_SCAN3 is enabled

    SCAN listener LISTENER_SCAN3 is running on node oel61

    [grid@oel61 ~]$ srvctl status scan

    SCAN VIP scan1 is enabled

    SCAN VIP scan1 is running on node oel61

    SCAN VIP scan2 is enabled

    SCAN VIP scan2 is running on node oel61

    SCAN VIP scan3 is enabled

    SCAN VIP scan3 is running on node oel61

    [grid@oel61 ~]$

    [grid@oel61 ~]$

    Verify the outcome of the configuration process

    [grid@oel61 ~]$ cluvfy comp gns -postcrsinst

    Verifying GNS integrity

    Checking GNS integrity…

    The GNS subdomain name “grid.bg.gj.com” is a valid domain name

    Checking if the GNS VIP belongs to same subnet as the public network…

    Public network subnets “192.168.2.0″ match with the GNS VIP “192.168.2.0″

    GNS VIP “gns.grid.bg.gj.com” resolves to a valid IP address

    GNS resolved IP addresses are reachable

    GNS resolved IP addresses are reachable

    WARNING:

    PRVF-5217 : An error occurred while trying to look up IP address for “oel61-vip.grid.bg.gj.com”

    GNS resource configuration check passed

    GNS VIP resource configuration check passed.

    GNS integrity check passed

    Verification of GNS integrity was successful.

    [grid@oel61 ~]$

  3. Recreate nodeapps to have DHCP addresses

    [grid@oel61 ~]$ srvctl stop database -d rmanrep

    [grid@oel61 ~]$

    [grid@oel61 ~]$ srvctl stop listener

    [grid@oel61 ~]$

    [grid@oel61 ~]$ srvctl stop vip -n oel61

    [grid@oel61 ~]$

    [grid@oel61 ~]$ srvctl stop nodeapps -f

    PRKO-2425 : VIP is already stopped on node(s): oel61

    [grid@oel61 ~]$

    [root@oel61 bin]# ./srvctl remove nodeapps -f

    [root@oel61 bin]#

    [root@oel61 bin]# ./srvctl add nodeapps -S 192.168.2.0/255.255.255.0/eth1 -l 6100 -r 6200

    [root@oel61 bin]#

    [grid@oel61 ~]$ srvctl config nodeapps

    Network exists: 1/192.168.2.0/255.255.255.0/eth1, type dhcp

    GSD exists

    ONS exists: Local port 6100, remote port 6200, EM port 2016

    [grid@oel61 ~]$ srvctl start nodeapps

    [grid@oel61 ~]$

    [grid@oel61 ~]$ srvctl status nodeapps

    VIP 192.168.2.123 is enabled

    VIP 192.168.2.123 is running on node: oel61

    Network is enabled

    Network is running on node: oel61

    GSD is disabled

    GSD is not running on node: oel61

    ONS is enabled

    ONS daemon is running on node: oel61

    [grid@oel61 ~]$

    [grid@oel61 ~]$ srvctl start listener

    [grid@oel61 ~]$

    [grid@oel61 ~]$ srvctl start scan

    [grid@oel61 ~]$ srvctl start scan_listener

    [grid@oel61 ~]$

    Verify the outcome of the configuration process

    [grid@oel61 ~]$ cluvfy comp gns -postcrsinst

    Verifying GNS integrity

    Checking GNS integrity…

    The GNS subdomain name “grid.bg.gj.com” is a valid domain name

    Checking if the GNS VIP belongs to same subnet as the public network…

    Public network subnets “192.168.2.0″ match with the GNS VIP “192.168.2.0″

    GNS VIP “gns.grid.bg.gj.com” resolves to a valid IP address

    GNS resolved IP addresses are reachable

    GNS resolved IP addresses are reachable

    GNS resolved IP addresses are reachable

    GNS resource configuration check passed

    GNS VIP resource configuration check passed.

    GNS integrity check passed

    Verification of GNS integrity was successful.

    [grid@oel61 ~]$

  4. Verify GNS lookup and presence of dynamically obtained VIP addresses

    [grid@oel61 ~]$ srvctl config scan

    SCAN name: oel-cluster-scan.grid.bg.gj.com, Network: 1/192.168.2.0/255.255.255.0/eth1

    SCAN VIP name: scan1, IP: /oel-cluster-scan.grid.bg.gj.com/192.168.2.115

    SCAN VIP name: scan2, IP: /oel-cluster-scan.grid.bg.gj.com/192.168.2.104

    SCAN VIP name: scan3, IP: /oel-cluster-scan.grid.bg.gj.com/192.168.2.103

    [grid@oel61 ~]$

    [grid@oel61 ~]$ srvctl config vip -n oel61

    VIP exists: /192.168.2.123/192.168.2.123/192.168.2.0/255.255.255.0/eth1, hosting node oel61

    [grid@oel61 ~]$

  5. Change SCAN entries in tnsnames.ora to reflect the new GNS domain. See the old and new tnsnames.ora content.

    [oracle@oel61 admin]$ cat tnsnames.ora.old

    # tnsnames.ora Network Configuration File: /u01/app/oracle/product/11.2.0/db_1/network/admin/tnsnames.ora

    # Generated by Oracle configuration tools.

    RMANREP =

    (DESCRIPTION =

    (LOAD_BALANCE = ON)

    (FAILOVER = ON)

     (ADDRESS = (PROTOCOL = TCP)(HOST = oel-cluster)(PORT = 1521))

    (CONNECT_DATA =

    (SERVER = DEDICATED)

     (SERVICE_NAME = RMANREP)

    )

    )

    [oracle@oel61 admin]$ cat tnsnames.ora

    # tnsnames.ora Network Configuration File: /u01/app/oracle/product/11.2.0/db_1/network/admin/tnsnames.ora

    # Generated by Oracle configuration tools.

    RMANREP =

    (DESCRIPTION =

    (LOAD_BALANCE = ON)

    (FAILOVER = ON)

    (ADDRESS = (PROTOCOL = TCP)(HOST = oel-cluster-scan.grid.bg.gj.com)(PORT = 1521))

    (CONNECT_DATA =

    (SERVER = DEDICATED)

    (SERVICE_NAME = RMANREP)

    )

    )

    [oracle@oel61 admin]$

    [oracle@oel61 admin]$ sqlplus system/sys1@rmanrep

    SQL*Plus: Release 11.2.0.3.0 Production on Fri Nov 11 18:07:01 2011

    Copyright (c) 1982, 2011, Oracle. All rights reserved.

    Connected to:

    Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 – 64bit Production

    With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,

    Data Mining and Real Application Testing options

    SQL> select * from gv$configured_interconnects;

    INST_ID NAME      IP_ADDRESS     IS_ SOURCE

    ———- ————— —————- — ——————————-

         1 eth0:2     169.254.17.103 NO

         1 eth3:1     169.254.114.63 NO

         1 eth2:1     169.254.165.47 NO

         1 eth0:1     169.254.236.186 NO

         1 eth1      192.168.2.11 YES

         1 eth1:1     192.168.2.61 YES

         1 eth1:2     192.168.2.123 YES

         1 eth1:3     192.168.2.115 YES

         1 eth1:4     192.168.2.104 YES

         1 eth1:5     192.168.2.103 YES

    10 rows selected.

    SQL>

  6. Verify GNS lookup and sub-domain forwarding

[root@oel61 named]# nslookup oel61-vip.grid.bg.gj.com

Server:        192.168.2.11

Address:    192.168.2.11#53

Name:    oel61-vip.grid.bg.gj.com

Address: 192.168.2.123

[root@oel61 named]# nslookup oel-cluster-scan.grid.bg.gj.com

Server:        192.168.2.11

Address:    192.168.2.11#53

Name:    oel-cluster-scan.grid.bg.gj.com

Address: 192.168.2.103

Name:    oel-cluster-scan.grid.bg.gj.com

Address: 192.168.2.104

Name:    oel-cluster-scan.grid.bg.gj.com

Address: 192.168.2.115

[root@oel61 named]#


November 11, 2011 - Posted by | oracle

2 Comments »

  1. I have been surfing on-line greater than three hours nowadays, but I never discovered any interesting article like yours. It is beautiful worth enough for me. In my opinion, if all webmasters and bloggers made good content material as you probably did, the net will likely be a lot more helpful than ever before.

    Comment by Buy Office 2007 | November 17, 2011 | Reply

  2. You made some first rate points there. I appeared on the web for the difficulty and found most people will go together with together with your website.

    Comment by Carter Montiel | November 17, 2011 | Reply


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 614 other followers

%d bloggers like this: