Oracle Real Application Clusters (RAC)

Dear All,

Some FAQ about RAC which I came across very recently.

What is RAC and how is it different from non RAC databases?

RAC stands for Real Application Clusters. It allows multiple nodes in a clustered system to mount and open a single database that resides on shared disk storage. Should a single system fail (node), the database service will still be available on the remaining nodes.

A non-RAC database is only available on a single system. If that system fails, the database service will be down (single point of failure).


Can any application be deployed on RAC?

Most applications can be deployed on RAC without any modifications and still scale linearly (well, almost).

Applications with 'hot' blocks (the same data blocks continuously accessed by processes on different nodes) may not work well. This is because data blocks will constantly be moved from one Oracle Instance to another. In such cases the application may need to be partitioned based on function or data to eliminate the contention.


Do you need special hardware to run RAC?

RAC requires the following hardware components:

A dedicated network interconnect - might be as simple as a fast network connection between nodes; and
A shared disk subsystem.
Example systems that can be used with RAC:

Windows Clusters
Linux Clusters
Unix Clusters like SUN PDB (Parallel DB).
IBM z/OS in SYSPLEX
HP Serviceguard extension for RAC


How many OCR and voting disks should one have?

For redundancy, one should have at lease two OCR disks and three voting disks (raw disk partitions). These disk partitions should be spread across different physical disks.

How does one convert a single instance database to RAC?

Oracle 10gR2 introduces a utility called rconfig (located in $ORACLE_HOME/bin) that will convert a single instance database to a RAC database.

$ cp $ORACLE_HOME/assistants/rconfig/sampleXMLs/ConvertToRAC.xml racconv.xml
$ vi racconv.xml
$ rconfig racconv.xml
One can also use dbca and enterprise manager to convert the database to RAC mode.

For prior releases, follow these steps:

Shut Down your Database:
SQL> CONNECT SYS AS SYSDBA
SQL> SHUTDOWN NORMAL
Enable RAC - On Unix this is done by relinking the Oracle software.
Make the software available on all computer systems that will run RAC. This can be done by copying the software to all systems or to a shared clustered file system.
Each instance requires its own set of Redo Log Files (called a thread). Create additional log files:
SQL> CONNECT SYS AS SYSBDA
SQL> STARTUP EXCLUSIVE

SQL> ALTER DATABASE ADD LOGFILE THREAD 2
SQL> GROUP G4 ('RAW_FILE1') SIZE 500k,
SQL> GROUP G5 ('RAW_FILE2') SIZE 500k,
SQL> GROUP G6 ('RAW_FILE3') SIZE 500k;

SQL> ALTER DATABASE ENABLE PUBLIC THREAD 2;
Each instance requires its own set of Undo segments (rollback segments). To add undo segments for New Nodes:
UNDO_MANAGEMENT = auto
UNDO_TABLESPACE = undots2
Edit the SPFILE/INIT.ORA files and number the instances 1, 2,...:
CLUSTER_DATABASE = TRUE (PARALLEL_SERVER = TRUE prior to Oracle9i).
INSTANCE_NUMBER = 1
THREAD = 1
UNDO_TABLESPACE = undots1 (or ROLLBACK_SEGMENTS if you use UNDO_MANAGEMENT=manual)
# Include %T for the thread in the LOG_ARCHIVE_FORMAT string.
# Set LM_PROCS to the number of nodes * PROCESSES
# etc....
Create the dictionary views needed for RAC by running catclust.sql (previously called catparr.sql):
SQL> START ?/rdbms/admin/catclust.sql
On all the computer systems, startup the instances:
SQL> CONNECT / as SYSDBA
SQL> STARTUP;


How does one stop and start RAC instances?

There are no difference between the way you start a normal database and RAC database, except that a RAC database needs to be started from multiple nodes. The CLUSTER_DATABASE=TRUE (PARALLEL_SERVER=TRUE) parameter needs to be set before a database can be started in cluster mode.

In Oracle 10g one can use the srvctl utility to start instances and listener across the cluster from a single node. Here are some examples:

$ srvctl status database -d RACDB
$ srvctl start database -d RACDB
$ srvctl start instance -d RACDB -i RACDB1
$ srvctl start instance -d RACDB -i RACDB2
$ srvctl stop database -d RACDB
$ srvctl start asm -n node2
Before Oracle 8.0, use the following command sequence from each node (using the old server manager):

SVRMGR> connect INTERNAL
SVRMGR> set retries 5
SVRMGR> startup parallel retry .. or SVRMGR> startup shared
You can also use the SET INSTANCE instanceN command to switch between instances (if defined in TNSNAMES.ORA).


Can I test if a database is running in RAC mode?

Use the DBMS_UTILITY package to determine if a database is running in RAC mode or not. Example:

BEGIN
IF dbms_utility.is_cluster_database THEN
dbms_output.put_line('Running in SHARED/RAC mode.');
ELSE
dbms_output.put_line('Running in EXCLUSIVE mode.');
END IF;
END;
/

For Oracle 8i and prior releases:

BEGIN
IF dbms_utility.is_parallel_server THEN
dbms_output.put_line('Running in SHARED/PARALLEL mode.');
ELSE
dbms_output.put_line('Running in EXCLUSIVE mode.');
END IF;
END;
/

Another method is to look at the database parameters. For example, from SQL*Plus:

SQL> show parameter CLUSTER_DATABASE
If the value of CLUSTER_DATABASE is FALSE then database is not running in RAC Mode.


How can I keep track of active instances?

You can keep track of active RAC instances by executing one of the following queries:

SELECT * FROM SYS.V_$ACTIVE_INSTANCES;
SELECT * FROM SYS.V_$THREAD;
To list the active instances from PL/SQL, use DBMS_UTILITY.ACTIVE_INSTANCES().


Can one see how connections are distributed across the nodes?

Select from gv$session. Some examples:

SELECT inst_id, count(*) "DB Sessions" FROM gv$session
WHERE type = 'USER' GROUP BY inst_id;
With login time (hour):

SELECT inst_id, TO_CHAR(logon_time, 'DD-MON-YYYY HH24') "Hour when connected", count(*) "DB Sessions"
FROM gv$session
WHERE type = 'USER'
GROUP BY inst_id, TO_CHAR(logon_time, 'DD-MON-YYYY HH24')
ORDER BY inst_id, TO_CHAR(logon_time, 'DD-MON-YYYY HH24');


What is pinging and why is it so bad?

Starting with Oracle 9i, RAC can transfer blocks from one instance to another across the interconnect (cache fusion). This method is much faster than the old "pinging" method, where one instance had to write the block to disk before another instance could read it.

Oracle 8i and below:

Pinging is the process whereby one instance requests another to write a set of blocks from its SGA to disk so it can obtain it in exclusive mode. This method of moving data blocks from one instance's SGA to another is extremely slow. The challenge of tuning RAC/OPS is to minimize pinging activity.


Thanks & Regards,
S.Grace Paul Regan