Oracle RAC 08 - Database software, listener and disk groups

Oracle RAC Solution · Previous: Grid patches and the multicast problem · Next: Database creation and user environments

With the cluster running on both nodes, three things still stand between it and a database: the database software itself, a listener for the database, and somewhere to put the files. Each has its own graphical tool, each runs once on oradb01 and acts on both nodes, and each runs as a particular user. This part walks through the three tools in the order they were used. None of them creates a database; that is the next part.

Three tools, two owners

The installation keeps the two Oracle products apart. Grid Infrastructure (Clusterware and ASM) belongs to the user grid, the database software to the user oracle, and each has its own home under /data/u01/app. Which tool runs as whom follows from that.

All three tools are X applications. The notes repeat the same remark before each of them: an X server has to be running on the workstation the installation is driven from. How the display got there (SSH forwarding or DISPLAY) is not written down.

The database software

The media were unpacked in /data/install/database, beside the Grid Infrastructure media. The installer was started as oracle; the notes switch to the user from a root shell and do not name the node. NETCA, ASMCA and DBCA are all started on oradb01 in the notes, so by inference the installer ran there too.

$ su - oracle
$ cd /data/install/database/
$ ./runInstaller

The answers that shape the result are few.

The first screens are about My Oracle Support. I gave a contact address and did not ask for security updates. The installer then could not reach My Oracle Support and showed a "Connection Failed" dialogue (the notes record "Connection Method: Proxy" on it), where the way forward was to tick "I want to remain uninformed of critical security issues in my configuration". Software updates were skipped.

The SSH screen has a "Setup" and a "Test" button. The keys of oracle already existed on both nodes from the preparation, so "Reuse private and public keys existing in the user home" was ticked and the installer only exchanged them. The password it asks for is the operating system password of oracle.

The whole answer set, screen by screen, is a Config document: Database software installer answers. The notes do not hold the prerequisite check screen of this installer; the checks and what failed in them are recorded only for Grid Infrastructure, in Grid Infrastructure installation.

Towards the end the installer stops and asks for a script to be run as root on both nodes.

$ /data/u01/app/oracle/db11204/root.sh

After the Grid Infrastructure root.sh, which configures a cluster and failed more than once (see Grid patches and the multicast problem), this one is an anticlimax. It asks for the local bin directory, finds that dbhome, oraenv and coraenv are already in /usr/local/bin from the Grid installation, and ends with the remark that entries will be added to /etc/oratab when a database is created.

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Finished product-specific root actions.

One more listener

The Grid Infrastructure installation already left listeners behind. Its status check shows a node listener resource, ora.LISTENER.lsnr, online on both nodes, and three SCAN listeners, LISTENER_SCAN1 to LISTENER_SCAN3, one for each address of oradb-scan. The SCAN port given to the Grid installer was 1521.

For the database I added a listener of its own with the Oracle Net Configuration Assistant, started as oracle on the first node.

$ su - oracle
$ netca

"Cluster configuration" is the choice that makes the listener a matter of the cluster and not of one node. The answer set is a Config document: NETCA answers.

Two things about this listener have to be said plainly. The first is its name. The header of the notes announces "Listener: opdb (TCP:1525)"; the name typed into NETCA is CLSDB, the same as the database. The port agrees, the name does not, and the notes never mention opdb again. I report both and cannot say from the notes whether the plan changed or the header was simply never updated.

The second is what the notes do not hold. There is no listener.ora, no tnsnames.ora, no lsnrctl status and no srvctl config listener output. The notes do not say why a second port was wanted beside 1521, on which addresses the listener listens, or how the instances were told to register with it. The /etc/hosts draft of the nodes has comments about "the case that the Oracle listener also listens on" the backup and management addresses, which hints at the thinking, but nothing more was written down.

flowchart TB
  cl["Database client"]
  subgraph scan["SCAN oradb-scan, 10.30.10.31 to 10.30.10.33"]
    s1["LISTENER_SCAN1"]
    s2["LISTENER_SCAN2"]
    s3["LISTENER_SCAN3"]
  end
  subgraph n1["oradb01"]
    l1["Node listener LISTENER"]
    c1["Listener CLSDB, TCP 1525"]
    i1["Instance clsdb1"]
  end
  subgraph n2["oradb02"]
    l2["Node listener LISTENER"]
    c2["Listener CLSDB, TCP 1525"]
    i2["Instance clsdb2"]
  end
  cl -- "TCP 1521" --> s1
  cl -- "TCP 1521" --> s2
  cl -- "TCP 1521" --> s3
  c1 -. "added for database CLSDB" .- i1
  c2 -. "added for database CLSDB" .- i2

The diagram shows only what the notes state: three SCAN listeners behind the name oradb-scan on port 1521, a node listener on each node (its port is not in the notes), the added listener on port 1525, and the two instances that exist after the next part. The SCAN listeners run on whichever node the cluster puts them; at the time of the Grid check one was on oradb02 and two on oradb01. I drew no line from an instance to a SCAN or node listener, because the notes show no registration.

Three disk groups

The cluster had one disk group so far: OCR on ASMDISK1, created by the Grid installer for the cluster registry and the voting file. The four remaining ASMLib disks from Storage and ASM disks were still unused. They were turned into three disk groups with the ASM Configuration Assistant. ASM belongs to Grid Infrastructure, so this tool runs as grid.

$ su - grid
$ asmca

On the tab "Disk Groups" I clicked Create three times and selected the member disks from the list the assistant offered. All disks showed the header status PROVISIONED, which is how a disk labelled by ASMLib and not yet in a group appears.

External redundancy means ASM does no mirroring: one copy of every extent, and the loss of a disk is the loss of the group. That is the usual choice when the LUNs come from a storage system that protects them itself. Whether this one does, I cannot show; the notes hold nothing about the iSCSI target. As I understand ASM, and not from the notes, with normal redundancy DATA would have had the capacity of one of its two disks and FRA and ONTRLG, with one disk each, could not have been created at all.

The sizes are worth a second look, because the notes disagree with themselves here. ASMCA shows ASMDISK2 and ASMDISK3 as the two disks of about 100 GB and ASMDISK4 as the one of about 200 GB, which matches the disk table (two data disks of 107.4 GB and a recovery disk of 214.7 GB). The oracleasm-discover output recorded earlier lists ASMDISK2 as the 214 GB disk and ASMDISK3 and ASMDISK4 as the smaller two. I cannot tell from the notes which listing is wrong; ASMCA is the one the disk groups were built from.

The answers are a Config document: ASMCA answers. The notes end with the third Create dialogue. There is no asmcmd lsdg, no crsctl status resource after the change, and nothing about allocation unit size or compatibility attributes, so I take it that the defaults of ASMCA 11.2.0.4 applied; the notes do not say.

Where this leaves the cluster

Of the three new disk groups, the database created in the next part uses two. ONTRLG exists from here on, and the notes show nothing being placed on it; Database creation and user environments comes back to that.