How to Install Oracle 11g Release 2 on OEL 5.4 on VirtualBox: Installing Grid Infrastructure


In Oracle 11g Release 2 you will find that things have changed even for single instance database installation. I will try to illustrate in this series of posts how to install a single instance Oracle 11g Release 2 database to your Linux machines.

As the first part of our installation series, we will start by installing brand new Grid Infrastructure which you might think to be a fancy name for CRS+ASM but you will find out later that it is a bit more.

VirtualBox Configuration

Here is the sufficient VirtualBox virtual hardware configuration for your 11g Release 2 playground (Keep in mind that this is the bare minimum configuration to have a painless installation. More resource is obviously better):

Hardware Amount Description
Memory 512MB Although minimum memory requirement for Oracle 11g Release 2 in a real production environment is documented to be 1024M, for all practical requirements of your playground 512MB will be sufficient.
Root Disk 16GB 16 GB root disk for OS+SWAP+Oracle Binary space will be sufficient
ASM Disks 6x2GB SCSI Disks Oracle 11g Release 2 Beta 2 AyarlarWe will be doing an ASM based installation so 6 disks over SCSI interface will be enough to simulate a real life experience with ASM.

You are now ready to start your VirtualBox for Oracle 11g Release 2 installation.

Install Grid Infrastructure

Pre-work

There are a few important tasks in OS level we should complete before starting grid infrastructure installation.

Physical Partition Creation

The first thing is to create physical partitions over your virtual SCSI devices. Actually this is not crucial for ASM installation because ASM can use physical disk as a whole without any partition. However if you wish to create ASMLIB you will need those partitions.

[root@localhost ~]# fdisk /dev/sda
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won’t be recoverable.Warning: invalid flag 0x0000 of partition table will be corrected by w(rite)Command (m for help): n
Command action
e     extended
p     primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-261, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-261, default 261):
Using default value 261Command (m for help): w
The partition table has been altered!

 

Calling ioctl() to re-read partition table.
Synching disks.

Changing ASM devices ownership to oracle

Physical partitions we have created are owned by root. They should be changed to be owned by ASM user (oracle in our case) in order to make them visible for ASM discovery.

[root@localhost ~]# lsla /dev/sd?1
brw-r—– 1 root disk 8,  1 Sep 21 05:03 /dev/sda1
brw-r—– 1 root disk 8, 17 Sep 21 05:03 /dev/sdb1
brw-r—– 1 root disk 8, 33 Sep 21 05:03 /dev/sdc1
brw-r—– 1 root disk 8, 49 Sep 21 05:03 /dev/sdd1
brw-r—– 1 root disk 8, 65 Sep 21 05:03 /dev/sde1
brw-r—– 1 root disk 8, 81 Sep 21 05:03 /dev/sdf1[root@localhost ~]# chown oracle:dba /dev/sd?1[root@localhost ~]# ls –la /dev/sd?1
brw-r—– 1 oracle dba 8,  1 Sep 21 05:03 /dev/sda1
brw-r—– 1 oracle dba 8, 17 Sep 21 05:03 /dev/sdb1
brw-r—– 1 oracle dba 8, 33 Sep 21 05:03 /dev/sdc1
brw-r—– 1 oracle dba 8, 49 Sep 21 05:03 /dev/sdd1
brw-r—– 1 oracle dba 8, 65 Sep 21 05:03 /dev/sde1
brw-r—– 1 oracle dba 8, 81 Sep 21 05:03 /dev/sdf1

In order to make those changes permanent (if you reboot the system Linux will set all device owners back to root otherwise) you should create a udev permission file

[root@localhost ~]# more /etc/udev/rules.d/99-oracle.rules
#ASM disks
KERNEL==”sda”, OWNER=”oracle”, GROUP=”dba”, MODE=”0660″
KERNEL==”sdb”, OWNER=”oracle”, GROUP=”dba”, MODE=”0660″
KERNEL==”sdc”, OWNER=”oracle”, GROUP=”dba”, MODE=”0660″
KERNEL==”sdd”, OWNER=”oracle”, GROUP=”dba”, MODE=”0660″
KERNEL==”sde”, OWNER=”oracle”, GROUP=”dba”, MODE=”0660″
KERNEL==”sdf”, OWNER=”oracle”, GROUP=”dba”, MODE=”0660″

Ensure /dev/shm is sufficiently sized

You should ensure that /dev/shm is minimum 256MB for a successful ASM installation (and 512-750M for RDBMS installation) with MEMORY_TARGET parameter.

[root@localhost ~]# dfha
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00 15G 9.9G 3.9G 72% /
proc 0 0 0 /proc
sysfs 0 0 0 /sys
devpts 0 0 0 /dev/pts
/dev/hda1 99M 32M 63M 34% /boot
tempfs 125M 0 125M 0% /dev/shm
none 0 0 0 /proc/sys/fs/binfmt_misc
sunrpc 0 0 0 /var/lib/nfs/rpc_pipefs
oracleasmfs 0 0 0 /dev/oracleasm

You can resize tempfs online by using

[root@localhost ~]# mount /dev/shm –o size=750M,remount

In order to make this mount operation persistent you should modify the tempfs line in /etc/fstab file as follows:

[root@localhost ~]# cat /etc/fstab | grep tempfs
tmpfs                   /dev/shm                tmpfs   size=750m        0 0

Notice that although we have VirtualBox instance with 512M memory, Linux allows us to mount tempfs with a size of 750M. This most probably due to lazy allocation of memory over tempfs.

Create installation directory and set its ownership

Final step is to create our software directory and set the required ownership to it.

[root@localhost ~]# mkdir -p /u01/app
[root@localhost ~]# chown -R oracle:oinstall /u01/app/oracle

Installation

Camtasia Studio (5)
[oracle@localhost ~]$ unzip linux_11gR2_grid.zip
[oracle@localhost ~]$ cd grid
[oracle@localhost grid]$ ./runInstaller
Camtasia Studio (7) Choose Install and Configure Grid Infrastructure for a Standalone Server option and click on Next > button.
Camtasia Studio (14) Set Selected Language language to English and click on Next > button.
Camtasia Studio (9) Next step is to perform ASM configuration for your grid. Choose External as the redundancy of DATA diskgroup. Now click on Change Discovery Path… button to define asm_diskstring parameter.
Camtasia Studio (15) Set Disk Discovery Path to /dev/sd?1 and click on OK
Camtasia Studio (16) Since /dev/sd?1 matches all six SCSI partitions and they are not members of any other diskgroup, installer will list all of them as Candidate disks.Check-out /dev/sda1,/dev/sdb1,/dev/sdc1 devices as members disks of DATA diskgroup then click on Next >Other disks will be used for Flash Recovery Area later on.
Camtasia Studio (17) Choose Use same passwords for these account for a simple configuration then set Specify Password and Confirm Password fields to same password strings and click on Next >.I will be using sysadm throughout the post for any Oracle password required.
Camtasia Studio (18) One of the security enhancements introduced in Release 2 is the separation of different levels of ASM access. This defines different roles for “Who can start/stop ASM instance ?”, “Who can add/drop disks to/from diskgroups?” or “Who can use those diskgroups at RDBMS level ?”For the simplicity of installation we will be setting all roles to dba group.Now set all three select lists to dba and click on Next >.
Camtasia Studio (52) Set Oracle Base to /u01/app/oracle and Software Location to /u01/app/oracle/product/11.2.0/grid. Then click on Next >.
Camtasia Studio (22) Next step is unique to first Oracle software installation as you all may know. You should set Inventory Directory (if it is not already set by installer) to /u01/app/oraInventory. Keep oraInventory Group Name as oinstall and click on Next >
Camtasia Studio (23) In this step installer will check the installation prerequisites as it does in previous releases.
Camtasia Studio (26) By 11g Release 2, if any of the prerequisites fail it will be reported in a tree structure with different categories. In my case majority of the kernel settings are automatically managed since I have installed oracle-validated-configuration rpm during OEL installation.Only problem seems to be insufficiently sized core.net.wmem_max which is defined to be the maximum socket send buffer size.When you click on Fix & Check Again button, installer will generate a single shell script for you to correct all fixable errors and after its execution it will recheck for any possible problems left.
Camtasia Studio (27) Run the generated script as root user

[root@localhost ~]# /tmp/CVU_11.2.0.0.2_oracle/runfixup.sh
Response file being used is :/tmp/CVU_11.2.0.0.2_oracle/fixup.response
Enable file being used is :/tmp/CVU_11.2.0.0.2_oracle/fixup.enable
Log file location: /tmp/CVU_11.2.0.0.2_oracle/orarun.log
Setting Kernel Parameters…
net.core.wmem_max = 262144
net.core.wmem_max = 1048576

When the script is executed click OK to restart the prerequisite check process.

Camtasia Studio (31) As you see kernel parameter problem has gone. Other three errors can not be corrected by installer automatically, but we know that those are not critical ones. First one is PhysicalMemory error due to our VirtualBox 512 MB memory size. The second one is insufficient SwapSize that can be by-passed also for a play ground. And the final problem is RunLevel of Linux which is also not a great deal for us.Now check Ignore All and click on Next > (button will be enabled after checking out Ignore All) to continue.
Camtasia Studio (32) On the summary screen confirm that everything is ok and click on Finish to start installation.
Camtasia Studio (35) After installer successfully completes copy,install,link,etc steps it will pop-up a root.sh execution dialog. Run the required scripts as root.

[root@localhost ~]# /u01/app/oraInventory/orainstRoot.sh
[root@localhost ~]# /u01/app/oracle/product/11.2.0/grid/root.sh

Then switch back to installation dialog and click OK in Execute Configuration scripts dialog to proceed.

Camtasia Studio (45) Final tasks for installer will be to configure new HA service for ASM, diskgroup (DATA) and the default listener (on port 1521) which will be automatically configured also .
Camtasia Studio (48) Finally we are done🙂 Your ASM, default listener, and HA service is ready to be used.

Post Installation Checks

Login ASM

[oracle@localhost ~]$ export ORACLE_HOME = /u01/app/oracle/product/11.2.0/grid [oracle@localhost ~]$ export ORACLE_SID = +ASM
[oracle@localhost ~]$ export PATH = $ORACLE_HOME/bin:$PATH
[oracle@localhost ~]$ sqlplus / as sysasm
SQL*Plus: Release 11.2.0.0.2 Beta on Mon Sep 21 05:45:58 2009 Copyright © 1982, 2009, Oracle. All right reserved. Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.0.2 – Beta
With the Automatic Storage Management option

 

SQL> select name from v$asm_diskgroup;

NAME
——————————
DATA

Check Default Listener Status

[oracle@localhost ~]$ lsnrctl status LSNRCTL for Linux: Version 11.2.0.1.0 – Production on 31-OCT-2009 12:05:36 Copyright (c) 1991, 2009, Oracle.  All rights reserved. Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1521)))
STATUS of the LISTENER
————————
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 11.2.0.1.0 – Production
Start Date                31-OCT-2009 11:51:00
Uptime                    0 days 0 hr. 14 min. 37 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u01/app/oracle/product/11.2.0/grid/network/admin/listener.ora
Listener Log File         /u01/app/oracle/diag/tnslsnr/localhost/listener/alert/log.xml
Listening Endpoints Summary…
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=localhost.localdomain)(PORT=1521)))
Services Summary…
Service “+ASM” has 1 instance(s).
Instance “+ASM”, status READY, has 1 handler(s) for this service…
The command completed successfully

Check HA Targets

[oracle@localhost ~]$ crs_stat -t
Name           Type           Target    State     Host
————————————————————
ora.DATA.dg    ora….up.type ONLINE    ONLINE    localhost
ora….ER.lsnr ora….er.type ONLINE    ONLINE    localhost
ora.asm        ora.asm.type   ONLINE    ONLINE    localhost
ora.cssd       ora.cssd.type  ONLINE    ONLINE    localhost
ora.diskmon    ora….on.type ONLINE    ONLINE    localhost

About kocakahin

Just a computer engineer

Posted on October 30, 2009, in Oracle and tagged , , , , , , , . Bookmark the permalink. 14 Comments.

  1. Hi,

    Extra ordinary article. Thanks a lot for sharing.

    Regards,
    Gitesh
    http://www.dbametrix.com

  2. Perfect timing! I just set up my OEL 5.4 virtual machine in VirtualBox, now on to the database. With your help, I should be able to a bit more now. Thanks for sharing.

    chet

  3. nice site but not well organized

    anyway kip it up

  4. A Couple of issues solved with google:

    On CentOS 5.4 (by default running SELinux in Enforce mode.) had some errors of this kind (while running root.sh):

    /home/oracle/app/oracle/product/11.2.0/grid/bin/crsctl.bin: error while loading shared libraries: /home/oracle/app/oracle/product/11.2.0/grid/lib/libclntsh.so.11.1: cannot restore segment prot after reloc: Permission denied

    Solution:
    # getenforce
    # setenforce 0

    And the error went away, but I ended up with a bad configuration.

    [root@localhost grid]# ./root.sh
    Running Oracle 11g root.sh script…

    The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME= /home/oracle/app/oracle/product/11.2.0/grid

    Enter the full pathname of the local bin directory: [/usr/local/bin]:
    The file “dbhome” already exists in /usr/local/bin. Overwrite it? (y/n)
    [n]:
    The file “oraenv” already exists in /usr/local/bin. Overwrite it? (y/n)
    [n]:
    The file “coraenv” already exists in /usr/local/bin. Overwrite it? (y/n)
    [n]:

    Entries will be added to the /etc/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root.sh script.
    Now product-specific root actions will be performed.
    2010-02-22 10:32:38: Checking for super user privileges
    2010-02-22 10:32:38: User has super user privileges
    2010-02-22 10:32:38: Parsing the host name
    Using configuration parameter file: /home/oracle/app/oracle/product/11.2.0/grid/crs/install/crsconfig_params
    Improper Oracle Clusterware configuration found on this host
    Deconfigure the existing cluster configuration before starting
    to configure a new Clusterware
    run ‘/home/oracle/app/oracle/product/11.2.0/grid/crs/install/rootcrs.pl -deconfig’
    to configure existing failed configuration and then rerun root.sh
    [root@localhost grid]# /home/oracle/app/oracle/product/11.2.0/grid/crs/install/rootcrs.pl -deconfig
    2010-02-22 10:32:49: Parsing the host name
    2010-02-22 10:32:49: Checking for super user privileges
    2010-02-22 10:32:49: User has super user privileges
    Using configuration parameter file: /home/oracle/app/oracle/product/11.2.0/grid/crs/install/crsconfig_params
    Oracle Clusterware stack is not active on this node
    Restart the clusterware stack (use /home/oracle/app/oracle/product/11.2.0/grid/bin/crsctl start crs) and retry
    Failed to verify resources

    Solved by using solution found in: http://www.asktom.cz/?p=513

    [root@localhost grid]# /home/oracle/app/oracle/product/11.2.0/grid/crs/install/roothas.pl -verbose -delete -force

    followed by running root.sh again

  5. Please can you specify how to create the shared disks in Virtualbox

    • Hi Mohamed,
      As far as I know it is not possible with VirtualBox yet. But a trick to it might be to use NFS as the shared storage for this you will need to create a third virtual box instance and expose it as a NFS server.

  6. Hi Mohammed,

    Yes, you can create shared disks using VirtualBox (at least on 3.2.8). Here’s how you do it:

    1) Create Disks
    VBoxManage createhd –filename c:\mydb\shared\vdocr.vdi –size 1024 –format VDI –variant Fixed –type shareable –remember

    where size is in GB

    2) Share the disks between Virtual Machines

    Once you have created the disks, open virtual machine settings and under storage simply click the “+” sign.

    Asif Momen

  7. I having a problem getting with udev permission. I created the file as you mention in the article but every time I reboot the permission come back to root. Can you help

  8. Thanks for this article – very helpful. There is one minor error: the udev entry has to look like this:

    KERNEL==”sdb1″, OWNER=”oracle” GROUP=”dba”, MODE=”0660″

    At least on OEL 5 – no comma between OWNER and GROUP. With the comma, the device reverts on reboot.

  9. Hi all,

    Thanks for sharing this document,

    I have first installed database 11.2.0.2.
    Can we install and configure grid and ASM after oracle database installed?

    Thanks and Regards
    Gangadhar

  1. Pingback: Mis Compartidos en Google Reader - November 2, 2009 - webmatica.cl

  2. Pingback: Log Buffer #168: a Carnival of the Vanities for DBAs | Pythian Group Blog

  3. Pingback: How to Install 11g Release 2 on OEL 5.4 on VirtualBox: Managing ASM with ASMCA « The great grandson of Husnu Sensoy

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: