Issue encounter during Oracle Database 11.2.0.4 installation on Oracle Linux 7

1. Packages "elfutilslibelfdevel0.97" and "pdksh5.2.14" Are Missing
While installing the 11.2.0.4 database software on a Oracle Linux 7 the Oracle Universal Installer (OUI) reports that packages "elfultilslibelfdevel0.97" and "pdksh5.2.14" are missing.

Solution:
a. Under the /database directory, modify the file /database/stage/cvu/cv/admin/cvu_config , search for CV_ASSUME_DISTID and replace OEL4 with OEL7 as below.
CV_ASSUME_DISTID=OEL4
to:
CV_ASSUME_DISTID=OEL7

b. Reinstall the 11.2.0.4 software using /database/runInstaller.
OUI should now perform the OEL7 prerequisite checks (which are identical to the RHEL7 prerequisite checks) and no longer report that packages "elfutilslibelfdevel0.97" and "pdksh5.2.14" are missing.

2. Error in invoking target 'agent nmhs' of makefile
While installing Oracle Database 11.2.0.4 software, you may get an error during the LINK stage of installation:
Exception String: Error in invoking target 'agent nmhs' of makefile '/u01/oracle/app/product/11.2.0/db_1/sysman/lib/ins_emagent.mk'.

Solution:
Modify the file /sysman/lib/ins_emagent.mk, search for MK_EMAGENT_NMECTL and add in -lnnz11 as below. 
#===========================
#  emdctl
#===========================
$(SYSMANBIN)emdctl:
        $(MK_EMAGENT_NMECTL) -lnnz11

Save the file and click “RETRY” to resume the installation. The installation should be able to complete successfully.

TWO_TASK

TWO_TASK
This UNIX environment variable specify a connection string. Connections that do not specify a database will connect to the database specified in TWO_TASK.

Eg.

TWO_TASK=TEST
export TWO_TASK
sqlplus hr
is the same as:

sqlplus scott@TEST

Take note that this variable will even overwrite $ORACLE_SID and cause ORA-01031: insufficient privileges when you connect as sysdba.

Eg.
SQL> conn / as sysdba
ERROR:
ORA-01031: insufficient privileges

To unset,
sh or ksh:
unset TWO_TASK
csh:
unsetenv TWO_TASK

Eg.
$ unset TWO_TASK
$ sqlplus /nolog
SQL> conn / as sysdba
Connected.

ORA-00600: internal error code, arguments: [729], [], [space leak]

Refer to error captured from udump.

Unix process pid: 18832, image: oracle@xxxxxx (TNS V1-V3)

*** 2009-08-20 20:11:02.518
*** SESSION ID:(33.19527) 2009-08-20 20:11:02.516
******** ERROR: UGA memory leak detected 9536 ********
******************************************************
HEAP DUMP heap name="session heap" desc=0x1a6e908
extent sz=0x108c alt=32767 het=32767 rec=0 flg=3 opc=3
parent=1a14c28 owner=900ac708 nex=0 xsz=0x2050
EXTENT 0
Chunk 1aa2e84 sz= 4228 free " "
EXTENT 1
Chunk 1a8493c sz= 888 free " "
Chunk 1a84cb4 sz= 740 freeable "define var info"
EXTENT 2
Chunk 1ab9094 sz= 536 free " "
Chunk 1ab92ac sz= 376 freeable "define var info"
Chunk 1ab9424 sz= 184 free " "
Chunk 1ab94dc sz= 740 freeable "define var info"
Chunk 1ab97c0 sz= 536 free " "
Chunk 1ab99d8 sz= 376 freeable "define var info"
Chunk 1ab9b50 sz= 740 freeable "define var info"
.
.
.
.
.
*** 2009-08-20 20:11:02.521
ksedmp: internal or fatal error
ORA-00600: internal error code, arguments: [729], [9536], [space leak], [], [], [], [], []

A space leak has been detected in the User Global Area (UGA). There is no data corruption as a result of this error. It is an internal memory housekeeping problem. The second argument,9536 is the number of bytes leaked.

The above dump show Memory leak. Oracle is trying to free memory allocated to a process. Chunk 1a84cb4,1ab92ac and the rest with freeable "define var info" show memory leaked.

This can be safely ignore for small memory leaks by adding the following event to init.ora:
event = "10262 trace name context forever, level 10240"
Then, restart your database. This event disables space leaks less than 10 kbytes.

You can see the details at Metalink Doc ID: 31056.1 ORA-600 [729] "UGA Space Leak" and Doc ID: 403584.1 Understanding and Diagnosing ORA-600 [729] Space Leak Errors

Bug:
Bug:2177050: ORA-600 [729] after application of the 8.1.7.3 patchset. The resulting trace file will include a memory dump which shows unfreed memory chunks with the tags "define var info" and/or "oactoid info".

Memory Notification: Library Cache Object loaded into SGA

Often you will see below error from alert log in Oracle Database 10g.
Memory Notification: Library Cache Object loaded into SGA
Heap size 2135K exceeds notification threshold (2048K)
Details in trace file

These warning messages is indicated as a Notification in alert messages and should not cause the session to fail. They appear as a result of new event messaging mechanism and memory manager in 10g Release 2.

This means that the process is just spending a lot of time in finding free memory extents during an allocation as the memory may be heavily fragmented. Fragmentation in memory is impossible to eliminate completely, however, continued messages of large allocations in memory indicate there are tuning opportunities on the application.

In 10.2.0.1 set the new undocumented parameter, the KGL heap size warning threshold.
Set _kgl_large_heap_warning_threshold to a reasonable high value or zero to prevent these warning messages. Value needs to be set in bytes.

SQL> alter system set "_kgl_large_heap_warning_threshold"=8388608 scope=spfile ;

SQL> shutdown immediate SQL> startup

SQL> show parameter _kgl
NAME TYPE VALUE
--------------------------------- ------- ----------------
_kgl_large_heap_warning_threshold integer 8388608


This messages does not appear in version 10.2.0.2 or later, so upgrade to the latest version is also another solution to this.

For details, refer to metalink document, Doc ID: 330239.1

Cloning a New VMware Testing Environment (RHEL4)

As a DBA, I perform a lot of testing on Oracle Database related functionalities and features. I do most of my testing on a VMware virtual machine. Instead of creating a new virtual machine each time, I have created/installed a master virtual machine for each platform and each time when I require a new testing environment, I'll clone from my master machine. Before everything is working, there are some configurations that need to be set.

Before shutting down the master VM:

Enable Telnet service.
1. Login as root and install telnet-server if it's not already installed. Go to Legacy Network Server under Add or Remove Packages,click on Details and check "telnet-server".

2. Hit Update to install the necessary packages. You will need to insert the RedHat CD/DVD as per instruction.

3. Run below commands:
# chkconfig telnet on
# service xinetd reload

Enable FTP service
1. login as root and install vsftpd, "Very Secure FTP Daemon" if it's not already installed. Just check "FTP Server" under Add or Remove Packages.

2. Hit Update to install the necessary packages. You will need to insert the RedHat CD/DVD
as per instruction.

3. Start vsftpd
# /etc/init.d/vsftpd start

4. To start vsftpd automatically during reboot,
# ln -s /etc/init.d/vsftpd /etc/rc3.d/S56vsftpd
# ln -s /etc/init.d/vsftpd /etc/rc4.d/S56vsftpd
# ln -s /etc/init.d/vsftpd /etc/rc5.d/S56vsftpd

To allow root user to telnet and ftp
1. Edit the file /etc/securetty and add the following to the end of the file:
pts/0
pts/1
pts/2
pts/3
pts/4
pts/5
pts/6
pts/7
pts/8
pts/9

2. Edit the files /etc/vsftpd.ftpusers and /etc/vsftpd.user_list and comment the 'root' line from each file.

Install VMware Client tools
1. Login as root and then choose Install VMware Tools... from the VM main menu. This will auto mount a cd contains VMware tool package.

2. Open up the cd and install from the rpm package.

3. Then, run the "vmware-config-tools.pl" as root and hit enter on the resolution option.
This should complete the VMware client tools. You can now launch "vmware-toolbox" to
configure time synchronization.

Shutdown and keep the master VM copy. Whenever necessary, copy from the master vm directory and File>>Open the new copied VM from VMware Server console.

On the new added virtual machine:
Probe a new network adaptor(s) MAC address for the new virtual machine
1. Go to Applications menu, System Settings >> Network
On every network adaptor, double click to launch the settings page. Under Hardware Device,
probe a new MAC address for that adaptor.

2. Change the IP addresses accordingly

3. Activate/reactivate the adaptor(s) to take the new network settings.

Change hostname to suite your environment
eg. from master to prod

1. Modify the file /etc/sysconfig/network
NETWORKING=yes
HOSTNAME="prod.localdomain"

2. Do not forget /etc/hosts
Add or modify the entry, Eg.
192.168.0.100 prod.localdomain prod

3. Reboot the OS.

Recovering a Deleted Datafile Without a Backup

Guess what, RMAN can handle lost datafiles even if you do not have a backup.

The conditions:
1. The control file knows about the datafile, that is, the user backed up the control file after datafile creation, but the datafile itself is not backed up.

2. If the datafile record is in the control file, then RESTORE creates the datafile in the original location or in a user-specified location (for example, with SET NEWNAME). The RECOVER command can then apply the necessary logs to the datafile.

3. The control file does not have the datafile record, that is, the user did not back up the control file after datafile creation. During recovery, the database will detect the missing datafile and report it to RMAN, which will create a new datafile and continue recovery by applying the remaining logs. If the datafile was created in a parent incarnation, it will be created during restore or recover as appropriate.

Example:
1. Make a full database backup of your ARCHIVELOG mode database.
RMAN> BACKUP DATABASE TAG="FULL";

2. You create a tablespace "TEST" containing a single datafile called /orcl/test01.dbf.

3. create a table and populate some data.

4. Archive all the active online redo logs.
SQL> ALTER SYSTEM ARCHIVE LOG CURRENT;

5. Delete the file /orcl/test01.dbf.

6. Take the tablespace with the missing datafile offline.
SQL> ALTER TABLESPACE TEST OFFLINE IMMEDIATE;

7. Restore the tablespace even without backup.
RMAN> RESTORE TABLESPACE "TEST";

8. Recover tablespace.
RMAN> RECOVER TABLESPACE "TEST";

9. Bring the recovered tablespace online.
SQL> ALTER TABLESPACE TEST ONLINE;

10. Verify the contents of the tablespace.

Disabling OS-Authentication

On the server where that host the database, edit the file $ORACLE_HOME/network/admin/sqlnet.ora

Set SQLNET.AUTHENTICATION_SERVICES=(NONE)

The second way will be remove oracle OS user from dba group or ORA_DBA on Windows.