After adding our legacy databases based on the 9.2.0.6 version. We found our self troubled by “configuring” these databases to stay UP without receiving a Collection Metric Error.

The OMS would not report any problem in the tracing log, however , the agent on the database server would throw us this error in the emagent.trc;

After applying the latest PSU on our OEM GC , no changes where seen in this behavior. To make this work we also updated our Intelligent Agent to 10.2.0.5.3 and following metalink note 828464.1.
We either had to upgrade our database version to atleast 9.2.0.8, or add the following parameter to $AGENT_HOME/sysman/config/emd.properties.

We added the parameter to our property file and restarted the agent.
After this the problem was resolved.

While using DBCA in 9.2.0 the process would die after working the setup up to creating a new database. This was done on Oracle Enterprise Linux 4 update 8 which was prepared using metalink note “Requirements for Installing Oracle 9iR2 on RHEL 4 [ID 303859.1]”

On the commandline the following error is thown ( screen died ) :

To overcome this i had to export another variable before DBCA is started again :

After this the DBCA would succesfully create a new database !

When starting DBCA from a 9.2.0 installation on Oracle Enterprise Linux 4 update 8, the following error was thrown

After some research i found this to be the solution.

Today i was asked to install a new server to host an old Oracle 9.2.0.5 database.
This installation is done on OEL 4.8 using NOTE : Requirements for Installing Oracle 9iR2 on RHEL 4 [ID 303859.1]

Shortly after installing and trying to find DBCA to create a database is was suprised to not find this binairy under ORACLE_HOME/bin.
When backtracking the situation , i found this error where user interaction had chosen to continue which resulted in skipping this component.

The installation log shows the following error :

At this point the installer tries to install DBCA among other binairies. When skipped the DBCA is not available after the installer completes.
To deal with this , you need to set a proper ORACLE_BASE before the Installer is started. If no valid ORACLE_BASE is provided , the installer will try to use /oradata which in most cases the oracle user is not allowed to create :) .

After a failed backup attempt on our infrastructure database ( offline ) . We found ourself ( as it seemed ) in a world of trouble when trying to startup the OPMN processes.
The metadata repostory database was up and still the OPMN processes would not start correctly ( init states and Stopped states where reported )
In the $ORACLE_HOME/opmn/logs/OIDCTL.log we found registration like:

First we checked the database and found datafiles which needed recovery.

After this still the environment was not starting properly.

The solution to it all was to stop all processes and clean up the ODS.ODS_PROCESS table by truncating it.

SQL> truncate table ods.ods_process;

After this , the environment was reborn and back in business !

Every one who used RMAN in previous version of the Oracle database is familiar with the duplicate target database method.
Now Oracle introduced the 11G new feature method of create duplicate database without using any type of staging area.
I was asked to create an duplicate instance as acceptance database from a running 11.2 production database.

Here is a quick overview on how to use the new method.

Create a new instance on the destination server using a clean pfile.
Make sure the listener is aware of this new instance and make sure both server are able to resolve both databases using a correctly configured tnsnames.

When every thing is set , you can now preform the following action to make the magic work for you :

All done !

As part of the weekly maintenance tasks described in the best practice paper on Oracle Enterprise Manager Grid Control. I created a cron job which performs the basic optimization on the repository database. This job will force the statistics to be updates on the most active tables of EMGC and its indexes.
Offcourse in a low target monitoring environment you won’t see a big performance boost here , but i have seen it do wonders on monitoring system which gather information for 500 targets and up.

#!/bin/sh
#############################################################
# Auteur  : John Paul van Helvoort
# Owner   : Yenlo B.V.
# Datum   : 02-02-2010
# file    : week_maintenance.sh
#
#############################################################

PATH=/usr/local/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/opt/oracle/bin:/usr/bin/X11:/sbin:$PATH          ; export PATH

# ===========    VARIABLES   ================

SID=`cat /etc/oratab|grep -v "^#"|grep -v "^oms10g"|grep -v "^agent10g"|cut -f1 -d: -s`
HOST=`uname -n`
MAILADRES="support@example.nl"

# ===========    FUNCTIONS   ================

# Set Environment DB

set_environment() {
  ORACLE_SID=$SID; export ORACLE_SID
  ORAENV_ASK="NO"; export ORAENV_ASK
  . /usr/local/bin/oraenv
}

maintenance(){

  set_environment

  # Query name from database
  SQL=`sqlplus /nolog <<EOF
       conn / as sysdba
       SET ECHO OFF PAGES 2000 FEED OFF HEAD OFF
       set timing on

       SPOOL /home/oracle/scripts/week_maintenance.log

       DELETE FROM SYSMAN.MGMT_TARGETS_DELETE;
       COMMIT;

       exec dbms_stats.unlock_table_stats('SYSMAN','MGMT_METRICS_RAW');
       exec dbms_stats.gather_table_stats('SYSMAN', 'MGMT_METRICS_RAW', null, .000001, false, 'for all indexed columns', null, 'global', true, null, null, null);

       exec dbms_stats.unlock_table_stats('SYSMAN','MGMT_METRICS_1HOUR');
       exec dbms_stats.gather_table_stats('SYSMAN', 'MGMT_METRICS_1HOUR', null, .000001, false, 'for all indexed columns', null, 'global', true, null, null, null);

       exec dbms_stats.unlock_table_stats('SYSMAN','MGMT_METRICS_1DAY');
       exec dbms_stats.gather_table_stats('SYSMAN', 'MGMT_METRICS_1DAY', null, .000001, false, 'for all indexed columns', null, 'global', true, null, null, null);

      SPOOL OFF
exit
EOF
`
}

mail_yenlo(){
   mail -s "OS, EMGC, Week Maintenance Report by $HOST" $MAILADRES < /home/oracle/scripts/week_maintenance.log
}

#############################
# Main
#############################

maintenance
mail_yenlo

I have scheduled it like this in cron to be performed every Sunday at 8′oclock.

Offcourse you are free to change the frequency if needed.

The plug-in OMS Configuration has failed its perform method

After installing the OEM software using the response file, its time to configure the OEM to complete the installation.

The following error is encountered with i ran it while trying to Configure OEM 10.2.0.5 against a 11.2.0 Oracle Database.

INFO: oracle.sysman.top.oms:PerformSecureCommand:runCmd:Command Output stdout:
‘Enter SYS user’s password :
Enter repository user password :
Enter My Oracle Support user password :
Getting temporary tablespace from database…
Could not connect to SYS/(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=oem.yenlo.nl)(PORT=1521)))(CONNECT_DATA=(SID=R001))): ORA-01017: invalid username/password; logon denied (DBD ERROR: OCISessionBegin)
‘
INFO: oracle.sysman.top.oms:PerformSecureCommand:runCmd:Command Output stderr:
”
INFO: oracle.sysman.top.oms:RepManager Create Repository Error = 50. Please check the log file at /u00/oracle/product/10.2.0/oms10g/sysman/log/emca_repos_create20_13_52.log
WARNING: oracle.sysman.top.oms:EmcpPlug:startProcessing:Exception in runner=RepManager Create Repository Error = 50. Please check the log file at /u00/oracle/product/10.2.0/oms10g/sysman/log/emca_repos_create20_13_52.log
INFO: oracle.sysman.top.oms:EmcpPlug:invoke:Completed EmcpPlug invoke method on an aggregate=oracle.sysman.top.oms for Action=configuration in step=2:microstep=0
INFO: oracle.sysman.top.oms:Internal PlugIn for {Micro Step state:step:2:configuration in CfmAggregateInstance: oracle.sysman.top.oms:10.2.0.3.0:common:family=CFM:oh=/u00/oracle/product/10.2.0/oms10g:label=2} failed with an unhandled exception:
java.lang.Exception: RepManager Create Repository Error = 50. Please check the log file at /u00/oracle/product/10.2.0/oms10g/sysman/log/emca_repos_create20_13_52.log
at oracle.sysman.emcp.util.EmcpPlug.startProcessing(EmcpPlug.java:389)
at oracle.sysman.emcp.util.EmcpPlug.invoke(EmcpPlug.java:360)
at oracle.sysman.emCfg.core.PerformMicroStep.runJavaClass(PerformMicroStep.java:540)
at oracle.sysman.emCfg.core.PerformMicroStep.executeMicroStep(PerformMicroStep.java:120)
at oracle.sysman.emCfg.core.ActionPerformer.performMicroStep(ActionPerformer.java:917)
at oracle.sysman.emCfg.core.ActionPerformer$Performer.run(ActionPerformer.java:1038)

INFO: oracle.sysman.top.oms:The plug-in OMS Configuration has failed its perform method
INFO: Cfm.save() was called
INFO: Cfm.save(): 15 aggregate instances saved
INFO: done waiting for Action from 20:09:54.795

SQL>alter user sys identified by default;

After this restart your ConfigureGC.pl command which should run fine now.

You will face yet another error ( on patching the agent10g ) :

Running OMS patch configuration using /u00/oracle/product/10.2.0/oms10g/oui/bin/runConfig.sh INV_PTR_LOC=/u00/oracle/product/10.2.0/oms10g/oraInst.loc ORACLE_HOME=/u00/oracle/product/10.2.0/oms10g ACTION=patchsetConfigure MODE=perform COMPONENT_XML={oracle.sysman.top.oms.10_2_0_5_0.xml} RESPONSE_FILE=/u00/oracle/product/10.2.0/oms10g/config.rsp
perform – mode is starting for action: patchsetConfigure

perform – mode finished for action: patchsetConfigure

You can see the log file: /u00/oracle/product/10.2.0/oms10g/cfgtoollogs/oui/configActions2010-03-05_08-51-19-PM.log

10205 OMS Configuration failed with errors. Please check the above log

This is your trigger to change the sys password back to whatever you gave up in the response file installation.

Alter that restart your ConfigureGC.pl again and see your installation continue successfully.

I ran into some weird problem while following instructions on how to install Enterprise Grid Control using a response file.

I edited the em_using_existing_db.rsp and filled in the following

The i ran the following command to start the installation :

This would throw the following error :

Please do not include quotes around the BASEDIR in en em_using_existing_db.rsp as this confuses the silent installer process.
So make it look like BASEDIR=/u01/app/oracle/product/10.2.0 instead of BASEDIR=”/u01/app/oracle/product/10.2.0″

Logfile output shows a correct notation of the ORACLE_HOME as meanted.

After removing these the installer went on just fine.

After updating my Enterprise Manager Grid Control Repostory RDBMS , i was noted on the Timezone version. As the Upgrade Analyzer sugested , i should update the Database using the DBMS_DST package.
Here is the output for it , plus a way to check whether or not error occured.

oracle@xx:~/scripts> sqlplus / as sysdba

SQL*Plus: Release 11.2.0.1.0 Production on Tue Feb 16 22:37:50 2010

Copyright (c) 1982, 2009, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup
ORACLE instance started.

Total System Global Area  535662592 bytes
Fixed Size		    1337720 bytes
Variable Size		  188745352 bytes
Database Buffers	  339738624 bytes
Redo Buffers		    5840896 bytes
Database mounted.
Database opened.
SQL> select property_name, substr(property_value, 1, 30) value from database_properties where property_name like 'DST_%' order by property_name;

PROPERTY_NAME
------------------------------
VALUE
-------------------------------------------------
DST_PRIMARY_TT_VERSION
4

DST_SECONDARY_TT_VERSION
0

DST_UPGRADE_STATE
NONE
SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.

We now need to start the database in upgrade mode to execute the DBMS_DST to upgrade our version 4 to 11.

SQL> startup upgrade
ORACLE instance started.

Total System Global Area  535662592 bytes
Fixed Size		    1337720 bytes
Variable Size		  188745352 bytes
Database Buffers	  339738624 bytes
Redo Buffers		    5840896 bytes
Database mounted.
Database opened.
SQL> exec dbms_dst.begin_upgrade(11);

PL/SQL procedure successfully completed.

SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.

We now can start our database in normal mode again.

SQL> startup
ORACLE instance started.

Total System Global Area  535662592 bytes
Fixed Size		    1337720 bytes
Variable Size		  188745352 bytes
Database Buffers	  339738624 bytes
Redo Buffers		    5840896 bytes
Database mounted.
Database opened.
SQL> select property_name, substr(property_value, 1, 30) value from database_properties where property_name like 'DST_%' order by property_name;

PROPERTY_NAME
------------------------------
VALUE
------------------------------------------------
DST_PRIMARY_TT_VERSION
11

DST_SECONDARY_TT_VERSION
4

DST_UPGRADE_STATE
UPGRADE

The state shows we are in a upgrade mode.
Here is to see which tables are to be updated.

SQL> select owner, table_name, upgrade_in_progress from all_tstz_tables where owner='SYSMAN';

OWNER			       TABLE_NAME		      UPG
------------------------------ ------------------------------ ---
SYSMAN			       AQ$_MGMT_ADMINMSG_BUS_L	      YES
SYSMAN			       AQ$_MGMT_ADMINMSG_BUS_S	      YES
SYSMAN			       AQ$_MGMT_HOST_PING_QTABLE_L    YES
SYSMAN			       AQ$_MGMT_HOST_PING_QTABLE_S    YES
SYSMAN			       AQ$_MGMT_LOADER_QTABLE_L       YES
SYSMAN			       AQ$_MGMT_LOADER_QTABLE_S       YES
SYSMAN			       AQ$_MGMT_NOTIFY_QTABLE_L       YES
SYSMAN			       AQ$_MGMT_NOTIFY_QTABLE_S       YES
SYSMAN			       MGMT_VIRT_SP_RSRC	      YES
SYSMAN			       MGMT_VIRT_SP_SHARED_DISK       YES
SYSMAN			       MGMT_VIRT_VM_MASTER	      YES
SYSMAN			       MGMT_PROV_NET_CONFIG	      YES
SYSMAN			       MGMT_PROV_OPERATION	      YES
SYSMAN			       MGMT_PROV_RPM_REP	      YES
SYSMAN			       MGMT_PROV_STAGING_DIRS	      YES
SYSMAN			       MGMT_PROV_SUITE_INST_MEMBERS   YES
SYSMAN			       MGMT_PROV_ASSIGNMENT	      YES
SYSMAN			       MGMT_PROV_BOOTSERVER	      YES
SYSMAN			       MGMT_PROV_CLUSTER_NODES	      YES
SYSMAN			       MGMT_PROV_DEFAULT_IMAGE	      YES
SYSMAN			       MGMT_PROV_DHCPSERVER	      YES
SYSMAN			       MGMT_PROV_IP_RANGE	      YES

Now to see if any of these updates are failing , perform this statement.

SQL> set serveroutput on;

SQL> declare num_of_failures number;
  2  begin
  3  dbms_dst.upgrade_database(num_of_failures);
  4  dbms_output.put_line(num_of_failures);
  5  dbms_output.put_line(num_of_failures);
  6  end
  7  ;
  8  /
Table list: SYSMAN.MGMT_PROV_IP_RANGE
Number of failures: 0
Table list: SYSMAN.MGMT_PROV_DHCPSERVER
Number of failures: 0
Table list: SYSMAN.MGMT_PROV_DEFAULT_IMAGE
Number of failures: 0
Table list: SYSMAN.MGMT_PROV_CLUSTER_NODES
Number of failures: 0
Table list: SYSMAN.MGMT_PROV_BOOTSERVER
Number of failures: 0
Table list: SYSMAN.MGMT_PROV_ASSIGNMENT
Number of failures: 0
Table list: SYSMAN.MGMT_PROV_SUITE_INST_MEMBERS
Number of failures: 0
Table list: SYSMAN.MGMT_PROV_STAGING_DIRS
Number of failures: 0
Table list: SYSMAN.MGMT_PROV_RPM_REP
Number of failures: 0
Table list: SYSMAN.MGMT_PROV_OPERATION
Number of failures: 0
Table list: SYSMAN.MGMT_PROV_NET_CONFIG
Number of failures: 0
Table list: SYSMAN.MGMT_VIRT_VM_MASTER
Number of failures: 0
Table list: SYSMAN.MGMT_VIRT_SP_SHARED_DISK
Number of failures: 0
Table list: SYSMAN.MGMT_VIRT_SP_RSRC
Number of failures: 0
Table list: SYSMAN.AQ$_MGMT_NOTIFY_QTABLE_S
Number of failures: 0
Table list: SYSMAN.AQ$_MGMT_NOTIFY_QTABLE_L
Number of failures: 0
Table list: SYSMAN.AQ$_MGMT_LOADER_QTABLE_S
Number of failures: 0
Table list: SYSMAN.AQ$_MGMT_LOADER_QTABLE_L
Number of failures: 0
Table list: SYSMAN.AQ$_MGMT_HOST_PING_QTABLE_S
Number of failures: 0
Table list: SYSMAN.AQ$_MGMT_HOST_PING_QTABLE_L
Number of failures: 0
Table list: SYSMAN.AQ$_MGMT_ADMINMSG_BUS_S
Number of failures: 0
Table list: SYSMAN.AQ$_MGMT_ADMINMSG_BUS_L
Number of failures: 0

PL/SQL procedure successfully completed.

SQL>

Now the database is using the latest Timezone version.