OMS decided to shutdown the agent because of the following reason sent from OMS: AGENT_TZ_MISMATCH

Today I was trying to start my OEM agent in one of my dev server and got below error while starting it :-

==>./emctl start agent
Oracle Enterprise Manager Cloud Control 12c Release 4
Copyright (c) 1996, 2014 Oracle Corporation.  All rights reserved.
Starting agent ...........failed.
Consult emctl.log and emagent.nohup in: /u01/oracle/product/agent/agent_inst/sysman/log

Cause :-

Crosschecked the logs as mentioned above and got below in emagent.nohup logfile 

highlighted one below is of our interest

 --- EMState agent

----- Wed Jun 14 11:58:16 2017::20552::Auto tuning the agent at time Wed Jun 14 11:58:16 2017 -----

----- Wed Jun 14 11:58:17 2017::20552::Finished auto tuning the agent at time Wed Jun 14 11:58:17 2017 -----

----- Wed Jun 14 11:58:17 2017::20552::Launching the JVM with following options: -Xmx128M -server -Djava.security.egd=file:///dev/./urandom -Dsun.lang.ClassLoader.allowArraySyntax=true -XX:+UseLinuxPosixThreadCPUClocks -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:+UseCompressedOops -----

----- Wed Jun 14 11:58:17 2017::20552::Agent Launched with PID 20672 at time Wed Jun 14 11:58:17 2017 -----

----- Wed Jun 14 11:58:17 2017::20552::Previous Thrash State(-1,-1) -----

----- Wed Jun 14 11:58:17 2017::20672::Time elapsed between Launch of Watchdog process and execing EMAgent is 1 secs -----

2017-06-14 11:58:17,396 [1:main] WARN - Missing filename for log handler 'wsm'

2017-06-14 11:58:17,402 [1:main] WARN - Missing filename for log handler 'opss'

2017-06-14 11:58:17,403 [1:main] WARN - Missing filename for log handler 'opsscfg'

OMS decided to shutdown the agent because of the following reason sent from OMS:  AGENT_TZ_MISMATCH

----- Wed Jun 14 11:58:26 2017::20552::Checking status of EMAgent : 20672 -----

----- Wed Jun 14 11:58:26 2017::20552::EMAgent exited at Wed Jun 14 11:58:26 2017 with return value 0. -----

----- Wed Jun 14 11:58:26 2017::20552::writeAbnormalExitTimestampToAgntStmp: exitCause=NORMAL : restartRequired=0 -----

----- Wed Jun 14 11:58:26 2017::20552::EMAgent was shutdown normally. -----

----- Wed Jun 14 11:58:26 2017::20552::Exiting watchdog loop

We can clearly see that there is some timezone mismatch issue of the agent with the OMS repository server

Solution :-

As above error state that there is a timezone mismatch between the agent server and OMS repository server.

or OMS repository database must be having incorrect timezone related details for agent server

Below steps resolved my error:-

[Server1][oracle][amit1]

EM_HOME/bin$

==>./emctl resetTZ agent

Oracle Enterprise Manager Cloud Control 12c Release 4

Copyright (c) 1996, 2014 Oracle Corporation.  All rights reserved.

Updating /u01/oracle/product/agent/agent_inst/sysman/config/emd.properties...

Successfully updated /u01/oracle/product/agent/agent_inst/sysman/config/emd.properties.

Login as the em repository user and run the  script:

exec mgmt_target.set_agent_tzrgn('Server1.test.com:9901','US/Eastern')

and commit the changes

This can be done for example by logging into sqlplus and doing

SQL> exec mgmt_target.set_agent_tzrgn('Server1.test.com:9901','US/Eastern')

SQL> commit

Login to OMS repository database and connect using SYSMAN and execute below procedure :-

exec mgmt_target.set_agent_tzrgn('Server1.test.com:9901','US/Eastern')

Do a clearstate and securing of your agent again and start it :-

==>./emctl clearstate agent

Oracle Enterprise Manager Cloud Control 12c Release 4

Copyright (c) 1996, 2014 Oracle Corporation.  All rights reserved.

EMD clearstate completed successfully

[Server1][oracle][amit1]

EM_HOME/bin$

==>./emctl secure agent

Oracle Enterprise Manager Cloud Control 12c Release 4

Copyright (c) 1996, 2014 Oracle Corporation.  All rights reserved.

Agent is already stopped...   Done.

Securing agent...   Started.

Enter Agent Registration Password :

Securing agent...   Successful.

[Server1][oracle][amit1]

EM_HOME/bin$

==>./emctl start agent

Oracle Enterprise Manager Cloud Control 12c Release 4

Copyright (c) 1996, 2014 Oracle Corporation.  All rights reserved.

Starting agent ..... started.

Agent started successfully

I hope this article helped you

Thanks 

Amit Rath

OMS 12.1.0.4 installation failed with "The database details which you have provided doesn't contain valid Management Repository. Provide correct database details and retry"

Yesterday I was installing OMS 121.0.4 and got below error while attempting to connect to database:-

OMS 12.1.0.4 installation failed with "The database details which you have provided doesn't contain valid Management Repository. Provide correct database details and retry"

Cause :-

Generally this error comes when the database we are using as OMS database was already used as a database for OMS installation before or A failed OMS installation already happened using this database.

Solution :-

we have to properly clean the database before using it again as a OMS database :-

Below steps I did to clean my database :-

1. Drop sysman related schemas.

DROP USER SYSMAN CASCADE;  ( will take time)
DROP USER SYSMAN_OPSS CASCADE;
DROP USER SYSMAN_MDS CASCADE;
DROP USER SYSMAN_APM CASCADE;
DROP USER SYSMAN_RO CASCADE;

2. Remove Synonyms related to sysman accounts :-

DECLARE
  CURSOR l_syn_csr IS
    SELECT 'DROP ' ||
      CASE owner
        WHEN 'PUBLIC'
          THEN 'PUBLIC SYNONYM '
        ELSE 'SYNONYM ' || owner || '.'
      END ||
      synonym_name AS cmd
    FROM
      dba_synonyms 
    WHERE
      table_owner IN (
        'SYSMAN',
        'SYSMAN_MDS',
        'MGMT_VIEW',
        'SYSMAN_BIP',
        'SYSMAN_APM',
        'BIP',
        'SYSMAN_OPSS',
        'SYSMAN_RO'
      );
BEGIN
  FOR l_syn_rec IN l_syn_csr LOOP
    BEGIN
      EXECUTE IMMEDIATE l_syn_rec.cmd;
    EXCEPTION
      WHEN OTHERS THEN
        dbms_output.put_line( '===> ' || l_syn_rec.cmd );
        dbms_output.put_line( sqlerrm );
    END;
  END LOOP;
END;
/

3. Removing remaining Objects and tablespaces :-

DROP USER mgmt_view CASCADE;
DROP TABLESPACE mgmt_ecm_depot_ts INCLUDING CONTENTS AND DATAFILES CASCADE CONSTRAINTS;
DROP TABLESPACE mgmt_tablespace   INCLUDING CONTENTS AND DATAFILES CASCADE CONSTRAINTS;
DROP TABLESPACE mgmt_ad4j_ts      INCLUDING CONTENTS AND DATAFILES CASCADE CONSTRAINTS;

4.  As proper database cleaning using RepManager dropall didn't happen, so we have to clean up the registry details :-

DELETE
  FROM
    schema_version_registry
  WHERE
    (comp_name,owner) IN (
      ('Authorization Policy Manager','SYSMAN_APM'),
      ('Metadata Services','SYSMAN_MDS'),
      ('Oracle Platform Security Services','SYSMAN_OPSS')
    );
commit;

5. Now the database clean up completed, Please continue with the installation, I tried and it completed successfully.

I hope this article helped you, Please go through the below article also for proper error details :-
https://sites.google.com/site/yetanotherocm/home/system-builds/cronulla/07-complete-oms-12-1-0-3-1-installation

Thanks
Amit Rath