Skip navigation
Oracle ORAchk & EXAchk 12.2.0.1.2

New Release ORAchk 12.2.0.1.2 & EXAchk 12.2.0.1.2 Now Available

Enhanced Elastic Stack integration

The existing ORAchk & EXAchk integration with the Elastic Stack (previously called ELK stack) has been enhanced.

Many new field tags have been added to the ORAchk & EXAchk JSON output.

The extra tags allow dashboard filtering based on facts such as:

  • Engineered System type
  • Engineered System version
  • Hardware type
  • Node name
  • OS version
  • Rack identifier
  • Rack type
  • Database version
  • And more...

The Kibana dashboard can be used to view health check compliance across the data center.

Now with these new field tags results can also be filtered based on any combination of exposed system attributes.

Collection Manager health score filtering

A new filter has been added to the Collection Manager:

Health Score <= ?

This allows results to be filtered to only show collections where the health score is less than or equal to the value specified.

The default value of the filter is 100, so will show all collections initially.

A number of other existing Collection Manager field labels have been renamed for ease of use.

Oracle Platinum Services supports Oracle Auto Service Request for qualified OEM fault events

EXAchk will now check the Oracle Platinum Services monitoring agent, the monitoring user, monitored databases, and their diagnostic directories.

This helps to ensure quality of Oracle Platinum Services monitoring and that relevant diagnostics can be supplied for faster fault resolution.

Oracle Database Security Assessment Tool included

ORAchk and EXAchk now include the Oracle Database Security Assessment Tool (DBSAT).

DBSAT analyzes database configurations and security policies to uncover security risks and improve the security posture of Oracle Databases.

All DBSAT results are included within the ORAchk and EXAchk report output, under the check "Validate database security configuration using database security assessment tool".

Have you ever struggled diagnosing problems with your Oracle Database?

 

Maybe an error like an ORA-00600 or ORA-04031, database performance problems, or some other internal errors?

 

You're expected to do more work with less resources all the time. You're under pressure to keep mission-critical apps up and running.

 

Then something goes wrong. Everyone's looking at you to get the stuff needed to figure out what went wrong and how to fix it.

 

The problem is, it's not always easy... You've got to run the right tools, at the right time.

If you've using a cluster you've got to collect everything from all the nodes.

 

You might need to use lots of different tools that you only ever use now and again, and they each have their own syntax.

Once you've managed to get all this data, it can be massive. Only a fraction of what you've collected is useful, but who knows which bit is relevant?

 

That's if you were able to get everything. If you were quick enough before it was overwritten...

 

In the meantime, this is costing your company money.

You've still got a problem and you still need to get it fixed....

 

This is where Oracle Trace File Analyzer (TFA) and the database support tools bundle comes into its own.....oh and it doesn't cost you anything.

 

Here's what's covered below:

 

What does TFA do?

  • TFA gives you one interface for all your diagnostic needs.
  • It gets all the right diagnostics at the right time.
  • It can collect data across your cluster and bring it all together into one place.

Put simply, it makes it easy to get everything you need to diagnose a database problem.

 

 

Automatic diagnostic collection

TFA will monitor your logs for significant problems. If auto collection is enabled it will:

  • Invoke any necessary diagnostics and collect all relevant log data at the time of a problem
  • Trim log files around the time of the problem, collecting only what is necessary for diagnosis
  • Collect and package all trimmed diagnostics. From all nodes in the cluster and consolidate everything on a single node
  • Store the collection in the TFA repository, all ready for you to access later

TFA-autonomous.png

 

 

 

On-demand analysis and collection

You can also run TFA on-demand via the command line tool tfactl. The tfactl tool functions as:

 

  • Command line interface, to specify all commands from command line:
$ tfactl <command>

 

  • Shell interface to set and change content then run command from within the shell:

$ tfactl
tfactl >
tfaclt > database MyDB
MyDB tfactl > oratop

 

 

  • Menu interface to navigate and select options then choose the command tou want to run:
$ tfactl menu

The tfactl command can perform analysis using a combination of different tools. All using a common standard syntax.

If you still need to collect diagnostics to understand more about the problem, then use the tfactl command to start a collection.

TFA will collect all relevant log data from a time of your choosing.

It will trim logs files around the time, collecting only what is necessary for diagnosis.

All diagnostics will be packaged on a single node.

 

TFA-ondemand.png

 

 

Supported platforms and versions

TFA is supported on:

  • Linux (OEL, RedHat, SUSE, Itanium & zLinux)
  • Oracle Solaris (SPARC & x86-64)
  • AIX
  • HPUX (Itanium & PA-RISC)
  • Microsoft Windows (64bit)

It needs uses JRE 1.8 which is included.

TFA supports all Oracle Databases and Grid versions from 10.2 upwards.

You may already actually have TFA installed. Grid install shipped with TFA since 11.2.0.4 and 12.1.0.2.

The version included with grid doesn't have a number of the additional database tools. You can download the full TFA Database support tools bundle from Doc 1513912.1

 

 

How TFA works

  tfa-architecture.png

1. TFA runs a daemon on each cluster node (or single instance if you're not using grid infrastructure).

2. The tfactl command communicates with the daemon and then the daemon coordinates with all nodes.

3. On each node the daemon handles script execution, collection of diagnostics and trimming of log contents

4. Then the cluster wide collection output is consolidated on the originating node.

The process is the same when TFA performs automatic collections except it's triggered by the daemon detecting a problem.

It is also possible to install TFA in non-daemon mode. In non daemon mode there are no automatic collections and only collections from the local host can be performed.

 

 

How to install TFA

Ideally TFA should be installed as root. This will give you the most bang for your buck. However, as that's not always possible you can install TFA as the ORACLE_HOME owner.

 

Install as root user on a unix system

Download the TFA zip, ftp to required machine and unzip.

Run:

$ ./installTFA<platform>

This will discover and install on all cluster nodes. It will also configure TFA in daemon mode allowing automatic collections.

Install as non root user (ORACLE_HOME owner) on a unix system

As stated above, where possible you should install TFA as root, but sometimes that's just not an option.

When installed as non root user, TFA will not run a daemon. As such will not perform automatic collections or collect from remote hosts.

On-demand collections will work, but you will be limited to only collecting local files you have access to read.

To install without a daemon, tell TFA where to install using the –extractto option and where to find a JRE using the -javahome option. Use the JRE already available in the RDBMS home unless you have a later version available.

./installTFA<platform> -extractto <install_dir> -javahome <jre_home>

 

Install on Windows

To install on Windows, download the appropriate Oracle Trace File Analyzer zip, copy to one of the required machines and unzip.

Open a command prompt as administrator and run the installation script specifying a perl_home, e.g. D:\oracle\product\12.2.0\dbhome_1\perl

install.bat -perlhome <perl_home>

The installation will prompt if you want to do a local or cluster install. If you select cluster install, it will be installed remotely on nodes of the cluster.

Alternatively you can perform a local install on each host. Then use tfactl syncnodes to generate and deploy the relevant SSL certificates.

 

Key TFA directories

These are the key directories in the TFA install. Depending on how it was installed the tfa_home and bin directory may be in a different location. For example, when installed as part of Grid Infrastructure the tfa_home will be under grid_home.

  • tfa/bin – Contains command line interface tfactl
  • tfa/repository – Where collections are stored
  • tfa/<node>/tfa_home/database – Contains Berkeley database that stores data about system
  • tfa/<node>/tfa_home/diag – Tools for troubleshooting TFA itself
  • tfa/<node>/tfa_home/diagnostics_to_collect – Things to be included in next collection then deleted
  • tfa/<node>/tfa_home/log – Contains logs about TFA operation
  • tfa/<node>/tfa_home/resources - Contains resource files such as log masking control files
  • tfa/<node>/tfa_home/output – Contains extra meta data about the environment

 

Data masking

If you want to mask sensitive data such as hostnames or IP address from log files this can be configured via the tfa_home/resources/mask_strings.xml. In here you can define all data replacements you want to make.

For example:

tfa-masking.png

 

Configure user access

By default the grid home owner and all database home owners will have access to their respective information via TFA. No one else will be able to perform collections.

If you want to enable or disable user access you can do it via:

$ tfactl access enable

Or

$ tfactl access disable

 

All commands will apply cluster wide unless you specify -local to restrict to only the local node.

Here's some more useful user access commands:

# List user aceess
$ tfactl access lsusers

#Add users
$ tfactl access add –user <user> [-local]

# Remove users
$ tfactl access remove –user <user> [-local]

# Remove everyone
$ tfactl access removeall [-local]

# Rest to default access
$ tfactl access reset

 

 

Configure TFA for automatic notification of problems

Once TFA is installed it will automatically monitor for significant problems and if detected automatically take a diagnostic collection.

To configure email notification of these problems and where the diagnostics are follow these steps:

 

TFA is only able to do automatic monitoring/collections if installed as root.

 

 

To set the notification email to use for a specific ORACLE_HOME, include the operating system owner in the command:

 

tfactl set notificationAddress=os_user:email

 

For example:

 

tfactl set notificationAddress=oracle:some.body@example.com

 

To set the notification email to use for any ORACLE_HOME:

 

tfactl set notificationAddress=email

 

For example:

 

tfactl set notificationAddress=another.body@example.com

 

 

 

Optionally Configure the SMTP server for TFA to use when sending email notifications.

Set the SMTP parameters when prompted:

 

tfactl set smtp

 

You can verify the smtp configuration using:

 

tfactl sendmail email_address

 

When TFA detects an ORA-00600 error has occurred (or other events), you will get an email notification like this:

ORA-600 Notification.jpg

 

 

View system & cluster summary

Realtime-summary-status.png

Realtime-summary-status.png

The summary command can be used to quickly understand the viability of systems and clusters. It shows a fast, easy to read summary of the status including any potential problems with important elements such as:

  • Clusterware
  • ASM
  • Patch
  • Listener
  • OS
  • Network
  • Summary is currently only supported on Linux

Usage:

tfactl summary

 

Then if you want to drill down any further choose one of the "_overview" options.

 

 

Use TFA to analyze logs & look for errors

You can use TFA to analyze all your logs across your cluster and tell you about any recent errors.

$ tfactl analyze –last 1d

[root@myserver1 ~]# /u01/app/12.1.0/grid/bin/tfactl analyze -since 1d

INFO: analyzing all (Alert and Unix System Logs) logs for the last 1440 minutes...  Please wait...

INFO: analyzing host: myserver1

 

 

                      Report title: Analysis of Alert,System Logs

                 Report date range: last ~1 day(s)

        Report (default) time zone: EST - Eastern Standard Time

               Analysis started at: 07-Feb-2017 05:30:10 AM EST

             Elapsed analysis time: 15 second(s).

                Configuration file: /u01/app/12.1.0/grid/tfa/myserver1/tfa_home/ext/tnt/conf/tnt.prop

               Configuration group: all

               Total message count:        198,046, from 02-May-2016 06:47:35 PM EDT to 07-Feb-2017 05:30:02 AM EST

  Messages matching last ~1 day(s):          5,545, from 06-Feb-2017 05:40:01 AM EST to 07-Feb-2017 05:30:02 AM EST

        last ~1 day(s) error count:              1, from 07-Feb-2017 05:29:58 AM EST to 07-Feb-2017 05:29:58 AM EST

last ~1 day(s) ignored error count:              0

last ~1 day(s) unique error count:              1

 

 

Message types for last ~1 day(s)

   Occurrences percent  server name          type

   ----------- -------  -------------------- -----

         5,496   99.1%  myserver1     generic

            48    0.9%  myserver1     WARNING

             1    0.0%  myserver1     ERROR

   ----------- -------

         5,545  100.0%

 

 

Unique error messages for last ~1 day(s)

   Occurrences percent  server name          error

   ----------- -------  -------------------- -----

             1  100.0%  myserver1     Errors in file /u01/app/oracle/diag/rdbms/orcl2/orcl2/trace/orcl2_ora_12272.trc  (incident=10151):

                                             ORA-00600: internal error code, arguments: [600], [], [], [], [], [], [], [], [], [], [], []

                                             Incident details in: /u01/app/oracle/diag/rdbms/orcl2/orcl2/incident/incdir_10151/orcl2_ora_12272_i10151.trc

                                             Use ADRCI or Support Workbench to package the incident.

                                             See Note 411.1 at My Oracle Support for error and packaging details.

 

   ----------- -------

             1  100.0%

 

 

 

 

INFO: analyzing all (Alert and Unix System Logs) logs for the last 1440 minutes...  Please wait...

INFO: analyzing host: myserver2

 

 

                      Report title: Analysis of Alert,System Logs

                 Report date range: last ~1 day(s)

        Report (default) time zone: EST - Eastern Standard Time

               Analysis started at: 07-Feb-2017 05:30:24 AM EST

             Elapsed analysis time: 5 second(s).

                Configuration file: /u01/app/12.1.0/grid/tfa/myserver2/tfa_home/ext/tnt/conf/tnt.prop

               Configuration group: all

               Total message count:        103,280, from 02-May-2016 06:59:52 PM EDT to 07-Feb-2017 05:30:01 AM EST

  Messages matching last ~1 day(s):          1,827, from 06-Feb-2017 05:40:01 AM EST to 07-Feb-2017 05:30:01 AM EST

        last ~1 day(s) error count:              0

last ~1 day(s) ignored error count:              0

last ~1 day(s) unique error count:              0

 

 

Message types for last ~1 day(s)

   Occurrences percent  server name          type

   ----------- -------  -------------------- -----

         1,827  100.0%  myserver2     generic

   ----------- -------

         1,827  100.0%

 

 

Unique error messages for last ~1 day(s)

   Occurrences percent  server name          error

   ----------- -------  -------------------- -----

   ----------- -------

             0  100.0%

 

You can also use TFA to find all occurrences of a specific error on any node.

This command will search for anything containing "ora-006", such as the infamous ORA-00600 error.

$ tfactl analyze -search “ora-006" -last 8h

[root@myserver1 ~]# /u01/app/12.1.0/grid/bin/tfactl analyze -search "ora-00600" -since 8h

INFO: analyzing all (Alert and Unix System Logs) logs for the last 480 minutes...  Please wait...

INFO: analyzing host: myserver1

 

 

                     Report title: Analysis of Alert,System Logs

                Report date range: last ~8 hour(s)

       Report (default) time zone: EST - Eastern Standard Time

              Analysis started at: 07-Feb-2017 05:35:02 AM EST

            Elapsed analysis time: 18 second(s).

               Configuration file: /u01/app/12.1.0/grid/tfa/myserver1/tfa_home/ext/tnt/conf/tnt.prop

              Configuration group: all

                        Parameter: ora-00600

              Total message count:        195,760, from 02-May-2016 06:47:35 PM EDT to 07-Feb-2017 05:30:02 AM EST

Messages matching last ~8 hour(s):          1,584, from 06-Feb-2017 09:40:01 PM EST to 07-Feb-2017 05:30:02 AM EST

                   Matching regex: ora-00600

                   Case sensitive: false

                      Match count: 1

 

 

[Source: /u01/app/oracle/diag/rdbms/orcl2/orcl2/trace/alert_orcl2.log, Line: 3267]

Feb 07 05:29:58 2017

Errors in file /u01/app/oracle/diag/rdbms/orcl2/orcl2/trace/orcl2_ora_12272.trc  (incident=10151):

ORA-00600: internal error code, arguments: [600], [], [], [], [], [], [], [], [], [], [], []

Incident details in: /u01/app/oracle/diag/rdbms/orcl2/orcl2/incident/incdir_10151/orcl2_ora_12272_i10151.trc

Use ADRCI or Support Workbench to package the incident.

See Note 411.1 at My Oracle Support for error and packaging details.

 

 

INFO: analyzing all (Alert and Unix System Logs) logs for the last 480 minutes...  Please wait...

INFO: analyzing host: myserver2

 

 

                     Report title: Analysis of Alert,System Logs

                Report date range: last ~8 hour(s)

       Report (default) time zone: EST - Eastern Standard Time

              Analysis started at: 07-Feb-2017 05:35:19 AM EST

            Elapsed analysis time: 5 second(s).

               Configuration file: /u01/app/12.1.0/grid/tfa/myserver2/tfa_home/ext/tnt/conf/tnt.prop

              Configuration group: all

                        Parameter: ora-00600

              Total message count:        102,012, from 04-Jan-2017 03:46:41 PM EST to 07-Feb-2017 05:30:01 AM EST

Messages matching last ~8 hour(s):            374, from 06-Feb-2017 09:36:33 PM EST to 07-Feb-2017 05:30:01 AM EST

                   Matching regex: ora-00600

                   Case sensitive: false

                      Match count: 0

 

 

Support tools included

The TFA download from Doc 1513912.1 also has the Database support tools bundle. This contains a whole host of the most useful database support tools.

  • ORAchk or EXAchk - Oracle Stack Health Checks: - Depending on the type system you're running on TFA will install either EXAchk for Engineered Systems or ORAchk for all others. See 1268927.2 for more details or ORAchk and 1070954.1 for more details or EXAchk.
  • oswatcher: Collect and archive OS metrics, useful for instance / node evictions & performance Issues (See 301137.1 for more details).
  • procwatcher: Automate & capture database performance diagnostics & session level hangs (See 459694.1 for more details).
  • oratop: Near real-time database monitoring (see 1500864.1 for more details).
  • alertsummary: Provides summary of events for one or more database or ASM alert files from all nodes
  • ls: Lists all files TFA knows about for a given file name pattern across all nodes
  • pstack: Generate process stack for specified processes across all nodes
  • grep: Search alert or trace files with a given database and file name pattern, for a search string
  • summary: High level summary of the configuration
  • vi: Open alert or trace files for viewing a given database and file name pattern in the vi editor
  • tail: Run a tail on an alert or trace files for a given database and file name pattern
  • param: Show all database and OS parameters that match a specified pattern
  • dbglevel: Set and unset multiple CRS trace levels with one command
  • history: Show the shell history for the tfactl shell
  • changes: Report any noted changes in the system setup over a given time period. This includes database a parameters, OS parameters, patches applied etc

 

Each tool can be run using tfactl in shell mode

Start tfactl shell with

$ tfactl

This will put you in the tfactl shell.

tfactl >

Use the shell to run a tool with the tool name:

tfaclt > orachk

Where necessary set context with database <dbname>

tfaclt > database MyDB

Then run the tool

MyDB tfactl > oratop

Clear context with database

MyDB tfactl > database

 

How to do diagnostic collections on demand

To perform an on-demand diagnostic collection use:

$ tfactl diagcollect

This will trim & collect all important log files updated in the past 4 hours across the whole cluster.

Collections will be stored in the repository directory

[root@myserver1 ~]# /u01/app/12.1.0/grid/bin/tfactl diagcollect

 

 

Collecting data for the last 4 hours for all components...

Collecting data for all nodes

 

 

Collection Id : 20170207054023myserver1

 

 

Repository Location in myserver1 : /u01/app/oracle/tfa/repository

 

 

Collection monitor will wait up to 60 seconds for collections to start

2017/02/07 05:40:27 EST : NOTE : Any file or directory name containing the string .com will be renamed to replace .com with dotcom

2017/02/07 05:40:27 EST : Collection Name : tfa_Tue_Feb_7_05_40_23_EST_2017.zip

2017/02/07 05:40:27 EST : Sending diagcollect request to host : myserver2

2017/02/07 05:40:27 EST : Scanning of files for Collection in progress...

2017/02/07 05:40:27 EST : Collecting extra files...

2017/02/07 05:40:32 EST : Getting list of files satisfying time range [02/07/2017 01:40:27 EST, 02/07/2017 05:40:32 EST]

2017/02/07 05:40:32 EST : Starting Thread to identify stored files to collect

2017/02/07 05:40:32 EST : Getting List of Files to Collect

2017/02/07 05:40:32 EST : Trimming file : myserver1/rdbms/emcc132/emcc1321/trace/alert_emcc1321.log with original file size : 59MB

2017/02/07 05:40:32 EST : Trimming file : myserver1/rdbms/emcc132/emcc1321/trace/emcc1321_cjq0_6273.trc with original file size : 645kB

2017/02/07 05:40:32 EST : Trimming file : myserver1/rdbms/emcc132/emcc1321/trace/emcc1321_dbrm_3710.trc with original file size : 1.3MB

2017/02/07 05:40:34 EST : Trimming file : myserver1/rdbms/emcc132/emcc1321/trace/emcc1321_lmd0_3742.trc with original file size : 1.1MB

2017/02/07 05:40:34 EST : Trimming file : myserver1/rdbms/emcc132/emcc1321/trace/emcc1321_lmhb_3776.trc with original file size : 1.9MB

2017/02/07 05:40:34 EST : myserver1: Zipped 100 Files so Far

2017/02/07 05:40:34 EST : Trimming file : myserver1/rdbms/orcl2/orcl2/trace/orcl2_dbrm_3526.trc with original file size : 1.3MB

2017/02/07 05:40:34 EST : Trimming file : myserver1/rdbms/orcl2/orcl2/trace/orcl2_lmhb_3542.trc with original file size : 163MB

2017/02/07 05:40:34 EST : Finished Getting List of Files to Collect

2017/02/07 05:40:35 EST : Trimming file : myserver1/var/log/messages with original file size : 822kB

2017/02/07 05:40:35 EST : Trimming file : myserver1/crs/myserver1/crs/trace/alert.log with original file size : 8.3MB

2017/02/07 05:40:35 EST : Trimming file : myserver1/crs/myserver1/crs/trace/crsd.trc with original file size : 4.9MB

2017/02/07 05:40:35 EST : Trimming file : myserver1/crs/myserver1/crs/trace/crsd_oraagent_oracle.trc with original file size : 7.1MB

2017/02/07 05:40:35 EST : Trimming file : myserver1/crs/myserver1/crs/trace/crsd_orarootagent_root.trc with original file size : 2.7MB

2017/02/07 05:40:35 EST : Trimming file : myserver1/crs/myserver1/crs/trace/evmd.trc with original file size : 7.2MB

2017/02/07 05:40:37 EST : Trimming file : myserver1/crs/myserver1/crs/trace/ohasd.trc with original file size : 6.4MB

2017/02/07 05:40:37 EST : Trimming file : myserver1/crs/myserver1/crs/trace/ohasd_cssdagent_root.trc with original file size : 1.9MB

2017/02/07 05:40:37 EST : Trimming file : myserver1/crs/myserver1/crs/trace/ohasd_cssdmonitor_root.trc with original file size : 2MB

2017/02/07 05:40:37 EST : Trimming file : myserver1/crs/myserver1/crs/trace/ohasd_oraagent_oracle.trc with original file size : 8.4MB

2017/02/07 05:40:38 EST : Trimming file : myserver1/tnslsnr/myserver1/listener/trace/listener.log with original file size : 39MB

2017/02/07 05:40:38 EST : Trimming file : myserver1/asm/+asm/+ASM1/trace/+ASM1_lmhb_2481.trc with original file size : 142MB

2017/02/07 05:40:38 EST : Trimming file : myserver1/asm/+asm/+ASM1/trace/+ASM1_lms0_2477.trc with original file size : 868kB

2017/02/07 05:40:38 EST : Trimming file : myserver1/tnslsnr/myserver1/listener_scan1/trace/listener_scan1.log with original file size : 23MB

2017/02/07 05:40:38 EST : Trimming file : myserver1/tnslsnr/myserver1/listener_scan2/trace/listener_scan2.log with original file size : 80MB

2017/02/07 05:40:38 EST : Trimming file : myserver1/tnslsnr/myserver1/listener_scan3/trace/listener_scan3.log with original file size : 80MB

2017/02/07 05:40:39 EST : myserver1: Zipped 200 Files so Far

2017/02/07 05:41:02 EST : Collecting ADR incident files...

2017/02/07 05:41:02 EST : Waiting for collection of extra files

2017/02/07 05:43:09 EST : Completed collection of extra files...

2017/02/07 05:43:12 EST : Completed Zipping of all files

2017/02/07 05:43:12 EST : Cleaning up temporary files

2017/02/07 05:43:12 EST : Finished Cleaning up temporary files

2017/02/07 05:43:12 EST : Finalizing the Collection Zip File

2017/02/07 05:43:12 EST : Finished Finalizing the Collection Zip File

2017/02/07 05:43:12 EST : Total Number of Files checked : 32988

2017/02/07 05:43:12 EST : Total Size of all Files Checked : 4.1GB

2017/02/07 05:43:12 EST : Number of files containing required range : 205

2017/02/07 05:43:12 EST : Total Size of Files containing required range : 813MB

2017/02/07 05:43:12 EST : Number of files trimmed : 23

2017/02/07 05:43:12 EST : Total Size of data prior to zip : 1.3GB

2017/02/07 05:43:12 EST : Saved 637MB by trimming files

2017/02/07 05:43:12 EST : Zip file size : 63MB

2017/02/07 05:43:12 EST : Total time taken : 165s

2017/02/07 05:43:12 EST : Remote Collection in Progress...

2017/02/07 05:43:28 EST : myserver2:Completed Collection

2017/02/07 05:43:28 EST : Completed collection of zip files.

 

 

Logs are being collected to: /u01/app/oracle/tfa/repository/collection_Tue_Feb_7_05_40_23_EST_2017_node_all

/u01/app/oracle/tfa/repository/collection_Tue_Feb_7_05_40_23_EST_2017_node_all/myserver1.tfa_Tue_Feb_7_05_40_23_EST_2017.zip

/u01/app/oracle/tfa/repository/collection_Tue_Feb_7_05_40_23_EST_2017_node_all/myserver2.tfa_Tue_Feb_7_05_40_23_EST_2017.zip

You can change the diagcollect timeframe with –last <n>h|d

 

 

One command SRDCs

For certain types of problems Oracle Support will ask you to run a Service Request Data Collection (SRDC). That's a series of different collections to gather everything they need to help diagnose your problem.

TFA can run SRDC collections for you with a single command:

$ tfactl diagcollcet -srdc <srdc_type>


To see all the available SRDC types run

 

$ tfactl diagcollect -srdc -h

 

 

ORA-00600 SRDC

To collect all diagnostics for everything needed to diagnose and resolve ORA-00600 errors run:

 

tfactl diagcollect –srdc ORA-00600

 

TFA will prompt you the information it needs based on the SRDC type. Here's an example of running the ORA-00600 srdc.

 

[oracle@myserver1 ~]$ /u01/app/12.1.0/grid/bin/tfactl diagcollect -srdc ORA-00600

Enter the time of the ORA-00600 [YYYY-MM-DD HH24:MI:SS,<RETURN>=ALL] :

Enter the Database Name [<RETURN>=ALL] :

 

1. Feb/07/2017 05:29:58 : [orcl2] ORA-00600: internal error code, arguments: [600], [], [], [], [], [], [], [], [], [], [], []

2. Jan/16/2017 06:55:08 : [orcl2] ORA-00600: internal error code, arguments: [600], [], [], [], [], [], [], [], [], [], [], []

 

Please choose the event : 1-2 [1]

 

1.Enter default for event date/time and database name

2.TFA scans the system to identify recent events in the system (up to 10)

3.Once the relevant event is chosen, proceeds with diagnostic collection

 

Selected value is : 1 ( Feb/07/2017 05:29:58 )

Scripts to be run by this srdc: ipspack rdahcve1210 rdahcve1120 rdahcve1110

Components included in this srdc: OS DATABASE NOCHMOS

Collecting data for local node(s)

Scanning files from Feb/06/2017 23:29:58 to Feb/07/2017 11:29:58

WARNING: End time entered is after the current system time.

 

Collection Id : 20170207061417myserver1

 

Repository Location in myserver1 : /u01/app/oracle/tfa/repository

 

Collection monitor will wait up to 60 seconds for collections to start

2017/02/07 06:14:19 EST : NOTE : Any file or directory name containing the string .com will be renamed to replace .com with dotcom

2017/02/07 06:14:19 EST : Collection Name : tfa_srdc_ora600_Tue_Feb_7_06_14_17_EST_2017.zip

2017/02/07 06:14:19 EST : Scanning of files for Collection in progress...

2017/02/07 06:14:19 EST : Collecting extra files...

2017/02/07 06:14:24 EST : Getting list of files satisfying time range [02/06/2017 23:29:58 EST, 02/07/2017 06:14:19 EST]

2017/02/07 06:14:24 EST : Starting Thread to identify stored files to collect

2017/02/07 06:14:24 EST : Getting List of Files to Collect

2017/02/07 06:14:27 EST : Trimming file : myserver1/rdbms/orcl2/orcl2/trace/orcl2_dbrm_3526.trc with original file size : 1.3MB

2017/02/07 06:14:27 EST : Trimming file : myserver1/rdbms/orcl2/orcl2/trace/orcl2_lmhb_3542.trc with original file size : 163MB

2017/02/07 06:14:28 EST : Trimming file : myserver1/var/log/messages with original file size : 912kB

2017/02/07 06:14:32 EST : Finished Getting List of Files to Collect

2017/02/07 06:14:32 EST : Collecting ADR incident files...

2017/02/07 06:14:32 EST : Waiting for collection of extra files

2017/02/07 06:14:57 EST : Completed collection of extra files...

2017/02/07 06:14:58 EST : Completed Zipping of all files

2017/02/07 06:14:58 EST : Cleaning up temporary files

2017/02/07 06:14:58 EST : Finished Cleaning up temporary files

2017/02/07 06:14:58 EST : Finalizing the Collection Zip File

2017/02/07 06:14:58 EST : Finished Finalizing the Collection Zip File

2017/02/07 06:14:58 EST : Total Number of Files checked : 32996

2017/02/07 06:14:58 EST : Total Size of all Files Checked : 4.1GB

2017/02/07 06:14:58 EST : Number of files containing required range : 76

2017/02/07 06:14:58 EST : Total Size of Files containing required range : 217MB

2017/02/07 06:14:58 EST : Number of files trimmed : 3

2017/02/07 06:14:58 EST : Total Size of data prior to zip : 67MB

2017/02/07 06:14:58 EST : Saved 159MB by trimming files

2017/02/07 06:14:58 EST : Zip file size : 12MB

2017/02/07 06:14:58 EST : Total time taken : 39s

2017/02/07 06:14:58 EST : Completed collection of zip files.

 

Logs are being collected to: /u01/app/oracle/tfa/repository/srdc_ora600_collection_Tue_Feb_7_06_14_17_EST_2017_node_local

/u01/app/oracle/tfa/repository/srdc_ora600_collection_Tue_Feb_7_06_14_17_EST_2017_node_local/myserver1.tfa_srdc_ora600_Tue_Feb_7_06_14_17_EST_2017.zip

4.All required files are identified

5.Trimmed where applicable

6.Package in a zip ready to provide to support

 

For a more in-depth view of using the ORA-00600 SRDC see ORA-600 (ORA-00600 Internal Error) Detection, Diagnosis & Resolution

 

 

 

ORA-04031 SRDC

$ tfactl diagcollect –srdc ora4031

Running the ORA-04031 srdc will collect:

  1. IPS Package
  2. Patch Listing
  3. AWR report
  4. Memory information
  5. RDA

 

Database performance SRDC

$ tfactl diagcollect –srdc dbperf

Running the dbperf srdc will collect:

  1. ADDM report
  2. AWR for good period and problem period
  3. AWR Compare Period report
  4. ASH report for good and problem period
  5. OS Watcher
  6. IPS Package (if errors during problem period)
  7. ORAchk (performance related checks)

 

One command SRDC in silent mode

You can also run an SRDC collection in non-interactive silent mode. Just provide all the required parameters up front, using a syntax like:

$ tfactl diagcollect –srdc <srdc_type> -database <db> -for <time>

Here's the same ORA-00600 collection using the ora600 srdc silently:

 

[oracle@myserver1 ~]$ /u01/app/12.1.0/grid/bin/tfactl diagcollect -srdc ora600 -database orcl2 -for "Feb/07/2017"

Selected value is : ( 1. Feb/07/2017 05:29:58 : [orcl2] ORA-00600: internal error code, arguments: [600], [], [], [], [], [], [], [], [], [], [], [] )

Scripts to be run by this srdc: ipspack rdahcve1210 rdahcve1120 rdahcve1110

Components included in this srdc: OS DATABASE NOCHMOS

Collecting data for local node(s)

Scanning files from Feb/06/2017 23:29:58 to Feb/07/2017 11:29:58

WARNING: End time entered is after the current system time.

 

 

Collection Id : 20170207062336myserver1

 

 

Repository Location in myserver1 : /u01/app/oracle/tfa/repository

 

 

Collection monitor will wait up to 60 seconds for collections to start

2017/02/07 06:23:40 EST : NOTE : Any file or directory name containing the string .com will be renamed to replace .com with dotcom

2017/02/07 06:23:40 EST : Collection Name : tfa_srdc_ora600_Tue_Feb_7_06_23_36_EST_2017.zip

2017/02/07 06:23:40 EST : Scanning of files for Collection in progress...

2017/02/07 06:23:40 EST : Collecting extra files...

2017/02/07 06:23:45 EST : Getting list of files satisfying time range [02/06/2017 23:29:58 EST, 02/07/2017 06:23:40 EST]

2017/02/07 06:23:45 EST : Starting Thread to identify stored files to collect

2017/02/07 06:23:45 EST : Getting List of Files to Collect

2017/02/07 06:23:47 EST : Trimming file : myserver1/rdbms/orcl2/orcl2/trace/orcl2_dbrm_3526.trc with original file size : 1.3MB

2017/02/07 06:23:47 EST : Trimming file : myserver1/rdbms/orcl2/orcl2/trace/orcl2_lmhb_3542.trc with original file size : 163MB

2017/02/07 06:23:48 EST : Trimming file : myserver1/var/log/messages with original file size : 914kB

2017/02/07 06:23:52 EST : Finished Getting List of Files to Collect

2017/02/07 06:23:52 EST : Collecting ADR incident files...

2017/02/07 06:23:52 EST : Waiting for collection of extra files

2017/02/07 06:24:08 EST : Completed collection of extra files...

2017/02/07 06:24:13 EST : Completed Zipping of all files

2017/02/07 06:24:13 EST : Cleaning up temporary files

2017/02/07 06:24:13 EST : Finished Cleaning up temporary files

2017/02/07 06:24:13 EST : Finalizing the Collection Zip File

2017/02/07 06:24:13 EST : Finished Finalizing the Collection Zip File

2017/02/07 06:24:13 EST : Total Number of Files checked : 32996

2017/02/07 06:24:13 EST : Total Size of all Files Checked : 4GB

2017/02/07 06:24:13 EST : Number of files containing required range : 76

2017/02/07 06:24:13 EST : Total Size of Files containing required range : 218MB

2017/02/07 06:24:13 EST : Number of files trimmed : 3

2017/02/07 06:24:13 EST : Total Size of data prior to zip : 68MB

2017/02/07 06:24:13 EST : Saved 159MB by trimming files

2017/02/07 06:24:13 EST : Zip file size : 13MB

2017/02/07 06:24:13 EST : Total time taken : 33s

2017/02/07 06:24:13 EST : Completed collection of zip files.

 

 

Logs are being collected to: /u01/app/oracle/tfa/repository/srdc_ora600_collection_Tue_Feb_7_06_23_36_EST_2017_node_local

/u01/app/oracle/tfa/repository/srdc_ora600_collection_Tue_Feb_7_06_23_36_EST_2017_node_local/myserver1.tfa_srdc_ora600_Tue_Feb_7_06_23_36_EST_2017.zip

 

Keeping TFA maintained to the latest version

Oracle releases a new version of TFA every 3 months.

Applying standard PSUs will automatically update TFA. However the PSU version does not contain the rest of the Database Support Tools Bundle updates.

So if you want the latest and greatest version of TFA with the database support tools bundle download it from Doc 1513912.1

Updating is simply the same as first time install. As root use the installTFA<platform> script. If TFA is already installed, it will update the existing.

$ ./installTFA<platform>

If you're not able to install as root then install as the ORACLE_HOME owner. Use use the -extractto option

$ ./installTFA<platform> -extractto <dir> -javahome <jre_home>

 

Summary

If you want to make diagnosing your Oracle Database much quicker and easier, get TFA now.

Welcome to the My Oracle Support Community! We highly encourage you to personalize your community display name to make your activity more memorable. Please see https://community.oracle.com/docs/DOC-1022508 for instructions.