OAS 6.4 stuck in signing in, after moving to exadata

SteveF-Oracle

@paamikumar
That is great to hear. I will mark your response as the answer.

I see you also made some additional improvements based on my comments in the SR.

* soft nofile 131072
* hard nofile 131072.

As well, as some other suggestions here:
Cleaning up initialization blocks, and other items that may contribute to throughput on the system.

We are happy your issue is resolved.
Please do consider updating to OAS 2024 to stay within error correction policy for critical patch updates and features fixes.

paamikumar

We did not make any other changes.

Thanks,

paamikumar

Hi Steve,

Thank you for responding. Here is my response.

1. Update to the latest patched client tools —- working on it
2. Clean diagnostic dump after the issue reproduces to assist in narrowing the issue/ check for any crashes, etc. - yes, yesterday i provided the diagnostic dump from last few days. —-We will limit it to the day.
   3. Questions about agent schedule/backlog —- we have disabled all agents in prod.
   4. RPD configuration for Exadata Oracle DB source needs some updates. —-We updated the db connections to use Oracle Exadata in our dev environment, but on testing found that the server variables are not getting updated. So we had to back off.
   5. Checking for initialization block failures, hangs. —-We have deferred all and cleaned all of them.

6. Checking ulimits. —-

ulimit -Ha


core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 512994
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 262144
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) unlimited
cpu time (seconds, -t) unlimited
max user processes (-u) 512994
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

ulimit -a


core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 512994
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 4096
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 512994
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

Thanks,

Paami

paamikumar

Thank you so much Mostafa for the update. Nothing is crashing right now. Will run the query you provided when issue occurs.

We already have DEFAULT_DB_MAX_EXEC_TIME = 3600

MARKING_QUERY_MAX_LOGICAL_QUERY_EXEC_TIME = 120

Also added the following to the instanceconfig.xml. Let's see what tomorrow brings us.

<Query>
<WorkerThreadSizeKb>4096</WorkerThreadSizeKb>
</Query>

Thanks,

Paami

paamikumar

We ran the select count(*),program from gv$session where LAST_CALL_ET > 600 and status = 'INACTIVE' group by program; when we had the issue today.

Get only 6 records

rcu

COUNT(*)	PROGRAM
1	oraagent.bin@pdoradbadm02.eprod.com (TNS V1-V3)
1	java.lang.Thread
8	Oracle REST Data Services
2	com.informatica.jdbc.oraclebase.ddae
1	JDBC Thin Client
datawarehouse
COUNT(*)	PROGRAM
1	JDBC Thin Client
2	oracle@pdoradb20corp.eprod.com (TNS V1-V3)
1	w3wp.exe
1	oraagent.bin@pdoradbadm01.eprod.com (TNS V1-V3)
9	nqsserver@pdobiapp02corp.eprod.com (TNS V1-V3)

paamikumar

we ran the query select count(*),program from gv$session where LAST_CALL_ET > 600 and status = 'INACTIVE' group by program; when we had the issue today. Just gets 6 records.

rcu

COUNT(*)	PROGRAM
1	oraagent.bin@pdoradbadm02.eprod.com (TNS V1-V3)
1	java.lang.Thread
8	Oracle REST Data Services
2	com.informatica.jdbc.oraclebase.ddae
1	JDBC Thin Client
datawarehouse
COUNT(*)	PROGRAM
1	JDBC Thin Client
2	oracle@pdoradb20corp.eprod.com (TNS V1-V3)
1	w3wp.exe
1	oraagent.bin@pdoradbadm01.eprod.com (TNS V1-V3)
9	nqsserver@pdobiapp02corp.eprod.com (TNS V1-V3)

paamikumar

select count(*),program from gv$session where LAST_CALL_ET > 600 and status = 'INACTIVE' group by program;

rcu

COUNT(*)	PROGRAM
1	oraagent.bin@pdoradbadm02.eprod.com (TNS V1-V3)
1	java.lang.Thread
8	Oracle REST Data Services
2	com.informatica.jdbc.oraclebase.ddae
1	JDBC Thin Client
datawarehouse
COUNT(*)	PROGRAM
1	JDBC Thin Client
2	oracle@pdoradb20corp.eprod.com (TNS V1-V3)
1	w3wp.exe
1	oraagent.bin@pdoradbadm01.eprod.com (TNS V1-V3)
9	nqsserver@pdobiapp02corp.eprod.com (TNS V1-V3)