This discussion is archived
1 2 3 Previous Next 38 Replies Latest reply: Nov 26, 2013 7:32 PM by jjmdb RSS

OEM12C: Any general performance tips/tricks for slow GUI.

Daryl E. Explorer
Currently Being Moderated
Just wondering if anyone has any tips here .. Navigation and such is very sluggish. Running instance and app on 32G, 16CPU box. Almost any click is a 4-6second wait for a response.

Anybody look into this and find anything that can help?

(Perhaps on the webtier???)

Thanks

Daryl.

Edited by: Daryl E. on Aug 21, 2012 10:53 AM
  • 1. Re: OEM12C: Any general performance tips/tricks for slow GUI.
    sxkumar Explorer
    Currently Being Moderated
    Can you share the details about your environment? How much resources does the OMS box have? What is your repository config?

    Also, have you looked at the sizing guidelines:

    http://www.oracle.com/technetwork/oem/framework-infra/em12c-sizing-1590739.pdf
  • 2. Re: OEM12C: Any general performance tips/tricks for slow GUI.
    pschwart Explorer
    Currently Being Moderated
    some other things to consider:

    - 32- or 64-bit OS?
    - are you running in VMs or on physical servers?
    - are you running the database and OMS on the same server?
    - have you set the recommended database parameters?
    - is the database dedicated to EM?
    - what other processes are top resource consumers on the server(s)?
    - what is the latency on your network?
  • 3. Re: OEM12C: Any general performance tips/tricks for slow GUI.
    Daryl E. Explorer
    Currently Being Moderated
    "Running instance and app on 32G, 16CPU box"

    - 32- or 64-bit OS? 64bit Rhel linux
    - are you running in VMs or on physical servers? Physical HP DL385
    - are you running the database and OMS on the same server? Yes..
    - have you set the recommended database parameters? Yes .. even more so than the recommendations. No noticeable (ie awr) performance causes. Nothing large in terms of disk reads, block gets, fulls scans etc ..

    - is the database dedicated to EM? Yes .. as per the otn install tar
    - what other processes are top resource consumers on the server(s)? box sits idle,
    - what is the latency on your network? Nothing unusual .. don't have exact numbers handy but typical 1gig ethernet connectivity.

    Am I alone here? You mean you have realistic response times (atleast as good as 10g OEM)?
    Do you not see the "fetching data" box when you scroll down anything - like say the metrics settings for a db or host?
    Reviews the sizing doc above and will take a closer look tomorrow at work but first glance says we are well above the recommendations.
    This OEM currently only has 7 Exadatas hooked up to it plus the oem box and db and one other linux server. Our 10g GC has 200-300 servers and runs circles around this config.

    What are your experiences like? I will post some timed examples tomorrow.
    Is there some web tuning that I should be doing perhaps?

    Daryl
  • 4. Re: OEM12C: Any general performance tips/tricks for slow GUI.
    mariakaval Newbie
    Currently Being Moderated
    Daryl,
    Can you also confirm, what browser and browser version are you using?
    If you go to a very simple page, e.g. <username> menu > Select My Home..., how long does it take for that simple page to load?
    In your post, one example you give is "Do you not see the "fetching data" box when you scroll down anything - like say the metrics settings for a db or host?"
    I want to see if I can replicate your issue. I go to a host home page. Then from the Host menu, I select Monitoring > Metric and Collection Settings. Is this the page you are referring to?
    Maria
  • 5. Re: OEM12C: Any general performance tips/tricks for slow GUI.
    Daryl E. Explorer
    Currently Being Moderated
    For the sake of this posting Internet Explorer 7.0.5730 (although I have played with Chrome as well)
    Entered edwp02 in search (an Exa node)
    5.95s lists the 8 objects that match (host, db, agent, listener,grid,asm ) [vs 1.18s in OEM10g]
    Click on the edwp02 host .. 8.93s [1.1s in OEM10g]
    Click on metric and settings 5.16s [2.06s]
    All metrics 4s (since we only have a few there with thresholds) [.99s]
    (can scroll no problems there - not too many) [all listed simply scroll the html page]

    Going over to see Host Metrics
    Click Metrics 9.86s [1.58s]

    Working on Incident Rules where I have created a small subset of the delivered ones
    Click on my Rule 3s to be able to clik "edit" [not quite the same but .84s to list rules]
    Click edit 7.3s [4.2s]
    Click Rules 3.08s [click metrics .92s]
    Click edit (my rules 8.5s)
    About 2 "pages" of metric alerts
    4.5s to scroll down -- "Fetching Data" [all listed no time to scroll]
    Cancel/Cancel to back out 15.47s

    Incident Mangaer 8.53 s to list 16 rows [3.65s to show a tab of OEM 10g alerts 25 of 62 shown]
    Click an inciden 8.18s to show the details in the bottom pane
    8.9s to ack and redraw the screen [1.8s to ack the whole page of them on 10g]


    Its like death by a thousand little hits..

    When Oracle demoed OEM12c they had the "usual" caveats .. using a laptop using vpn etc .. but is this the user experiences others are seeing?

    1ms lag time from my desktop browser to the OEM webserver/DB.
  • 6. Re: OEM12C: Any general performance tips/tricks for slow GUI.
    36023 Newbie
    Currently Being Moderated
    Deryl, what is the exact version of EM, the 12c 2011 Oct release 12.1.0.1.0 or 12c BP1 2012 Feb release 12.1.0.1.1?
  • 7. Re: OEM12C: Any general performance tips/tricks for slow GUI.
    Daryl E. Explorer
    Currently Being Moderated
    BP1 plus a couple recent one-offs specifically.. (removing verbose parts)

    EM Platform (OMS) 12.1.0.1.0

    Interim patches (12) :

    Patch 13915263 : applied on Wed Aug 15 10:22:00 EDT 2012
    Unique Patch ID: 14950054
    Created on 15 May 2012, 09:06:02 hrs PST8PDT

    Patch 13955792 : applied on Wed Aug 15 10:22:00 EDT 2012
    Unique Patch ID: 14950054
    Created on 15 May 2012, 09:00:18 hrs PST8PDT

    Patch 14037537 : applied on Wed Aug 15 10:22:00 EDT 2012
    Unique Patch ID: 14950054
    Created on 15 May 2012, 07:33:51 hrs PST8PDT

    Patch 13811033 : applied on Wed Aug 15 10:21:58 EDT 2012
    Unique Patch ID: 14950054
    Created on 15 May 2012, 07:47:58 hrs PST8PDT

    Patch 13718592 : applied on Wed Aug 15 10:21:49 EDT 2012
    Unique Patch ID: 14950054
    Created on 15 May 2012, 07:51:10 hrs PST8PDT

    Patch 13809908 : applied on Wed Aug 15 10:21:49 EDT 2012
    Unique Patch ID: 14950054
    Created on 15 May 2012, 09:09:12 hrs PST8PDT

    Patch 13708765 : applied on Wed Aug 15 10:21:44 EDT 2012
    Unique Patch ID: 14950054
    Created on 15 May 2012, 08:48:18 hrs PST8PDT

    Patch 13623729 : applied on Wed Aug 15 10:21:39 EDT 2012
    Unique Patch ID: 14950054
    Created on 15 May 2012, 08:59:20 hrs PST8PDT

    Patch 13682608 : applied on Wed Aug 15 10:21:39 EDT 2012
    Unique Patch ID: 14950054
    Created on 15 May 2012, 09:03:49 hrs PST8PDT

    Patch 13587440 : applied on Wed Aug 15 10:21:38 EDT 2012
    Unique Patch ID: 14950054
    Created on 15 May 2012, 08:54:36 hrs PST8PDT

    Patch 13508056 : applied on Wed Aug 15 10:21:37 EDT 2012
    Unique Patch ID: 14950054
    Created on 15 May 2012, 08:56:48 hrs PST8PDT

    Patch 13242773 : applied on Mon Mar 12 11:41:38 EDT 2012 <<BP1
    Created on 10 Feb 2012, 05:21:29 hrs PST8PDT



    DB:
    11.2.0.2 (i think that was part of the tar ball)
    Unlikely that we have patched this at all.
    Looks like I I will get that.

    Daryl
  • 8. Re: OEM12C: Any general performance tips/tricks for slow GUI.
    Daryl E. Explorer
    Currently Being Moderated
    Looking at the sizing doc .. I smoke that in terms of power. By that classification I am a small install.
    CPU and HW outlined previosly far exceeds the 2 cpu suggestion.
    db
    memory_target of 12G
    redo 500M
    processes 300 (not close to hitting this with only me on the OMS)

    Still reading ..

    Daryl
  • 9. Re: OEM12C: Any general performance tips/tricks for slow GUI.
    BrianP Journeyer
    Currently Being Moderated
    I had a combined DB+OEM12cBP1 install on a 64-bit Linux box smaller than what you describe, and had none of this slowness.

    Is there any chance your WebLogic server isn't running in production mode? Go to the WL admin console, click on GCDomain and check the Configuration tab and look for "Production mode: true".

    Is anything else (like DNS lookups) slow when run from this server?
  • 10. Re: OEM12C: Any general performance tips/tricks for slow GUI.
    Daryl E. Explorer
    Currently Being Moderated
    Not a weblogics guru .. but it shows
    Production Mode: true

    Poking around I see this:

    threadpool Warning ThreadPool has stuck threads

    Daryl
  • 11. Re: OEM12C: Any general performance tips/tricks for slow GUI.
    Daryl E. Explorer
    Currently Being Moderated
    I did a restart to clear that message.
    No change to response times.
    6.1s to return a search for a node
    20.24s to bring the host homepage from the search list
    7.8s to bring up the metric settings
    for examples.

    The weblogics console now shows this again

    Self-Tuning Thread Pool (Filtered - More Columns Exist)
    Showing 1 to 1 of 1 PreviousNext
    Active Execute Threads Execute Thread Total Count Execute Thread Idle Count Queue Length Pending User Request Count Completed Request Count Hogging Thread Count Standby Thread Count Throughput Health
    42 43 39 0 0 6632 2 1 9.49050949050949 Warning

    Daryl
  • 12. Re: OEM12C: Any general performance tips/tricks for slow GUI.
    BrianP Journeyer
    Currently Being Moderated
    Well, that exhausts my WebLogic debugging knowledge. Probably time to start checking through the log files to see if there's anything meaningful logged by the OMS or WebLogic.
  • 13. Re: OEM12C: Any general performance tips/tricks for slow GUI.
    Daryl E. Explorer
    Currently Being Moderated
    The GCDomain.log file has entries like every minute -- but doesnt appear to coincide with any GUI activity

    ####<Aug 22, 2012 12:44:15 PM EDT> <Notice> <Security> <oem12cnc.zzz.net> <EMGC_ADMINSERVER> <[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <c42881dc44ccc33b:131719d7:1394eb22272:-8000-0000000000000760> <1345653855492> <BEA-090898> <Ignoring the trusted CA certificate "OU=Security Communication RootCA2,O=SECOM Trust Systems CO.\,LTD.,C=JP". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>
    ####<Aug 22, 2012 12:44:15 PM EDT> <Notice> <Security> <oem12cnc.zzz.net> <EMGC_ADMINSERVER> <[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <c42881dc44ccc33b:131719d7:1394eb22272:-8000-0000000000000760> <1345653855494> <BEA-090898> <Ignoring the trusted CA certificate "CN=VeriSign Universal Root Certification Authority,OU=(c) 2008 VeriSign\, Inc. - For authorized use only,OU=VeriSign Trust Network,O=VeriSign\, Inc.,C=US". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>
    ####<Aug 22, 2012 12:44:15 PM EDT> <Notice> <Security> <oem12cnc.zzz.net> <EMGC_ADMINSERVER> <[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <c42881dc44ccc33b:131719d7:1394eb22272:-8000-0000000000000760> <1345653855500> <BEA-090898> <Ignoring the trusted CA certificate "CN=KEYNECTIS ROOT CA,OU=ROOT,O=KEYNECTIS,C=FR". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>

    But I see really long dumps..
    ####<Aug 22, 2012 12:42:26 PM EDT> <Warning> <oracle.ods.virtualization.exception> <oem12cnc.rim.net> <EMGC_OMS1> <126::[ACTIVE] ExecuteThread: '34' for queue: 'weblogic.kernel.Default (self-tuning)'> <SYSMAN> <> <004lxphJa282fL1_vtt1iX0002vm0001MC> <1345653746535> <BEA-000000> <OVD-40066
    javax.naming.NameNotFoundException: [LDAP: error code 32 - No Such Object]; remaining name 'dc=oracle,dc=com'
    at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3066)
    at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2987)
    at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2794)
    ..
    ..
  • 14. Re: OEM12C: Any general performance tips/tricks for slow GUI.
    Daryl E. Explorer
    Currently Being Moderated
    For the security error above, metalink says this .. but I dont know where I can set this. Perhaps buried in emctl.pl somewhere?

    WebLogic Server: Getting "java.security.cert.CertificateParsingException: PKIX: Unsupported OID" exception with SHA256 algorithm [ID 1348287.1]

    Solution
    In WLS 10.3.3 and higher, to resolve this issue you can enable the JSEE SSL provider with the following JVM startup argument:

    -Dweblogic.security.SSL.enableJSSE=true
1 2 3 Previous Next

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points