Skip to Main Content

Oracle Database Discussions

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

Interested in getting your voice heard by members of the Developer Marketing team at Oracle? Check out this post for AppDev or this post for AI focus group information.

High Memory Utilization - HPUX with 11g R2

user585870Nov 15 2013 — edited Nov 16 2013

Hi,

Observed continous 100% CPU utilization in our Oracle 11g R2 database server which is running on HP UX 11.31 server. Even when no activity in the database OEM still shows memory utilization still in 100%.

ADDM Report Shows:

Finding 1: Virtual Memory Paging,

Significant virtual memory paging was detected on the host operating system.

Host operating system was experiencing significant paging but no

      particular root cause could be detected. Investigate processes that do

      not belong to this instance running on the host that are consuming

      significant amount of virtual memory. Also consider adding more physical

      memory to the host.

If anyone come accross the same issue, please share the solution/workaround

Comments

Sandeep Kumar sk

The standard architecture is to use the 3 tier architecture of Web Tier, Middleware Tier and Database. In production, the OHS with webgates are in the DMZ so that users can access the application. I have not shown the protected applications below but they are in the MW or Application tier along with OAM, OIM, OUD etc. The Database tier is third tier.

---------------------------------------------------------------------------

                    INTERNET

  OHS/Webgate-1,  OHS/Webgate-2    (Web Tier)

            OAM,OIM Server                     (MW Tier)

            Database                                  (DB Tier)

------------------------------------------------------------------------------

The diagram does not show protected applications- the webgates are each for the protected application which can be in the middle tier along with OAM, OIM, OUD server. Below is the Identity Management architecture from Oracle docs that shows all components in Highly available production environment. Highly available components shown is 2 hosts for each of the components in a cluster - ie 2 OAM server, 2 OIM servers to ensure availability in case one of the host is unavailable.Highly available clusters are preferred for production environments but if you do not have any such requirement then go for a single instance for each component (OAM, OIM, OUD etc).

The diagram below does not show/mention tiers but the tiers are obvious (Tier1 being Web tier, Tier2 is middleware tier and Tier 3 is Database). Tier 3 below shows RAC database but should also be valid with non-RAC database.

IDM_Architecture.JPG

3336843

Thanks a lot for excellent explanation. If I have understood correctly, it means

> OHS can be configured as a reverse - proxy as well?  I want to bring WAF into the security requirements of the architecture, since even with SSO in place I want to inspect layer 7 traffic for web-application attacks. What would be appropriate place of this control placement.? Should it be place before the request for web-application reaches the Tier 1 or it should happen between tier 1 and tier 2. I have read that OHS , supports mod_security which is open-source WAF, but in our case we are having a separate WAF appliance in place 

> Also on Middle tier can IBM websphere be introduced, so the concept of technology diversity is registered , this allows to protect from compromise due to vulnerability in single instance of Weblogic or Oracle products family, Are there any performance limitations or compatibility issues I should be aware of before considering to have a difference middleware then Oracle Webgate.

Sandeep Kumar sk

>>What would be appropriate place of this control placement.?

waf protect web based apps and are typically deployed in-line. So you would place the WAF in front of the webserver, ie OHS in your case. In the below figure (taken from Imperva website), it shows Web Servers on the right hand side. In your case this is the OHS server and the WAF has been placed before the Web server (in-line mode)

WAF.JPG

Placing the WAF in-line mode will help inspection of the web traffic and protect the web application against cross-site scripting, buffer overflows, sql injection attacks, session hijacking, URL tampering etc. In this mode you will deploy WAF in Tier-1. Note: There are other modes of deployment of WAF like out-of-band, bridge mode etc depending upon your requirement. However in-line mode is used commonly.

>>Also on Middle tier can IBM websphere be introduced, so the concept of technology diversity is registered , this allows to protect from compromise due to vulnerability in single instance of Weblogic or Oracle products family, Are there any performance limitations or compatibility issues I should be aware of before considering to have a difference middleware then Oracle Webgate.

Just so we are clear, Webgate is not middleware. Webgate will be installed on a webserver. Here is definition of webgate from Oracle "A WebGate is a web-server plug-in for Oracle Access Manager (OAM) that intercepts HTTP requests and forwards them to the Access Server for authentication and authorization." Middle tier is the Weblogic and you are correct that IBM Websphere is a middleware similar to the Oracle weblogic. Back to your question of considering a different middleware other than Oracle- the answer is first about compatibility. You will have to ensure that your Oracle Identity Management products like OAM, OIM can be installed on a IBM websphere- so refer to the certification matrix to ensure compatibility. That said, you should be fine using the Oracle webglogic as middle tier since they are proven and optimized for Oracle Identity management products and certified by Oracle. This partly answers your question regarding performance limitations. So I would use Oracle Weblogic as middle tier as it is proven, robust and definitely compatible. Going websphere route will require you to undergo thorough testing and of course making sure Oracle has certified websphere as middleware for OAM, OIM etc. In addition if you go websphere route the issue of support will be crucial- like will IBM support help in case of issues with Oracle produts OAM, OIM ?. So I guess the choice is clear now- go with Oracle weblogic as middle tier. I am not saying not to use IBM websphere as middle tier. IBM websphere is an excellent product and turns out that IBM have their own competing Identity Management suite of products as well, just fyi.

3336843

Thanks a lot,

On part of DB protection I have questions that wasn't asked in previous discussion on topic.

> Should Oracle AVDF be used for oracle DB protection?

> Where would the application DB reside i.e tier 3 for non-oracle DB e.g CRM and HRMS. I'm thinking of seperate control e.g DAM/DAP i.e IBM Infosphere for its protection. Your comments?

Sandeep Kumar sk

> Should Oracle AVDF be used for oracle DB protection?

AVDF is actually two products- Audit Vault and Database Firewall. So yes Database firewall will be required/good to have, for protecting all sql queries (monitor and/or block sql queries) that are sent to database. You typically front end the Database firewall, ie put the DB firewall in front of database. Audit Vault will keep/store all the various audits logs that are collected in your environment, including the DB as well and from other sources like CRM, HRMS etc and also Operating systems in your infrastructure. So Audit Vault product is for your infrastructure and the Database firewall can support Oracle as well non-Oracle databases. Here is link that provides good overview of AVDF.

> Where would the application DB reside i.e tier 3 for non-oracle DB e.g CRM and HRMS. I'm thinking of seperate control e.g DAM/DAP i.e IBM Infosphere for its protection. Your comments?

All DB reside in tier 3, Oracle or non-Oracle. Now if you are referring to DAM and DAP capbilities then Oracle AVDF will provide you these capabilities. IBM Infosphere is a platform itself and it could provide you similar capabilities but for your environment (Oracle Identity management products, including CRM and HRMS), the Oracle AVDF should suffice.

Please mark question as answered if it has resolved your issue.

3336843

Helpful as always, but it have read that in order to populate the Audit Vault, auditing has to be enabled on the DB level and there is a known issue in this regard which relates to performance.

If you look at the Database vendors’ tech documents, they will tell you that there will be a 20-40% impact on your Database servers depending on what you audit.

While the tools are often free you need to look at the cost to your database server and TCO.

Consider If enablement of the native audit has a 50% cost the database server. – Doing the math – if you cut your database servers capacity by 50% you will need to purchase twice as many DB licenses, servers and supporting OS licenses. This can get expensive.

Sandeep Kumar sk

Yes enabling audit will have performance issues but if you follow Oracle Audit best practices as this test report says then you will not notice much performance hit. Here is a quote from the test report "The new Oracle 11g default audit settings are appropriate for any database and do not represent an important overheard. However one should not forget to use Oracle Audit best practices, control the space used by the audit and perform regular clean-up with the 11.2’ DBMS_AUDIT_MGMT package". Hopefully this answers your original question in the post. Marking the post as answered helps others who are interested in similar information.

3336843

Thanks for sharing the report. I read it , and it seems that a lot of effort was done to meet the performance requirement which for my viewpoint is still high 15.8% on additional CPU usage. Also, many changes were required to achieve these result wouldn't be considered in production environment. For e.g

> redo log writes asynchronous. This solution is only valid if you accept data loss in the event of a crash of the database.

> Choice of doing audit on the DB itself or the OS that has effect on performance as well, as discussed in the paper.

> Requires housekeeping e.g - Perform cleanup operation on all audit trail types

> Also there is no one audit trail under which the test can be performed.

> There is no standard performance architecture for non-oracle DB each has its different performance constraints e.g Requirements for SQL Server, Sybase ASE, and IBM DB2 Databases

> Also, there is no constant performance benchmark guarantee by Oracle, as it is subject to size of DB, # of transactions carried. For most DAM/DAP tools put to max 2% overhead cost, which is constant throughout the operations. 

>. Also, what is the ratio of database to # of Audit vault server, I see a bottleneck as audit data is read from the collector running with Audit vault agent on target DB, and writing it back to the relational DB which is again Oracle. There would be time the type of reading is not as same as time of writing to Audit vault DB.

> Also, I know that OAVDF doesn't have the fine-grain control as DAM/DAP solution will have which is more "security audit" then "compliance audits" . For e.g able to identify source IP or application, protocol violations, looking at triggers and stored procedures. In cases policy needs to be applied on a sensitive object e.g certain columns where the privilege admin is not authorized to see. More so, the ability to redact data or control # of rows returned from a SQL query.

truma

>>I have seen more practical implementations using a reverse proxy mode , instead of having OIAM agents on multiple web application servers. What do you suggest?

Depending on what are your requirements. Do you want to actual firewall the sql requests or just audit?

For Audit not only agents are overkill but often the proxy is not required.

The data audit can be done in so called sniffer mode with port mirroring.

Here there is an example of sniffer vs proxy mode.

https://www.datasunrise.com/blog/datasunrise-database-security-suite-overview/

Note the firewall or data masking would require at least proxy or pickup modes

Hope this helps.

truma

1 - 9
Locked Post
New comments cannot be posted to this locked post.

Post Details

Locked on Dec 14 2013
Added on Nov 15 2013
6 comments
1,140 views