1 2 Previous Next 29 Replies Latest reply: Jun 7, 2013 2:25 AM by karthiksingh_dba RSS

    EBS R12.1.3 HTTP is down.

    karthiksingh_dba
      Dear Legends,

      We have a VISION instance in Amazon, Multi node as there was an IP Change only in APPS Tier and So I have created a new Context file and then tried to start the apps tier but not successful in HTTP.

      /u02/E-BIZ/inst/apps/VISION_ip-10-116-174-104/admin/scripts/adapcctl.sh start
      script returned:
      ****************************************************

      You are running adapcctl.sh version 120.7.12010000.2

      Starting OPMN managed Oracle HTTP Server (OHS) instance ...
      opmnctl: opmn is already running.
      opmnctl: starting opmn managed processes...
      ================================================================================
      opmn id=ip-10-116-174-104.ec2.internal:6200
      0 of 1 processes started.

      ias-instance id=VISION_ip-10-116-174-104.ip-10-116-174-104.ec2.internal
      ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
      ias-component/process-type/process-set:
      HTTP_Server/HTTP_Server/HTTP_Server/
      Error
      --> Process (index=1,uid=196299954,pid=3497)
      failed to start a managed process after the maximum retry limit
      Log:
      /u02/E-BIZ/inst/apps/VISION_ip-10-116-174-104/logs/ora/10.1.3/opmn/HTTP_Server~1.log
      adapcctl.sh: exiting with status 0
      adapcctl.sh: check the logfile /u02/E-BIZ/inst/apps/VISION_ip-10-116-174-104/logs/appl/admin/log/adapcctl.txt for more information ...

      ./adopmnctl status
      ============
      Processes in Instance: VISION_ip-10-116-174-104.ip-10-116-174-104.ec2.internal
      --------------------------------------------------------------+---------
      ias-component | process-type | pid | status
      --------------------------------------------------------------+---------
      OC4JGroup:default_group | OC4J:oafm | 3757 | Alive
      OC4JGroup:default_group | OC4J:forms | 3686 | Alive
      OC4JGroup:default_group | OC4J:oacore | 3541 | Alive
      HTTP_Server | HTTP_Server | N/A | Down

      adopmnctl.sh: exiting with status 0

      Tries so for Did as suggested:
      1. ID 821419.1, ID 744433.1
      2. ln -s /usr/lib/libgdbm.so.2.0.0 /usr/lib/libdb.so.2
      ln: creating symbolic link `/usr/lib/libdb.so.2' to `/usr/lib/libgdbm.so.2.0.0': File exists
      3. Checked for any *.lock files in All the components but none.
      4. Changed the newly created context file according to Amazon, because amazon web entry URL would accept only ec2 which is the Public DNS not the ip2.

      Q: Shall i increase the default start timeout parameter value from 300 to a larger value as 600 or 1200? Will it solve my problem?
      Info: Did Autoconfig in both the tiers and both ran successfully without any errors.

      HTTP_Server~1.log says
      ===============
      13/05/24 14:58:27 Start process
      --------
      /u02/E-BIZ/inst/apps/VISION_ip-10-116-174-104/ora/10.1.3/Apache/Apache/bin/apachectl startssl: execing httpd
      --------
      13/05/24 14:58:29 Start process
      --------
      /u02/E-BIZ/inst/apps/VISION_ip-10-116-174-104/ora/10.1.3/Apache/Apache/bin/apachectl startssl: execing httpd

      Please help me and provide your valuable suggestions. Thanks in advance.
      Regards,
      Karthik

      Edited by: karthiksingh_dba on May 24, 2013 12:24 PM
        • 1. Re: EBS R12.1.3 HTTP is down.
          Hussein Sawwan-Oracle
          You need to clean FND_NODES table and run AutoConfig after changing the IP Address -- How to Clean Nonexistent Nodes or IP Addresses From FND_NODES [ID 260887.1]
          /u02/E-BIZ/inst/apps/VISION_ip-10-116-174-104/logs/ora/10.1.3/opmn/HTTP_Server~1.log
          Any errors in this log file?

          Thanks,
          Hussein
          • 2. Re: EBS R12.1.3 HTTP is down.
            karthiksingh_dba
            Hi Hussein,

            Thanks for your response.

            I executed the select query and it displays the old one it seems.

            Node Name M C W A F
            ------------------------------ - - - - -
            IP-10-98-95-193 O N N N N -> DB NODE
            IP-10-32-141-136 O Y Y Y Y -> OLD APPS NODE
            AUTHENTICATION O N N N N
            IP-10-116-174-104 O Y Y Y Y -> NEW APPS NODE

            So Now I have to execute the FND_CLEAN from DB Tier or APPS Tier?

            Thanks,
            Karthik
            • 3. Re: EBS R12.1.3 HTTP is down.
              karthiksingh_dba
              /u02/E-BIZ/inst/apps/VISION_ip-10-116-174-104/logs/ora/10.1.3/opmn/HTTP_Server~1.log
              Any errors in this log file?
              13/05/24 14:58:27 Start process
              /u02/E-BIZ/inst/apps/VISION_ip-10-116-174-104/ora/10.1.3/Apache/Apache/bin/apachectl startssl: execing httpd
              13/05/24 14:58:29 Start process
              /u02/E-BIZ/inst/apps/VISION_ip-10-116-174-104/ora/10.1.3/Apache/Apache/bin/apachectl startssl: execing httpd

              I need to execute the FND_CLEAN as APPS or SYSTEM User?
              • 4. Re: EBS R12.1.3 HTTP is down.
                Hussein Sawwan-Oracle
                I need to execute the FND_CLEAN as APPS or SYSTEM User?
                As apps user (it doesn't matter on what node) then run AutoConfig on the DB tier node followed by the Apps tier node.

                Do you have SSL configured?

                Thanks,
                Hussein
                • 5. Re: EBS R12.1.3 HTTP is down.
                  karthiksingh_dba
                  As apps user (it doesn't matter on what node) then run AutoConfig on the DB tier node followed by the Apps tier node.
                  Okay, let me run through APPS user from DBTIER itself and update you.
                  Do you have SSL configured?
                  No.
                  Thanks,
                  Karthik
                  • 6. Re: EBS R12.1.3 HTTP is down.
                    karthiksingh_dba
                    Hi,

                    I ran as per the doc, but no use. Auto Config ran on DB and Apps Successfully. Still I'm not able to see the HTTP UP.

                    After FND Clean

                    Node Name M C W A F
                    ------------------------------ - - - - -
                    IP-10-98-95-193 O N N N N
                    IP-10-116-174-104 O Y Y Y Y
                    AUTHENTICATION O N N N N

                    FYI Changes did in context.xml file

                    <externURL oa_var="s_external_url">http://ip-10-116-174-104.ec2.internal:80</externURL>
                    Change the above line to
                    <externURL oa_var="s_external_url">http://ip-10-116-174-104.ec2.internal:8000</externURL>

                    <chronosURL oa_var="s_chronosURL">http://ip-10-116-174-104.ec2.internal:80/oracle_smp_chronos/oracle_smp_chronos_sdk.gif</chronosURL>
                    Changed to
                    <chronosURL oa_var="s_chronosURL">http://ip-10-116-174-104.ec2.internal:8000/oracle_smp_chronos/oracle_smp_chronos_sdk.gif</chronosURL>

                    <EndUserMonitoringURL oa_var="s_endUserMonitoringURL">http://ip-10-116-174-104.ec2.internal:80/oracle_smp_chronos/oracle_smp_chronos_sdk.gif</EndUserMonitoringURL>
                    Changed to
                    <EndUserMonitoringURL oa_var="s_endUserMonitoringURL">http://ip-10-116-174-104.ec2.internal:8000/oracle_smp_chronos/oracle_smp_chronos_sdk.gif</EndUserMonitoringURL>

                    <webentryhost oa_var="s_webentryhost">ip-10-116-174-104</webentryhost>
                    <webentrydomain oa_var="s_webentrydomain">ec2.internal</webentrydomain>
                    Changed to
                    <webentryhost oa_var="s_webentryhost">ec2-10-20-30-40</webentryhost>
                    <webentrydomain oa_var="s_webentrydomain">compute-1.amazonaws.com</webentrydomain>

                    <login_page oa_var="s_login_page">http://ec2-10-20-30-40.compute-1.amazonaws.com/OA_HTML/AppsLogin</login_page>
                    <httplistenparameter oa_var="s_http_listen_parameter">8000</httplistenparameter>
                    <httpslistenparameter oa_var="s_https_listen_parameter">4443</httpslistenparameter>
                    Changed to
                    <login_page oa_var="s_login_page">http://ec2-10-20-30-40.compute-1.amazonaws.com/OA_HTML/AppsLogin</login_page>
                    <httplistenparameter oa_var="s_http_listen_parameter">80</httplistenparameter>
                    <httpslistenparameter oa_var="s_https_listen_parameter">443</httpslistenparameter>

                    If I leave these things remain unchanged then I could see the status of HTTP is UP and all services are also UP, but when I browse through the PUBLIC DNS URL, I could not connect.

                    Also Just now I have tried the following thing

                    <login_page oa_var="s_login_page">http://ec2-10-20-30-40.compute-1.amazonaws.com/OA_HTML/AppsLogin</login_page>
                    <httplistenparameter oa_var="s_http_listen_parameter">8000</httplistenparameter>
                    <httpslistenparameter oa_var="s_https_listen_parameter">4443</httpslistenparameter>

                    and check the opmnctl status

                    Processes in Instance: VISION_ip-10-116-174-104.ip-10-116-174-104.ec2.internal
                    --------------------------------------------------------------+---------
                    ias-component | process-type | pid | status
                    --------------------------------------------------------------+---------
                    OC4JGroup:default_group | OC4J:oafm | 24838 | Alive
                    OC4JGroup:default_group | OC4J:forms | 24769 | Alive
                    OC4JGroup:default_group | OC4J:oacore | 24623 | Alive
                    HTTP_Server | HTTP_Server | 24566 | Alive

                    Everything is UP, but I'm not able to connect to EBS Login Page. Please help me I'm breaking my heads. Thanks in advance.

                    Thanks,
                    Karthik

                    Edited by: karthiksingh_dba on May 24, 2013 1:32 PM
                    • 7. Re: EBS R12.1.3 HTTP is down.
                      Hussein Sawwan-Oracle
                      What is the error when accessing the main login page?

                      What is the error you get in Apache log files this time?

                      Thanks,
                      Hussein
                      • 8. Re: EBS R12.1.3 HTTP is down.
                        karthiksingh_dba
                        What is the error when accessing the main login page?
                        Unable to connect or The page cannot be displayed.
                        What is the error you get in Apache log files this time?
                        May I know the path for which Apache logs you are referring?

                        Thanks,
                        Karthik
                        • 9. Re: EBS R12.1.3 HTTP is down.
                          Hussein Sawwan-Oracle
                          May I know the path for which Apache logs you are referring?
                          access_log* and error_log* under $LOG_HOME/ora/10.1.3/Apache directory.

                          Please also check logs under $LOG_HOME/ora/10.1.3/j2ee/forms/forms_<default_group_1>

                          Thanks,
                          Hussein
                          • 10. Re: EBS R12.1.3 HTTP is down.
                            karthiksingh_dba
                            Hi,

                            You mean the Apache logs from this location: $LOG_HOME/ora/10.1.3/Apache?

                            Under that we have
                            error_log.1369353600
                            access_log.1369353600

                            In error_log

                            [Fri May 24 16:31:23 2013] [notice] configured -- resuming normal operations
                            [Fri May 24 16:31:23 2013] [notice] Accept mutex: fcntl (Default: fcntl)
                            [Fri May 24 16:38:24 2013] [error] [client 203.196.190.74] [ecid: 1369427904:10.116.174.104:24583:0:22,0] File does not exist: /u02/E-BIZ/inst/apps/VISION_ip-10-116-174-104/portal/favicon.ico
                            [Fri May 24 17:04:55 2013] [notice] configured -- resuming normal operations
                            [Fri May 24 17:04:55 2013] [notice] Accept mutex: fcntl (Default: fcntl)

                            Or You mean any other Apache log location?
                            • 11. Re: EBS R12.1.3 HTTP is down.
                              karthiksingh_dba
                              access_log* and error_log* under $LOG_HOME/ora/10.1.3/Apache directory.
                              access_log* provides
                              127.0.0.1 - - [24/May/2013:17:42:55 -0400] "HEAD /index.html HTTP/1.1" 200 0 0 "-" "-"
                              127.0.0.1 - - [24/May/2013:17:43:15 -0400] "HEAD /index.html HTTP/1.1" 200 0 0 "-" "-"
                              127.0.0.1 - - [24/May/2013:17:43:35 -0400] "HEAD /index.html HTTP/1.1" 200 0 0 "-" "-"
                              127.0.0.1 - - [24/May/2013:17:43:55 -0400] "HEAD /index.html HTTP/1.1" 200 0 0 "-" "-"
                              127.0.0.1 - - [24/May/2013:17:44:15 -0400] "HEAD /index.html HTTP/1.1" 200 0 0 "-" "-"
                              Please also check logs under $LOG_HOME/ora/10.1.3/j2ee/forms/forms_<default_group_1>
                              Under this providing the last few lines from the logs.
                              SERVER.LOG
                              13/05/24 15:51:53.213 10.1.3.5.0 Started
                              13/05/24 15:51:55.624 10.1.3.5.0 Started
                              13/05/24 16:07:08.58 10.1.3.5.0 Stopped (Shutdown executed by oc4jadmin from 127.0.0.1 (localhost.localdomain))
                              13/05/24 16:10:28.326 10.1.3.5.0 Started
                              13/05/24 16:10:31.44 10.1.3.5.0 Started
                              13/05/24 16:28:46.504 10.1.3.5.0 Stopped (Shutdown executed by oc4jadmin from 127.0.0.1 (localhost.localdomain))
                              13/05/24 16:31:42.901 10.1.3.5.0 Started
                              13/05/24 16:31:45.461 10.1.3.5.0 Started
                              13/05/24 17:01:44.877 10.1.3.5.0 Stopped (Shutdown executed by oc4jadmin from 127.0.0.1 (localhost.localdomain))
                              13/05/24 17:05:15.213 10.1.3.5.0 Started
                              13/05/24 17:05:17.955 10.1.3.5.0 Started

                              SYSTEM-APPLICATION.LOG
                              13/05/24 15:51:53.753 10.1.3.5.0 Started
                              13/05/24 16:10:28.868 10.1.3.5.0 Started
                              13/05/24 16:31:43.428 10.1.3.5.0 Started
                              13/05/24 17:05:15.862 10.1.3.5.0 Started

                              RMI.LOG
                              13/05/24 16:10:28.824 10.1.3.5.0 Started
                              13/05/24 16:28:46.391 10.1.3.5.0 Stopped (Shutdown executed by oc4jadmin from 127.0.0.1 (localhost.localdomain))
                              13/05/24 16:31:43.383 10.1.3.5.0 Started
                              13/05/24 17:01:44.707 10.1.3.5.0 Stopped (Shutdown executed by oc4jadmin from 127.0.0.1 (localhost.localdomain))
                              13/05/24 17:05:15.818 10.1.3.5.0 Started

                              JMS.LOG
                              13/05/24 16:10:28.639 10.1.3.5.0 Started
                              13/05/24 16:28:46.498 10.1.3.5.0 Stopped (Shutdown executed by oc4jadmin from 127.0.0.1 (localhost.localdomain))
                              13/05/24 16:31:43.156 10.1.3.5.0 Started
                              13/05/24 17:01:44.872 10.1.3.5.0 Stopped (Shutdown executed by oc4jadmin from 127.0.0.1 (localhost.localdomain))
                              13/05/24 17:05:15.474 10.1.3.5.0 Started

                              GLOBAL-APPLICATION.LOG
                              13/05/24 16:31:44.226 10.1.3.5.0 Started
                              13/05/24 16:31:45.818 defaultWebApp: 10.1.3.5.0 Started
                              13/05/24 16:31:45.880 jmsrouter_web: 10.1.3.5.0 Started
                              13/05/24 17:01:44.415 defaultWebApp: 10.1.3.5.0 Stopped
                              13/05/24 17:01:44.501 jmsrouter_web: 10.1.3.5.0 Stopped
                              13/05/24 17:05:16.793 10.1.3.5.0 Started
                              13/05/24 17:05:18.172 defaultWebApp: 10.1.3.5.0 Started
                              13/05/24 17:05:18.227 jmsrouter_web: 10.1.3.5.0 Started

                              APPLICATION.LOG
                              13/05/24 16:31:45.190 10.1.3.5.0 Started
                              13/05/24 16:31:45.977 formsweb: FormsServlet init():
                              configFileName: /u02/E-BIZ/inst/apps/VISION_ip-10-116-174-104/ora/10.1.2/forms/server/appsweb.cfg
                              testMode: false
                              13/05/24 16:31:45.986 formsweb: 10.1.3.5.0 Started
                              13/05/24 17:01:44.506 formsweb: 10.1.3.5.0 Stopped
                              13/05/24 17:05:17.720 10.1.3.5.0 Started
                              13/05/24 17:05:18.328 formsweb: FormsServlet init():
                              configFileName: /u02/E-BIZ/inst/apps/VISION_ip-10-116-174-104/ora/10.1.2/forms/server/appsweb.cfg
                              testMode: false
                              13/05/24 17:05:18.341 formsweb: 10.1.3.5.0 Started

                              In the LOG.XML
                              <PAYLOAD>
                              <MSG_TEXT>Internal error raised tyring to instantiate web-application: ascontrol defined in web site OC4J 10g (10.1.3) Default Web Site. Application: ascontrol does not exist. Error creating Web application: ascontrol</MSG_TEXT>
                              </PAYLOAD>

                              I think I should put an entry in /etc/hosts like ip-10-116-174-104.ec2.internal, if this is the case I cannot reboot the system because If I reboot it will provide another new IP because it is having DHCP. So without these setting or without reboot is there a FIX for this issue?
                              • 12. Re: EBS R12.1.3 HTTP is down.
                                Hussein Sawwan-Oracle
                                Please see if the solution in (R12: Cannot Login After Oracle Application Server 10.1.3.4 Upgrade [ID 973637.1]) helps.

                                Thanks,
                                Hussein
                                • 13. Re: EBS R12.1.3 HTTP is down.
                                  karthiksingh_dba
                                  Hi Hussein,
                                  Hussein Sawwan wrote:
                                  Please see if the solution in (R12: Cannot Login After Oracle Application Server 10.1.3.4 Upgrade [ID 973637.1]) helps.
                                  I verified these patches said here were already applied.

                                  Thanks,
                                  Karthik
                                  • 14. Re: EBS R12.1.3 HTTP is down.
                                    karthiksingh_dba
                                    Hi Hussein,

                                    Also I would suspect the PORTS, WEB ENTRY URL, LOGIN parameters in the context file. Because the Old settings were working fine, when it has the OLD IP. Now the change was NEW IP and NEW CONTEXT FILE, so I manually did some changes according to Amazon using our OLD Context file Backup.

                                    Regards,
                                    Karthik
                                    1 2 Previous Next