1 2 Previous Next 28 Replies Latest reply: Oct 26, 2012 9:41 AM by Hussein Sawwan-Oracle RSS

    Workflow error

    dkoracle
      Hi Team,

      No changes been done but suddenly workflow stopped running in Production today..Please help me

      Db : 11.2.0.2
      App: R12.1.3
      COMPONENT_TYPE                 COMPONENT_NAME                                                                   COMPONENT_STATUS               ERROR
      ------------------------------ -------------------------------------------------------------------------------- ------------------------------ -----------------------------------------------------------------
      WF_MAILER                      Workflow Notification Mailer                                                     STARTING
      WF_JAVA_AGENT_LISTENER         Workflow Java Error Agent Listener                                               RUNNING
      WF_JAVA_AGENT_LISTENER         Workflow Java Deferred Agent Listener                                            RUNNING
      WF_JAVA_AGENT_LISTENER         Web Services IN Agent                                                            STOPPED
      WF_JAVA_AGENT_LISTENER         WF_JMS_IN Listener(M4U)                                                          RUNNING
      WF_DOCUMENT_WEB_SERVICES       Web Services OUT Agent                                                           STOPPED
      WF_AGENT_LISTENER              ECX Inbound Agent Listener                                                       STOPPED
      WF_AGENT_LISTENER              Workflow Inbound Notifications Agent Listener                                    RUNNING
      WF_AGENT_LISTENER              Workflow Error Agent Listener                                                    RUNNING
      WF_AGENT_LISTENER              Workflow Deferred Notification Agent Listener                                    RUNNING
      WF_AGENT_LISTENER              Workflow Deferred Agent Listener                                                 RUNNING
      
      COMPONENT_TYPE                 COMPONENT_NAME                                                                   COMPONENT_STATUS               ERROR
      ------------------------------ -------------------------------------------------------------------------------- ------------------------------ -----------------------------------------------------------------
      WF_AGENT_LISTENER              ECX Transaction Agent Listener                                                   STOPPED
      WF_AGENT_LISTENER              Workflow Inbound JMS Agent Listener                                              STOPPED
      Log file :


      [Sep 5, 2012 9:12:01 AM CDT]:1346854321395:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:1798236383:58687:1346854310823:0:EXCEPTION:[SVC-GSM-WFMLRSVC-585462 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start()]:Successfully started the Component Monitor thread
      [Sep 5, 2012 9:12:01 AM CDT]:1346854321401:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[ComponentMonitor,5,main]:1798236383:58687:1346854321401:1:EXCEPTION:[SVC-GSM-WFMLRSVC-585462 : oracle.apps.fnd.cp.gsc.SvcComponentMonitor.performInit()]:Successfully initialized -> SvcComponentMonitor{Processor{mThread=Thread[ComponentMonitor,5,main],mCurrentLoopSleep=0,mMinLoopSleep=60,mMaxLoopSleep=0,mReadTimedOutFlag=false,mCurrentErrorCount=0,mMaxErrorCount=-2147483648,mErrorLoopSleep=60,mErrorFlag=false,mControlEvent=noEvent,mStopFlag=false,mSuspendFlag=false,mLastSuspendFlag=false},mCurrentLoopCount=4,mOnDemandLoopCount=5}
      [Sep 5, 2012 9:12:01 AM CDT]:1346854321403:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[ComponentMonitor,5,main]:1798236383:58687:1346854321401:1:EXCEPTION:[SVC-GSM-WFMLRSVC-585462 : oracle.apps.fnd.cp.gsc.Processor.run()]:Processor has successfully been initialized. Loop will now start.
      [Sep 5, 2012 9:12:01 AM CDT]:1346854321401:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[GSMQueueProcessor,5,main]:1798236383:58687:1346854321402:2:EXCEPTION:[SVC-GSM-WFMLRSVC-585462 : oracle.apps.fnd.cp.gsc.Processor.run()]:Processor has successfully been initialized. Loop will now start.
      [Sep 5, 2012 9:12:01 AM CDT]:1346854321401:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:1798236383:58687:1346854310823:0:EXCEPTION:[SVC-GSM-WFMLRSVC-585462 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start()]:Successfully started the state machine
      [Sep 5, 2012 9:12:01 AM CDT]:1346854321405:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:1798236383:58687:1346854310823:0:EXCEPTION:[SVC-GSM-WFMLRSVC-585462 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start()]:Successfully started the container -> SvcComponentContainer{mComponentControlWorkflowContext=[oracle.apps.fnd.wf.apps.WorkflowContext@5e222e $Revision: 120.14.12010000.18 $ extends [oracle.apps.fnd.common.Context@5e222e $Revision: 120.7.12010000.15 $  {sessionId=0x5e222e,conn=oracle.jdbc.driver.T4CConnection@12bf892,state=oracle.apps.fnd.common.StateStack@15fb38}] {isDedicated=false}],mComponentMonitor=SvcComponentMonitor{Processor{mThread=Thread[ComponentMonitor,5,main],mCurrentLoopSleep=0,mMinLoopSleep=60,mMaxLoopSleep=0,mReadTi
      [Sep 5, 2012 9:12:01 AM CDT]:1346854321409:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[ComponentMonitor,5,main]:1798236383:58687:1346854321401:1:EXCEPTION:[SVC-GSM-WFMLRSVC-585462 : oracle.apps.fnd.cp.gsc.SvcComponentMonitor.startAutomaticComponents()]:Starting automatic component 10006
      [Sep 5, 2012 9:12:11 AM CDT]:1346854331367:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:1798236383:58687:1346854331367:3:EXCEPTION:[SVC-GSM-WFMLRSVC-585462 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.onBusinessEvent(BusinessEvent)]:(BusinessEvent{name=oracle.apps.fnd.cp.gsc.SvcComponent.start, key=SVC:05-SEP-2012, priority=50, correlationId=null, sendDate=Wed Sep 05 09:12:01 CDT 2012, receiveDate=null, From Agent:  , To Agent:  , Last Subscription=  , Error Message=null, Error Stack=null, CONTAINER_TYPE=GSM, CONTAINER_PROCESS_ID=585462, COMPONENT_ID=10006, BES_PAYLOAD_OBJECT=false, BES_FROM_AGENT=WF_CONTROL@PROD})
      [Sep 5, 2012 9:12:21 AM CDT]:1346854341375:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:1798236383:58687:1346854331367:3:EXCEPTION:[SVC-GSM-WFMLRSVC-585462 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.handleComponentEvent(int, String, String)]:Successfully retrieved component details from the database



      Thanks,
      dkoracle
        • 1. Re: Workflow error
          dkoracle
          Hi Team,

          Now its status is :

          Only WF_MAILER is not running
          COMPONENT_TYPE                 COMPONENT_NAME                                                                   COMPONENT_STATUS               ERROR
          ------------------------------ -------------------------------------------------------------------------------- ------------------------------ -----------------------------------------------------------------
          WF_MAILER                      Workflow Notification Mailer                                                     STARTING
          WF_JAVA_AGENT_LISTENER         Workflow Java Error Agent Listener                                               RUNNING
          WF_JAVA_AGENT_LISTENER         Workflow Java Deferred Agent Listener                                            RUNNING
          WF_JAVA_AGENT_LISTENER         Web Services IN Agent                                                            RUNNING
          WF_JAVA_AGENT_LISTENER         WF_JMS_IN Listener(M4U)                                                          RUNNING
          WF_DOCUMENT_WEB_SERVICES       Web Services OUT Agent                                                           RUNNING
          WF_AGENT_LISTENER              ECX Inbound Agent Listener                                                       RUNNING
          WF_AGENT_LISTENER              Workflow Inbound Notifications Agent Listener                                    RUNNING
          WF_AGENT_LISTENER              Workflow Error Agent Listener                                                    RUNNING
          WF_AGENT_LISTENER              Workflow Deferred Notification Agent Listener                                    RUNNING
          WF_AGENT_LISTENER              Workflow Deferred Agent Listener                                                 RUNNING
          
          COMPONENT_TYPE                 COMPONENT_NAME                                                                   COMPONENT_STATUS               ERROR
          ------------------------------ -------------------------------------------------------------------------------- ------------------------------ -----------------------------------------------------------------
          WF_AGENT_LISTENER              ECX Transaction Agent Listener                                                   RUNNING
          WF_AGENT_LISTENER              Workflow Inbound JMS Agent Listener                                              RUNNING
          
          13 rows selected.
          • 2. Re: Workflow error
            Srini Chavali-Oracle
            There could be many causes for this issue. Pl see the troubleshooting steps in these MOS Docs

            Notification Mailer Does Not Start, Remains In Status Starting [ID 371830.1]
            Workflow Mailer Will Not Start Stays In Starting Status [ID 837893.1]
            Workflow Mailer Remains in Starting Status but does not Start [ID 728015.1]
            Workflow Notification Mailer Remains in Starting Status [ID 1400353.1]

            HTH
            Srini
            • 3. Re: Workflow error
              Hussein Sawwan-Oracle
              Have you tried to bounce all the services and see if this helps (DB & Apps)?

              Does AutoConfig complete successfully?

              Can you find any errors in the database log file?

              Any errors at the OS level?

              Please run the diagnostics scripts in (Oracle Workflow Release 12 Diagnostics [ID 469822.1]) and see if it returns any errors.

              Thanks,
              Hussein
              • 4. Re: Workflow error
                dkoracle
                Hi Hussein/Srini,

                Just bought the permission to bounce PROD.. After bouncing this was the result
                COMPONENT_ID COMPONENT_NAME                                                                   COMPONENT_STATUS               COMPONENT_TYPE
                ------------ -------------------------------------------------------------------------------- ------------------------------ ------------------------------
                       10000 ECX Inbound Agent Listener                                                       STOPPED                        WF_AGENT_LISTENER
                       10001 ECX Transaction Agent Listener                                                   STOPPED                        WF_AGENT_LISTENER
                       10002 Workflow Deferred Agent Listener                                                 RUNNING                        WF_AGENT_LISTENER
                       10003 Workflow Deferred Notification Agent Listener                                    RUNNING                        WF_AGENT_LISTENER
                       10004 Workflow Error Agent Listener                                                    RUNNING                        WF_AGENT_LISTENER
                       10005 Workflow Inbound Notifications Agent Listener                                    RUNNING                        WF_AGENT_LISTENER
                       10006 Workflow Notification Mailer                                                     STARTING                       WF_MAILER
                       10040 WF_JMS_IN Listener(M4U)                                                          RUNNING                        WF_JAVA_AGENT_LISTENER
                       10021 Web Services OUT Agent                                                           STOPPED                        WF_DOCUMENT_WEB_SERVICES
                       10020 Web Services IN Agent                                                            STOPPED                        WF_JAVA_AGENT_LISTENER
                       10022 Workflow Java Deferred Agent Listener                                            RUNNING                        WF_JAVA_AGENT_LISTENER
                
                COMPONENT_ID COMPONENT_NAME                                                                   COMPONENT_STATUS               COMPONENT_TYPE
                ------------ -------------------------------------------------------------------------------- ------------------------------ ------------------------------
                       10023 Workflow Java Error Agent Listener                                               RUNNING                        WF_JAVA_AGENT_LISTENER
                       10041 Workflow Inbound JMS Agent Listener                                              STOPPED                        WF_AGENT_LISTENER
                • 5. Re: Workflow error
                  Hussein Sawwan-Oracle
                  Can you check the contents of the Workflow log file now?

                  What about the diagnostic scripts?

                  Thanks,
                  Hussein
                  • 6. Re: Workflow error
                    dkoracle
                    Here is the log file output
                    [Sep 5, 2012 12:20:45 PM CDT]:1346865645836:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:1798236383:66529:1346865645555:0:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:Successfully asked for BES control connection to be established (asynchronous)
                    [Sep 5, 2012 12:20:45 PM CDT]:1346865645884:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:1798236383:66529:1346865645555:0:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:Registered a subscription for oracle.apps.fnd.cp.gsc.SvcComponent.refresh
                    [Sep 5, 2012 12:20:45 PM CDT]:1346865645884:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:1798236383:66529:1346865645555:0:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:Registered a subscription for oracle.apps.fnd.cp.gsc.SvcComponent.resume
                    [Sep 5, 2012 12:20:45 PM CDT]:1346865645884:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:1798236383:66529:1346865645555:0:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:Registered a subscription for oracle.apps.fnd.cp.gsc.SvcComponent.start
                    [Sep 5, 2012 12:20:45 PM CDT]:1346865645884:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:1798236383:66529:1346865645555:0:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:Registered a subscription for oracle.apps.fnd.cp.gsc.SvcComponent.stop
                    [Sep 5, 2012 12:20:45 PM CDT]:1346865645884:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:1798236383:66529:1346865645555:0:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:Registered a subscription for oracle.apps.fnd.cp.gsc.SvcComponent.suspend
                    [Sep 5, 2012 12:20:45 PM CDT]:1346865645884:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:1798236383:66529:1346865645555:0:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:Registered a subscription for oracle.apps.fnd.wf.mailer.Mailer.notification.summary
                    [Sep 5, 2012 12:20:45 PM CDT]:1346865645885:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:1798236383:66529:1346865645555:0:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:Have waited 0 seconds for BES system to establish connection to the control queue.  Waiting another 2 seconds
                    [Sep 5, 2012 12:20:47 PM CDT]:1346865647887:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:1798236383:66529:1346865645555:0:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:Waited 2 seconds for BES system to establish connection to the control queue
                    [Sep 5, 2012 12:20:56 PM CDT]:1346865656283:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:1798236383:66529:1346865645555:0:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start()]:Successfully started BES listener
                    [Sep 5, 2012 12:20:56 PM CDT]:1346865656315:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:1798236383:66529:1346865645555:0:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start()]:Successfully reset component statuses for this container
                    [Sep 5, 2012 12:20:56 PM CDT]:1346865656318:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:1798236383:66529:1346865645555:0:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start()]:Successfully updated the container's database status, if any
                    [Sep 5, 2012 12:20:56 PM CDT]:1346865656319:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:1798236383:66529:1346865645555:0:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start()]:Successfully named container session (took logical lock)
                    [GC 22140K->7730K(63360K), 0.0179280 secs]
                    [Sep 5, 2012 12:20:56 PM CDT]:1346865656368:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:1798236383:66529:1346865645555:0:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start()]:Successfully started the Component Monitor thread
                    [Sep 5, 2012 12:20:56 PM CDT]:1346865656374:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[ComponentMonitor,5,main]:1798236383:66529:1346865656374:1:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentMonitor.performInit()]:Successfully initialized -> SvcComponentMonitor{Processor{mThread=Thread[ComponentMonitor,5,main],mCurrentLoopSleep=0,mMinLoopSleep=60,mMaxLoopSleep=0,mReadTimedOutFlag=false,mCurrentErrorCount=0,mMaxErrorCount=-2147483648,mErrorLoopSleep=60,mErrorFlag=false,mControlEvent=noEvent,mStopFlag=false,mSuspendFlag=false,mLastSuspendFlag=false},mCurrentLoopCount=4,mOnDemandLoopCount=5}
                    [Sep 5, 2012 12:20:56 PM CDT]:1346865656375:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[GSMQueueProcessor,5,main]:1798236383:66529:1346865656375:2:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.Processor.run()]:Processor has successfully been initialized.  Loop will now start.
                    [Sep 5, 2012 12:20:56 PM CDT]:1346865656374:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:1798236383:66529:1346865645555:0:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start()]:Successfully started the state machine
                    [Sep 5, 2012 12:20:56 PM CDT]:1346865656375:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[ComponentMonitor,5,main]:1798236383:66529:1346865656374:1:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.Processor.run()]:Processor has successfully been initialized.  Loop will now start.
                    [Sep 5, 2012 12:20:56 PM CDT]:1346865656378:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[main,5,main]:1798236383:66529:1346865645555:0:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.start()]:Successfully started the container -> SvcComponentContainer{mComponentControlWorkflowContext=[oracle.apps.fnd.wf.apps.WorkflowContext@7244ca $Revision: 120.14.12010000.18 $ extends [oracle.apps.fnd.common.Context@7244ca $Revision: 120.7.12010000.15 $  {sessionId=0x7244ca,conn=oracle.jdbc.driver.T4CConnection@10e35d5,state=oracle.apps.fnd.common.StateStack@23756}] {isDedicated=false}],mComponentMonitor=SvcComponentMonitor{Processor{mThread=Thread[ComponentMonitor,5,main],mCurrentLoopSleep=0,mMinLoopSleep=60,mMaxLoopSleep=0,mReadTi
                    [Sep 5, 2012 12:20:56 PM CDT]:1346865656413:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[ComponentMonitor,5,main]:1798236383:66529:1346865656374:1:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentMonitor.startAutomaticComponents()]:Starting automatic component 10006
                    [Sep 5, 2012 12:21:06 PM CDT]:1346865666327:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:1798236383:66529:1346865666328:3:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.onBusinessEvent(BusinessEvent)]:(BusinessEvent{name=oracle.apps.fnd.cp.gsc.SvcComponent.start, key=SVC:05-SEP-2012, priority=50, correlationId=null, sendDate=Wed Sep 05 12:20:56 CDT 2012, receiveDate=null, From Agent:  , To Agent:  , Last Subscription=  , Error Message=null, Error Stack=null, CONTAINER_TYPE=GSM, CONTAINER_PROCESS_ID=585489, COMPONENT_ID=10006, BES_PAYLOAD_OBJECT=false, BES_FROM_AGENT=WF_CONTROL@PROD})
                    [Sep 5, 2012 12:21:06 PM CDT]:1346865666332:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:1798236383:66529:1346865666328:3:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.handleComponentEvent(int, String, String)]:Successfully retrieved component details from the database
                    Will shortly post you the diag script report..

                    Thanks Hussien
                    • 7. Re: Workflow error
                      Hussein Sawwan-Oracle
                      I do not see any errors in the logs.
                      Will shortly post you the diag script report..
                      Thanks.

                      If this is a production issue, you can create a SR with severity 1

                      Regards,
                      Hussein
                      • 8. Re: Workflow error
                        dkoracle
                        Hussien i saw there are several steps provided for diagnostic tests please tell me do i have to follow all the steps ? Which are steps i need to spool the output for posting here
                        • 9. Re: Workflow error
                          Hussein Sawwan-Oracle
                          Only the ones that return error/warning messages.

                          Please make sure you run the scripts in (Diagnostic Test Catalog for 12.1.3 [ID 1083807.1]), under "Oracle Workflow Cartridge (WF)" section.

                          Thanks,
                          Hussein
                          • 10. Re: Workflow error
                            dkoracle
                            Hi Hussien,

                            SR guy gave these steps to perform



                            Please perform the following:

                            1. upload the output of $FND_TOP/sql/wfver from PROD instance.

                            2. Please execute the solution in this note:

                            Note 736898.1 How to Rebuild Mailer's Queue when it is Inconsistent or Corrupted ?

                            3. Bounce all apps 11i or R 12 processes (this SR did not indicate which apps release)
                            and bounce database to 1) clear up the network congestion and 2)kill the dead processes.






                            Regards,
                            Dkoracle
                            • 11. Re: Workflow error
                              Hussein Sawwan-Oracle
                              2. Please execute the solution in this note:

                              Note 736898.1 How to Rebuild Mailer's Queue when it is Inconsistent or Corrupted ?

                              3. Bounce all apps 11i or R 12 processes (this SR did not indicate which apps release)
                              and bounce database to 1) clear up the network congestion and 2)kill the dead processes.
                              Stop all application services (if possible) and verify if any exists at the OS level (using ps command) and kill any processes then start the application processes.

                              Thanks,
                              Hussein
                              • 12. Re: Workflow error
                                dkoracle
                                Hi Hussein,

                                After completely bounced.. I checked the service components from OAM
                                Some components had Startup mode="automatic" and status="running"
                                Some components had Startup mode='manual" and status="Stopped" ( Then i explicitly started one by one ) then status changed to "Running"

                                Now all turned to running status except "workflow notification mailer" it has Startup mode="Automatic" but Status="System Deactivated"

                                After that i captured the log file...



                                [Sep 5, 2012 2:39:13 PM CDT]:1346873953843:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:1798236383:66529:1346873821092:43:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentStateMachine.updateComponentStatus(Logger, Connection, int, String, String)]:BEGIN (Logger{mLog=[$Header: AppsLog.java 120.4.12010000.2 2009/06/17 15:14:37 rsantis ship $ @18092261 {oracle.apps.fnd.profiles.ExtendedProfileStore@61373f}],mUniqueId=SVC-GSM-WFMLRSVC-585489,mLevel=4}, oracle.jdbc.driver.T4CConnection@10e35d5, 10006, STOPPED_ERROR, Could not handle the control event -> oracle.apps.fnd.cp.gsc.SvcComponentContainerException: Could not start component; performing rollback -> oracle.apps.fnd.cp.gsc.SvcComponentException: Vali
                                [Sep 5, 2012 2:40:01 PM CDT]:1346874001101:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[ComponentMonitor,5,main]:1798236383:66529:1346865656374:1:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentMonitor.startAutomaticComponents()]:Starting automatic component 10006
                                [Sep 5, 2012 2:40:01 PM CDT]:1346874001193:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:1798236383:66529:1346874001193:44:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.onBusinessEvent(BusinessEvent)]:(BusinessEvent{name=oracle.apps.fnd.cp.gsc.SvcComponent.start, key=SVC:05-SEP-2012, priority=50, correlationId=null, sendDate=Wed Sep 05 14:40:01 CDT 2012, receiveDate=null, From Agent: , To Agent: , Last Subscription= , Error Message=null, Error Stack=null, CONTAINER_TYPE=GSM, CONTAINER_PROCESS_ID=585489, COMPONENT_ID=10006, BES_PAYLOAD_OBJECT=false, BES_FROM_AGENT=WF_CONTROL@PROD})
                                [Sep 5, 2012 2:40:11 PM CDT]:1346874011199:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:1798236383:66529:1346874001193:44:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.handleComponentEvent(int, String, String)]:Successfully retrieved component details from the database
                                [Sep 5, 2012 2:42:13 PM CDT]:1346874133920:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:1798236383:66529:1346874001193:44:ERROR:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.handleComponentEvent(int, String, String)]:Could not start component; performing rollback -> oracle.apps.fnd.cp.gsc.SvcComponentException: Validation failed for the following parameters -> {ACCOUNT=Unable to connect to the mail account. Check the host, user name and password.}. Parameters were -> {RESET_NLS=N, FROM=Workflow Mailer PROD, INBOUND_MAX_RET_EMAIL_SIZE=100, INBOUND_SSL_ENABLED=Y, CANCELED=WFMAIL:CANCELED, OUTBOUND_CONNECTION_TIMEOUT=120, SEND_UNSOLICITED_WARNING=Y, CLOSED=WFMAIL:CLOSED, NODENAME=WFMAIL, OUTBOUND_SER
                                [Sep 5, 2012 2:42:13 PM CDT]:1346874133920:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:1798236383:66529:1346874001193:44:ERROR:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.onBusinessEvent(BusinessEvent)]:Error in handling component event, oracle.apps.fnd.cp.gsc.SvcComponent.start, for component 10006
                                [Sep 5, 2012 2:42:13 PM CDT]:1346874133920:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:1798236383:66529:1346874001193:44:ERROR:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.onBusinessEvent(BusinessEvent)]:oracle.apps.fnd.cp.gsc.SvcComponentContainerException: Could not start component; performing rollback -> oracle.apps.fnd.cp.gsc.SvcComponentException: Validation failed for the following parameters -> {ACCOUNT=Unable to connect to the mail account. Check the host, user name and password.}. Parameters were -> {RESET_NLS=N, FROM=Workflow Mailer PROD, INBOUND_MAX_RET_EMAIL_SIZE=100, INBOUND_SSL_ENABLED=Y, CANCELED=WFMAIL:CANCELED, OUTBOUND_CONNECTION_TIMEOUT=120, SEND_UNSOLICITED_WARNING=Y, CLOSED=
                                     at oracle.apps.fnd.cp.gsc.SvcComponentContainer.handleComponentEvent(SvcComponentContainer.java:2221)
                                     at oracle.apps.fnd.cp.gsc.SvcComponentContainer.onBusinessEvent(SvcComponentContainer.java:301)
                                     at oracle.apps.fnd.wf.bes.DispatchThread.run(DispatchThread.java:57)

                                [Sep 5, 2012 2:42:13 PM CDT]:1346874133921:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:1798236383:66529:1346874001193:44:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentStateMachine.updateComponentStatus(Logger, Connection, int, String, String)]:BEGIN (Logger{mLog=[$Header: AppsLog.java 120.4.12010000.2 2009/06/17 15:14:37 rsantis ship $ @18092261 {oracle.apps.fnd.profiles.ExtendedProfileStore@61373f}],mUniqueId=SVC-GSM-WFMLRSVC-585489,mLevel=4}, oracle.jdbc.driver.T4CConnection@10e35d5, 10006, STOPPED_ERROR, Could not handle the control event -> oracle.apps.fnd.cp.gsc.SvcComponentContainerException: Could not start component; performing rollback -> oracle.apps.fnd.cp.gsc.SvcComponentException: Vali
                                [Sep 5, 2012 2:43:01 PM CDT]:1346874181155:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[ComponentMonitor,5,main]:1798236383:66529:1346865656374:1:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentMonitor.startAutomaticComponents()]:Starting automatic component 10006
                                [Sep 5, 2012 2:43:01 PM CDT]:1346874181284:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:1798236383:66529:1346874181284:45:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.onBusinessEvent(BusinessEvent)]:(BusinessEvent{name=oracle.apps.fnd.cp.gsc.SvcComponent.start, key=SVC:05-SEP-2012, priority=50, correlationId=null, sendDate=Wed Sep 05 14:43:01 CDT 2012, receiveDate=null, From Agent: , To Agent: , Last Subscription= , Error Message=null, Error Stack=null, CONTAINER_TYPE=GSM, CONTAINER_PROCESS_ID=585489, COMPONENT_ID=10006, BES_PAYLOAD_OBJECT=false, BES_FROM_AGENT=WF_CONTROL@PROD})
                                [Sep 5, 2012 2:43:11 PM CDT]:1346874191290:-1:-1:fcpgapps01.ameron.com:192.168.40.100:-1:-1:1:20420:SYSADMIN(0):-1:Thread[BES Dispatch Thread,5,main]:1798236383:66529:1346874181284:45:EXCEPTION:[SVC-GSM-WFMLRSVC-585489 : oracle.apps.fnd.cp.gsc.SvcComponentContainer.handleComponentEvent(int, String, String)]:Successfully retrieved component details from the database

                                Edited by: dkoracle on Sep 6, 2012 1:24 AM
                                • 13. Re: Workflow error
                                  Hussein Sawwan-Oracle
                                  Please see these docs.

                                  Unable To Connect To The Mail Account. Check The Host, User Name And Password [ID 1123684.1]
                                  Configurnig IMAP4 For Workflow Mailer Errors: Unable to connect to the mail account. Check the host, user name and password. [ID 1081089.1]
                                  Workflow Java Mailer & Notifications FAQ [ID 562551.1]
                                  How to Troubleshoot 'Validation failed' Errors when Starting the Mailer [ID 463354.1]
                                  Unable To Make A Network Connection For IMAP Username/Password In Workflow Mailer Configuration [ID 784247.1]
                                  Mailer cannot Connect to IMAP Server on Oracle Applications R12.1.1 [ID 1300542.1]
                                  Cannot Start Workflow Notification Mailer. 'System Deactivated' [ID 602624.1]
                                  Java Notification Mailer configuration fails on EMail Servers configuration page [ID 1239213.1]

                                  Please see these old threads since the same issue/fix was reported many times in the forum before -- https://forums.oracle.com/forums/search.jspa?threadID=&q=%22Unable+to+connect+to+the+mail+account%22&objID=c3&dateRange=all&userID=&numResults=15&rankBy=10001

                                  Thanks,
                                  Hussein
                                  • 14. Re: Workflow error
                                    dkoracle
                                    Hi Hussein,

                                    I didnt get any update from oracle support.. This has been resolved now

                                    Surfed some oracle notes in mysupport and realised that in my environment recommended value for job_queue_processes was 10 but current value was just 2
                                    Completely stopped services on both Apps and DB node by issuing "kill -9 -1" later on edited in pfile given the value as 10.

                                    Started the services and once again provided the account details for imap inbound server then bounced complete mailer service. It was working fine till now

                                    But please help me to understand how job_queue_process parameter linked with Notification mailer. Also please provide me some metalink note to understand the architecture of this workflow. I really understand there are lot of challenges involved here need to know about workflow configurations, setups and troubleshooting technique tips
                                    Need your guidance to move in right way..

                                    Thanks hussien for your support


                                    dkoracle
                                    1 2 Previous Next