1 2 Previous Next 16 Replies Latest reply on May 6, 2015 12:38 PM by GilmarLima

    Oracle BI Scheduler Service is stopping

    Deepak16

      Hi All,

       

      I have installed OBIEE 10g ( vesion : 10.1.3.4.1 ) on Microsoft Windows server 2003.

      I am facing issue with Oracle BI Scheduler service when particular ibot is running.

      unable to find out root cause for this as error log is not getting crated.

      Can anyone please guide me how to identify root cause for this.

       

      Thanks.

        • 1. Re: Oracle BI Scheduler Service is stopping
          kt1

          I Would look in the following log files.

           

          1.nqserver.log

          2.nqscheuler.log

          3.saw.log

          • 2. Re: Oracle BI Scheduler Service is stopping
            GilmarLima

            Hello Guys,

             

            Same thing is happening to me for a few weeks now.

            Basically I have to restart OBIEE everytime a scheduled report runs... evertytime!

             

            I have just 3 reports running BI Publisher, and they are super simple. The datamodel has 2 selects(select * from...) and it has one parameter.

             

            I'm not using burst, but I do have it scheduled to be delivered by email.

             

            The odd part is that it runs successfully the first one and deliver all the others get stuck or fail, or both and only after restart the failure email is sent.

             

            Here is the the email error message:

             

            Job Name : Report Title march 25th

            Job Executed Date : March 25, 2015 1:36:32 PM WET Status : Job was failed.

             

            View History : http://xxxxxxx

            Detail Message:

            ::RIPPLE_OUTPUT_FAILED::[INSTANCE_ID=INSTANCE_NAME.1419863589585] [OUTPUT_ID=1122] ::DOCUMENT_GENERATION_FAILED::[INSTANCE_ID=INSTANCE_NAME.1419863589585] [OUTPUT_ID=1122][ReportProcessor]Error rendering documentoracle.xdo.servlet.scheduler.ProcessingException: [ReportProcessor]Error rendering document

                            at oracle.xdo.enterpriseScheduler.bursting.BurstingReportProcessor.renderReport(BurstingReportProcessor.java:445)

                            at oracle.xdo.enterpriseScheduler.bursting.BurstingReportProcessor.onMessage(BurstingReportProcessor.java:127)

                            at oracle.xdo.enterpriseScheduler.util.Checkpoin

             

            Please any help is greatly appreciated.

            • 3. Re: Oracle BI Scheduler Service is stopping
              GilmarLima

              Also I have this log error, but it is sanscrit to me... and when I look it up it points me to a problem on Bursting but it is not even enabled on the Reports Properties.

              LOG: bi_server1-stdout-obiee_error_log:

              #

              # ROTATION_TYPE = TIME

              # TIME_INTERVAL_MINS = 1440

              #

               

               

              <Mar 25, 2015 12:00:48 AM BRT> <Error> <Diagnostics> <BEA-320142> <An error was encountered while performing size based data retirement on archive EventsDataArchive

              weblogic.diagnostics.accessor.DiagnosticDataAccessException: weblogic.store.PersistentStoreException: weblogic.store.PersistentStoreException: [Store:280029]The persistent store record 15,920 could not be found

                at weblogic.diagnostics.archive.wlstore.PersistentStoreDataArchive.deleteDataRecords(PersistentStoreDataArchive.java:1368)

                at weblogic.diagnostics.archive.wlstore.PersistentStoreDataArchive.retireOldestRecords(PersistentStoreDataArchive.java:1211)

                at weblogic.diagnostics.archive.DataRetirementByQuotaTaskImpl.performDataRetirement(DataRetirementByQuotaTaskImpl.java:92)

                at weblogic.diagnostics.archive.DataRetirementByQuotaTaskImpl.run(DataRetirementByQuotaTaskImpl.java:49)

                at weblogic.diagnostics.archive.DataRetirementTaskImpl.run(DataRetirementTaskImpl.java:261)

                Truncated. see log file for complete stacktrace

              Caused By: weblogic.store.PersistentStoreException: weblogic.store.PersistentStoreException: [Store:280029]The persistent store record 15,920 could not be found

                at weblogic.diagnostics.archive.wlstore.PersistentStoreDataArchive.readRecord(PersistentStoreDataArchive.java:698)

                at weblogic.diagnostics.archive.wlstore.PersistentStoreDataArchive.readRecord(PersistentStoreDataArchive.java:668)

                at weblogic.diagnostics.archive.wlstore.PersistentStoreDataArchive.getWrapper(PersistentStoreDataArchive.java:1767)

                at weblogic.diagnostics.archive.wlstore.PersistentStoreDataArchive.removeGarbageInPage(PersistentStoreDataArchive.java:1813)

                at weblogic.diagnostics.archive.wlstore.PersistentStoreDataArchive.cleanupPages(PersistentStoreDataArchive.java:1697)

                Truncated. see log file for complete stacktrace

              Caused By: weblogic.store.PersistentStoreException: [Store:280029]The persistent store record 15,920 could not be found

                at weblogic.store.io.file.FileStoreIO$TypeRecord.getSlot(FileStoreIO.java:1064)

                at weblogic.store.io.file.FileStoreIO.readInternal(FileStoreIO.java:244)

                at weblogic.store.io.file.FileStoreIO.read(FileStoreIO.java:235)

                at weblogic.store.internal.ReadRequest.run(ReadRequest.java:34)

                at weblogic.store.internal.StoreRequest.doTheIO(StoreRequest.java:64)

                Truncated. see log file for complete stacktrace

              >

              • 4. Re: Oracle BI Scheduler Service is stopping
                TomBlakeley

                Hi,

                 

                Always best to start a new thread with questions -

                 

                so please do so.

                 

                In the meantime - check out the following doc:

                BI Publisher Bursting Report Job Fails with [Error rendering documentoracle.xdo.servlet.scheduler.ProcessingException: [ReportProcessor]Error rendering document] (Doc ID 1636180.1)

                • 5. Re: Oracle BI Scheduler Service is stopping
                  GilmarLima

                  Hello Tom

                   

                  Just didn't want to create a topic of a topic already created.

                   

                  I have tried to open the link, but it takes me to an Oracle Support Registration page (my company has an EWL) and I dont know where to get the Support Identifier it requests.

                   

                  But does this doc asks to follow the procedure below? If so I have tried... and I have basically all outputs available and bursting is deactivated from the report.

                  Also just by running the report OBI Publisher stalls and I can't navigate to other pages of.

                  <Found online serach for: BI Publisher Bursting Report Job Fails with [Error rendering documentoracle.xdo.servlet.scheduler.ProcessingException: >

                  1. Login to servername:port/xmlpserver.
                  2. Go to the folder from where you click on the 'Open' for the report.
                  3. This time instead of Clicking OPEN, click on Edit.
                  4. This brings to a screen where you see templates defined for your report.
                  5. On theRight Hand side, one will 'View Thumbnails' and 'View s List'. Click on 'View aList'.
                  6. Please make sure Excel output is part of outputs enabled for the template and also is the default output.
                  • 6. Re: Oracle BI Scheduler Service is stopping
                    TomBlakeley

                    Can you make sure that the excel MHTML option is included?

                    • 7. Re: Oracle BI Scheduler Service is stopping
                      GilmarLima

                      Yes, actually is it quite odd, because after running any report (scheduled or not)  basically all services goes down and obiee gets stuck, once the report finishes (when it does finishes) then I can navigate again. See below  the state of the service:

                      mhtml.jpg

                      Service after running a report on report viewer:

                      obidown.jpg

                      • 8. Re: Oracle BI Scheduler Service is stopping
                        GilmarLima

                        One more thing is that the EAS service never starts now. This happend last friday night probably and since then the server went crazy.obidown_eas.jpg

                        • 9. Re: Oracle BI Scheduler Service is stopping
                          TomBlakeley

                          Sorry I was not clear!

                           

                          Could you go into the Layout Output Format and then where you select the Output Formats make sure that Excel (mhtml) is checked?

                          • 10. Re: Oracle BI Scheduler Service is stopping
                            GilmarLima

                            Oh ok, I have both MHTML and Excel (mhtml) outputs enabled, even though on the scheduled job I choose the Excel (xlsx).

                            Funny thing is that other report I have (a simple one that is basically an extract) that i sent every morning just to make sure the server is up and running runs fine. only this report has gone cuckoo :/

                            This report (the damaged one) is very chart heavy and shows a long timeline, it shows basically every 2 hours for 2 months (about 720 points) per chart, there are 5 charts alike.

                             

                            workload_layout.jpg

                            • 11. Re: Oracle BI Scheduler Service is stopping
                              GilmarLima

                              So Tom, thank you very much for your help

                               

                              It was a simple report at first but like I said now it plots over 720 points per chart and there are about 6 to 7 charts on each report I think it is too much for the server, I have now restricted the data to 2 weeks worth of data which runs fine.

                               

                              But shouldnt it be able to work with a huge ammount of data like this?

                              • 12. Re: Oracle BI Scheduler Service is stopping
                                TomBlakeley

                                Okay, so it seems like you may have a memory issue?

                                 

                                Are you able to watch what happens on the box while running the report? Does memory spike? and then services crash?

                                • 13. Re: Oracle BI Scheduler Service is stopping
                                  GilmarLima

                                  Hello Tom,

                                   

                                  Yes I just did it. I claimed victory too soon yesterday

                                  It didnt run tonight complaning about not being able to logon to the BI Server...

                                  Anywhoo... So two layouts of the same report runs smoothly, but one is still giving me the same rendering error from time to time.

                                  I opened the server monitoring to check the processed and server performance (please see below). My server has 8GB Ram with a Intel XEON 2.2GHZ.

                                  I have Oracle DB running on this server as well so it has some apps competing for resources, but it doesn't look like it should be faulting like that.

                                   

                                  I took this screen shot, the spike between 3:51 and 3:56 is the faulty layout running (without errors this time). Is this too much resource for such a simple report?

                                  It's data model has 3 queries, like I said they are pretty simple and direct queries no joins.

                                  obiee_perfomance.jpg

                                  • 14. Re: Oracle BI Scheduler Service is stopping
                                    TomBlakeley

                                    Lets run a test.

                                     

                                    Can you take the SQL query and execute it directly against the underlying database? Watch the memory usage when the query is executing.

                                     

                                    8GB is a little on the smaller side at first look.

                                    1 2 Previous Next