1 2 Previous Next 18 Replies Latest reply: Jun 9, 2014 2:57 AM by Srinath Menon-Oracle RSS

    Contribution Folder Link Not Working!!

    ManojC

      Hi,

       

        In one of the stage environment the contribution folder link is not working. Once I click on the Browse Content --> Contribution Folder link the content server page was spinning for few minutes and returns the error(Unable to execute service COLLECTION_DISPLAY and function loadContentInCollection and ORA-01013: user requested cancel of current operation ). I am not canceling the operation.

      This issue is really sporadic when sometimes I am able to view the folders under contribution folder, but not able to access any child folder. Do I have to create any indices for the db tables. Please advise me how to fix this issue.

       

      Environment : 10.1.3.5.1 (100623) (Build:7.2.4.74)

      Folder_g : 2008_07_25 (build 24) ($Rev: 65025 $)

       

      Query used(When I click on Contribution Folder link) :

      SELECT Documents.dID,dRevisionID,dWebExtension,dRendition1, <Custom Metadata's>,dDocName, dDocTitle, dDocAuthor, dDocType, dSecurityGroup, dRevLabel, dDocAccount, dCreateDate, dReleaseDate, dInDate, dOutDate, dIsCheckedOut, dRevRank, dCheckoutUser, dStatus, dReleaseState, dProcessingState, dIndexerState, dPublishType, dPublishState, dOriginalName, dFormat, dFileSize, dExtension, dIsPrimary, dIsWebFormat, dDocID, dRevClassID FROM Documents,DocMeta,Revisions WHERE Documents.dID=Revisions.dID AND Documents.dID=DocMeta.dID AND dIsWebFormat=0 AND dIsPrimary<>0 AND ( ((DocMeta.xCollectionID=2))) AND NOT (dStatus='EXPIRED' OR dStatus='DELETED')

       

      Thanks


        • 1. Re: Contribution Folder Link Not Working!!
          Srinath Menon-Oracle

          Hi Manoj ,

           

          Most likely the query is not completing in a timely fashion , that is why the ORA error is showing up. Run the same query from SQL Developer and check if there are any performance improvement recommendations . Most likely you might need to add some indexes to improve the performance of this query .

           

          Thanks,

          Srinath

          • 2. Re: Contribution Folder Link Not Working!!
            William Phelps

            Also please check how many items are actually being returned by the query.  If this is Folders_g, there's a limit of 1000 items (it's somewhat hardwired into the system code) that can be inserted into a folder.  If you have somehow gotten around this intentional limitation, you might be seeing this issue because of bypassing the intended system limit.

            • 3. Re: Contribution Folder Link Not Working!!
              ManojC

              Hi,

               

                Thanks for your reply. As of now I see only 24 folder entries in Collections table in stage(I assume you are referring number of folders(1000) under contribution folder). Actually this issue is really sporadic, sometimes I am able to view the folders under contribution folder but couldn't view the child folders under that. I ran the error query in sql developer and got the results in no time(<2secs).

              I have enabled verbose error tracing(systemdatabase) and didn't find anything specific. Could you please advise me how can I proceed further to fix this issue.

               

              Thanks 

              • 4. Re: Contribution Folder Link Not Working!!
                William Phelps

                Is it truly sporadic, or do you see odd behavior at certain times of the day?  I don't think it's a Content Server issue, rather something environmental.  If it were something wrong in the core, you'd see issues more consistently.

                • What other applications share the file system/database with the Content Server? The timeout could come from the database being unable to respond due to other applications.
                • Is the behavior experienced by regular users? admin users (those having access to all security groups/accounts)?  Both sets of users having issues?  Any ACLs on the content?  Is there external LDAP connections in play here?
                • Does it happen at certain times of the day? Like during a backup window for the file system or database?
                • 5. Re: Contribution Folder Link Not Working!!
                  ManojC

                  Hi William,

                   

                           This issue is really sporadic. When I try to access contribution folders it works fine and even I am able to access child folders as well. When I try to access the child folders again after few seconds then the error occurs.

                   

                  What other applications share the file system/database with the Content Server? The timeout could come from the database being unable to respond due to other applications.

                  --> The database instance is shared with other four stage content server instances. When I checked with the DBA he mentioned that there was no symptoms related to this on DB side.

                   

                   

                  Is the behavior experienced by regular users? admin users (those having access to all security groups/accounts)?  Both sets of users having issues?  Any ACLs on the content?  Is there external LDAP connections in play here?

                  --> Both the sets of users having problem in accessing folders. We don't have any ACL and extenal LDAP is configured.

                   

                   

                  Does it happen at certain times of the day? Like during a backup window for the file system or database?

                  --> No. It occurs more frequently.

                   

                  Thanks

                  • 6. Re: Contribution Folder Link Not Working!!
                    William Phelps

                    More questions then.

                     

                    - Is this just occurring in one environment? or all environments?

                    - If you put the same folder structure in another environment, does the same behavior occur? (This test may be the most telling one.)

                    - How many times a day? Again, is it at a predictable time of day?  Or with a certain number of users on the system?

                    • 7. Re: Contribution Folder Link Not Working!!
                      ManojC

                      Hi  William,

                       

                      -

                       

                      We are using Folder_g component in only one environment.

                      - I couldn't test this step due to unavailability of instances.

                      - It occurs all th time in a day. It happens to work sometimes and after few seconds if try to access folders then it fails. I have coordinated with DBA for any errors in database but he was not able find anything.

                       

                      Please find the error trace.

                       

                      The service stack for this request is

                      --COLLECTION_DISPLAY (**no captured values**)

                      intradoc.data.DataException: !csDbUnableToCreateResultSet,SELECT Documents.dID\,dRevisionID\,dWebExtension\,dRendition1\,dDocName\, dDocTitle\, dDocAuthor\, dDocType\, dSecurityGroup\, dRevLabel\, dDocAccount\, dCreateDate\, dReleaseDate\, dInDate\, dOutDate\, dIsCheckedOut\, dRevRank\, dCheckoutUser\, dStatus\, dReleaseState\, dProcessingState\, dIndexerState\, dPublishType\, dPublishState\, dOriginalName\, dFormat\, dFileSize\, dExtension\, dIsPrimary\, dIsWebFormat\, dDocID\, dRevClassID FROM Documents\,DocMeta\,Revisions WHERE Documents.dID=Revisions.dID AND Documents.dID=DocMeta.dID AND dIsWebFormat=0 AND dIsPrimary&lt;&gt;0 AND ( ((DocMeta.xCollectionID=2))) AND NOT (dStatus=&#39;EXPIRED&#39; OR dStatus=&#39;DELETED&#39;)!$ORA-01013: user requested cancel of current operation

                      at intradoc.jdbc.JdbcWorkspace.handleSQLException(JdbcWorkspace.java:2371)

                      at intradoc.jdbc.JdbcWorkspace.createResultSetSQL(JdbcWorkspace.java:806)

                      at collections.HelperHandler.getResultSetSQL(HelperHandler.java:1074)

                      at collections.CollectionSearchHandler.getAllContentMatching(CollectionSearchHandler.java:422)

                      at collections.CollectionSearchHandler.getAllContentMatching(CollectionSearchHandler.java:384)

                      at collections.CollectionUserHandler.loadContentInCollection(CollectionUserHandler.java:3964)

                      at collections.CollectionUserHandler.loadContentInCollection(CollectionUserHandler.java:3939)

                      at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source)

                      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

                      at java.lang.reflect.Method.invoke(Method.java:615)

                      at intradoc.common.IdcMethodHolder.invokeMethod(ClassHelperUtils.java:617)

                      at intradoc.common.ClassHelperUtils.executeMethodReportStatus(ClassHelperUtils.java:293)

                      at intradoc.server.ServiceHandler.executeAction(ServiceHandler.java:79)

                      at intradoc.server.Service.doCodeEx(Service.java:490)

                      at intradoc.server.Service.doCode(Service.java:472)

                      at intradoc.server.ServiceRequestImplementor.doAction(ServiceRequestImplementor.java:1360)

                      at intradoc.server.Service.doAction(Service.java:452)

                      at intradoc.server.ServiceRequestImplementor.doActions(ServiceRequestImplementor.java:1201)

                      at intradoc.server.Service.doActions(Service.java:447)

                      at intradoc.server.ServiceRequestImplementor.executeActions(ServiceRequestImplementor.java:1121)

                      at intradoc.server.Service.executeActions(Service.java:433)

                      at intradoc.server.ServiceRequestImplementor.doRequest(ServiceRequestImplementor.java:635)

                      at intradoc.server.Service.doRequest(Service.java:1707)

                      at intradoc.server.ServiceManager.processCommand(ServiceManager.java:359)

                      at intradoc.server.IdcServerThread.run(IdcServerThread.java:197)

                      Caused by: java.sql.SQLException: ORA-01013: user requested cancel of current operation

                      at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)

                      at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:331)

                      at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:288)

                      at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:745)

                      at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:210)

                      at oracle.jdbc.driver.T4CStatement.executeForRows(T4CStatement.java:961)

                      at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1072)

                      at oracle.jdbc.driver.T4CStatement.executeMaybeDescribe(T4CStatement.java:845)

                      at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1146)

                      at oracle.jdbc.driver.OracleStatement.doScrollExecuteCommon(OracleStatement.java:4098)

                      at oracle.jdbc.driver.OracleStatement.doScrollStmtExecuteQuery(OracleStatement.java:4234)

                      at oracle.jdbc.driver.OracleStatement.executeQuery(OracleStatement.java:1320)

                      at intradoc.jdbc.JdbcWorkspace.createResultSetSQL

                       

                      Thanks


                      • 8. Re: Contribution Folder Link Not Working!!
                        William Phelps

                        If you turn on systemdatabase tracing in Content Server, in the console output, you will see the start time and end time for the execution of the query, something like what's shown below.  The snippet below ran in 437 milliseconds when subtracting the start time from the end time.  When your system hangs, what do your times look like?

                         

                        Also, is the database an Oracle RAC database by chance?  Or some other type of clustered arrangement?

                         

                        >systemdatabase/6 05.30 09:41:06.735 IdcServer-556 !csMonitorActiveDbConnections,2 
                        >systemdatabase/6 05.30 09:41:06.735 IdcServer-556 release pool connection
                        >systemdatabase/6 05.30 09:41:06.782 IdcServer-557 Reusing connection retrieved from external pool.
                        >systemdatabase/6 05.30 09:41:06.782 IdcServer-557 !csMonitorActiveDbConnections,3
                        >systemdatabase/6 05.30 09:41:06.782 IdcServer-557 Assigned connection to this thread, took 0.73 ms. connect
                        >systemdatabase/6 05.30 09:41:06.782 IdcServer-557 !csMonitorReadActions,12640
                        >systemdatabase/6 05.30 09:41:06.782 IdcServer-557 1.17 ms. SELECT * FROM UserExtendedAttributes WHERE UPPER(dUserName)=UPPER('weblogic')[Executed. Returned row(s): false]
                        >systemdatabase/6 05.30 09:41:07.172 IdcServer-557 !csMonitorActiveDbConnections,2
                        >systemdatabase/6 05.30 09:41:07.172 IdcServer-557 release pool connection
                        • 9. Re: Contribution Folder Link Not Working!!
                          ManojC

                          Hi William,

                           

                                   Please find the error trace below with query execution time.

                           

                          systemdatabase 06.03 08:38:04.895 IdcServerThread-91424 SELECT * FROM Collections, ColMeta WHERE Collections.dCollectionID=ColMeta.dCollectionID AND Collections.dCollectionID=2
                          systemdatabase 06.03 08:38:04.908 IdcServerThread-91424 12 ms. SELECT * FROM Collections, ColMeta WHERE Collections.dCollectionID=ColMeta.dCollectionID AND Collections.dCollectionID=2[Executed. Returned row(s): true]
                          systemdatabase 06.03 08:38:04.908 IdcServerThread-91424 SELECT * FROM Collections, ColMeta WHERE Collections.dCollectionID=ColMeta.dCollectionID AND Collections.dCollectionID=3
                          systemdatabase 06.03 08:38:04.914 IdcServerThread-91424 5.53 ms. SELECT * FROM Collections, ColMeta WHERE Collections.dCollectionID=ColMeta.dCollectionID AND Collections.dCollectionID=3[Executed. Returned row(s): true]
                          systemdatabase 06.03 08:38:04.914 IdcServerThread-91424 SELECT * FROM Collections, ColMeta WHERE Collections.dCollectionID=ColMeta.dCollectionID AND Collections.dCollectionID=1
                          systemdatabase 06.03 08:38:04.922 IdcServerThread-91424 7.80 ms. SELECT * FROM Collections, ColMeta WHERE Collections.dCollectionID=ColMeta.dCollectionID AND Collections.dCollectionID=1[Executed. Returned row(s): true]
                          systemdatabase 06.03 08:38:04.932 IdcServerThread-91424 Connection with id of 'IdcServerThread-91424.7' is removed from active connections with key of 'IdcServerThread-91424'.
                          systemdatabase 06.03 08:38:04.932 IdcServerThread-91424 release pool connection
                          systemdatabase 06.03 08:38:45.248 IdcServerThread-91406 SELECT COUNT(*) from Config
                          systemdatabase 06.03 08:38:45.250 IdcServerThread-91406 2.08 ms. SELECT COUNT(*) from Config[Executed. Returned row(s): true]
                          systemdatabase 06.03 08:38:45.250 IdcServerThread-91406 !csDbUnableToCreateResultSet,SELECT Documents.dID\,dRevisionID\,dWebExtension\,dRendition1\, <Custom Metadatas>\,dDocName\, dDocTitle\, dDocAuthor\, dDocType\, dSecurityGroup\, dRevLabel\, dDocAccount\, dCreateDate\, dReleaseDate\, dInDate\, dOutDate\, dIsCheckedOut\, dRevRank\, dCheckoutUser\, dStatus\, dReleaseState\, dProcessingState\, dIndexerState\, dPublishType\, dPublishState\, dOriginalName\, dFormat\, dFileSize\, dExtension\, dIsPrimary\, dIsWebFormat\, dDocID\, dRevClassID FROM Documents\,DocMeta\,Revisions WHERE Documents.dID=Revisions.dID AND Documents.dID=DocMeta.dID AND dIsWebFormat=0 AND dIsPrimary<>0 AND ( ((DocMeta.xCollectionID=786))) AND NOT (dStatus='EXPIRED' OR dStatus='DELETED')!$ORA-01013: user requested cancel of current operation

                          systemdatabase 06.03 08:38:45.250 IdcServerThread-91406 1401802725250 ms. [ORA-01013: user requested cancel of current operation
                          ]
                          systemdatabase 06.03 08:38:45.373 IdcServerThread-91406 Connection with id of 'IdcServerThread-91406.2' is removed from active connections with key of 'IdcServerThread-91406'.
                          systemdatabase 06.03 08:38:45.373 IdcServerThread-91406 release pool connection


                           

                          Thanks

                          • 10. Re: Contribution Folder Link Not Working!!
                            Jiri.Machotka-Oracle

                            For ORA-01013 see ORA-01013: user requested cancel of current operation tips

                             

                            It seems that it times out. Can you try to execute the query SELECT Documents.dID... from a sql console (sqlplus, etc.)? Or, perhaps, replace the column names with count(*)?

                            • 11. Re: Contribution Folder Link Not Working!!
                              ManojC

                              Hi Jiri,

                               

                                    I am able to execute the folder query from sql developer in <2 secs. From UI I am not interrupting the operation. It is getting timed-out after 60 secs.

                               

                              Thanks


                              • 12. Re: Contribution Folder Link Not Working!!
                                William Phelps

                                Interesting.  If you notice in your output, you have the output of two separate threads.  "IdcServerThread-91424" shows the majority of the SQL statement execution, but "IdcServerThread-91406" shows a timeout after 1401802725250 ms.  That's a pretty long time for that thread to stay open - if my math is right, isn't that like 16000+ DAYS???... you probably need to talk to your DBA and see if you have any hung sessions, but definitely something is wrong here.  You might want to tune your JDBC connections to timeout after a set period, rather than waiting indefinitely.

                                 

                                Since the output from two different threads is shown, is it possible to get the complete trace from just one thread?  Also, do you have some other external process that's accessing the Content Server doing a similar search?

                                • 13. Re: Contribution Folder Link Not Working!!
                                  ManojC

                                  Hi William,

                                   

                                  I have verified with my DBA and he mentioned that there are no hung sessions present in database. We have JDBC timeout for 40 secs and I am not sure why content server throwing up timeout as 1401802725250  ms. There is no external process performing this similar search.

                                   

                                  Please find the error trace from single thread.

                                   

                                  systemdatabase 05.29 12:00:38.677     IdcServerThread-217192 Connection with last id of IdcServerThread-216960.13 is added to active connections with key of 'IdcServerThread-217192'.
                                  systemdatabase 05.29 12:00:38.677     IdcServerThread-217192 Assigned connection to this thread, took 0.16 ms.
                                  systemdatabase 05.29 12:00:38.677     IdcServerThread-217192 Preparing connection for use, id initialized as IdcServerThread-217192.13
                                  systemdatabase 05.29 12:00:38.677     IdcServerThread-217192 SELECT * FROM Links,Collections,ColMeta WHERE dLinkCollectionID=4 AND dLinkType=1 AND Links.dLinkItemID=Collections.dCollectionID AND Collections.dCollectionID=ColMeta.dCollectionID
                                  systemdatabase 05.29 12:00:38.683     IdcServerThread-217192 5.45 ms. SELECT * FROM Links,Collections,ColMeta WHERE dLinkCollectionID=4 AND dLinkType=1 AND Links.dLinkItemID=Collections.dCollectionID AND Collections.dCollectionID=ColMeta.dCollectionID[Executed. Returned row(s): false]
                                  systemdatabase 05.29 12:00:38.685     IdcServerThread-217192 SELECT * FROM Collections, ColMeta WHERE Collections.dCollectionID=ColMeta.dCollectionID AND dParentCollectionID=4
                                  systemdatabase 05.29 12:00:38.696     IdcServerThread-217192 10 ms. SELECT * FROM Collections, ColMeta WHERE Collections.dCollectionID=ColMeta.dCollectionID AND dParentCollectionID=4[Executed. Returned row(s): false]
                                  systemdatabase 05.29 12:00:38.703     IdcServerThread-217192 SELECT Documents.dID,dRevisionID,dWebExtension,dRendition1, dDocName, dDocTitle, dDocAuthor, dDocType, dSecurityGroup, dRevLabel, dDocAccount, dCreateDate, dReleaseDate, dInDate, dOutDate, dIsCheckedOut, dRevRank, dCheckoutUser, dStatus, dReleaseState, dProcessingState, dIndexerState, dPublishType, dPublishState, dOriginalName, dFormat, dFileSize, dExtension, dIsPrimary, dIsWebFormat, dDocID, dRevClassID FROM Documents,DocMeta,Revisions WHERE Documents.dID=Revisions.dID AND Documents.dID=DocMeta.dID AND dIsWebFormat=0 AND dIsPrimary<>0 AND ( ((DocMeta.xCollectionID=4))) AND NOT (dStatus='EXPIRED' OR dStatus='DELETED')
                                  systemdatabase 05.29 12:01:39.684     IdcServerThread-217192 SELECT COUNT(*) from Config
                                  systemdatabase 05.29 12:01:39.686     IdcServerThread-217192 1.94 ms. SELECT COUNT(*) from Config[Executed. Returned row(s): true]
                                  systemdatabase 05.29 12:01:39.686     IdcServerThread-217192        !csDbUnableToCreateResultSet,SELECT Documents.dID\,dRevisionID\,dWebExtension\,dRendition1\, dDocName\, dDocTitle\, dDocAuthor\, dDocType\, dSecurityGroup\, dRevLabel\, dDocAccount\, dCreateDate\, dReleaseDate\, dInDate\, dOutDate\, dIsCheckedOut\, dRevRank\, dCheckoutUser\, dStatus\, dReleaseState\, dProcessingState\, dIndexerState\, dPublishType\, dPublishState\, dOriginalName\, dFormat\, dFileSize\, dExtension\, dIsPrimary\, dIsWebFormat\, dDocID\, dRevClassID FROM Documents\,DocMeta\,Revisions WHERE Documents.dID=Revisions.dID AND Documents.dID=DocMeta.dID AND dIsWebFormat=0 AND dIsPrimary<>0 AND ( ((DocMeta.xCollectionID=4))) AND NOT (dStatus='EXPIRED' OR dStatus='DELETED')!$ORA-01013: user requested cancel of current operation

                                  systemdatabase 05.29 12:01:39.686     IdcServerThread-217192 1401382899686 ms. [ORA-01013: user requested cancel of current operation

                                   

                                   

                                  Thanks

                                  • 14. Re: Contribution Folder Link Not Working!!
                                    Srinath Menon-Oracle

                                    Hi ,

                                     

                                    From SQL Developer - SQL Advisor , run the query and see if there are any cost analysis / better plan recommendation shown . Most likely you might need to create an index to get the query executed faster .

                                     

                                    Thanks,

                                    Srinath

                                    1 2 Previous Next