Skip to Main Content

SQL & PL/SQL

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

Interested in getting your voice heard by members of the Developer Marketing team at Oracle? Check out this post for AppDev or this post for AI focus group information.

LEAD and LAG taking hours on a very large table!

Sam_PApr 1 2022 — edited Apr 3 2022

Hello,
I have a source table (partitioned and indexed) with approx. 156 Million records. I am trying to create a target table using CTAS method by reading records from the source table with a simple SELECT query that does simple LEAD/LAG operations on a date-column and it is taking over 8 hrs!!
Upon searching online, one developer recommended to try the MATCH_RECOGNIZE technique as it is claimed to run faster than the Analytic SQL functions on very large tables. I have not tested this yet as I need help in converting the SQL query into the MATCH_RECOGNIZE statement, which would result in the same result-set.
CREATE TABLE TB_TARGET
PARALLEL 8
NOLOGGING
AS
SELECT /*+ PARALLEL_INDEX(8) PARALLEL(8) */
PAT_ID,
VISIT_DATE_DT,
ROW_NUMBER() OVER (PARTITION BY PAT_ID ORDER BY VISIT_DATE_DT) AS ROW_NUM,
ED_UCC_DT,
LAG(ED_UCC_DT IGNORE NULLS) OVER (PARTITION BY PAT_ID ORDER BY VISIT_DATE_DT) AS PREV_ED_UCC_DT,
LEAD(ED_UCC_DT IGNORE NULLS) OVER (PARTITION BY PAT_ID ORDER BY VISIT_DATE_DT) AS NEXT_ED_UCC_DT
FROM TB_SOURCE; --Very large table!

The result set for a single PAT_ID record (100123456) looks like this:-
image.png
Questions:-
1. How do I speed this query up when it runs over 156 Million records with multiple LEAD and LAG operations added into the above query on other columns in addition to the above column?
2. If the fastest solution is to use MATCH_RECOGNIZE, could you please help me with composing a SQL query statement such that it yields the same results as shown above (along with NULLs)?
3. Any other solution is welcomed as long as I can reduce the query execution time to an acceptable duration other than 8 hrs.
Thank you in advance.

This post has been answered by alvinder on Apr 6 2022
Jump to Answer

Comments

Christine Lei-Oracle

Have you tried to wrap the child data the other way around as

const bufferingDP = new BufferingDataProvider(childArrayDataProvider);
this.childDataProvider(new ListDataProviderView(bufferingDP));

Hopefully in this way, ListDataProviderView would apply filter on all buffered data.

DaveArch

Thanks for your reply Christine.
Unfortunately that doesn’t work because wrapping it the other way around means that the functions specific to BufferingDataProvider are not available i.e. getSubmittableItems()
node.js:3173 Uncaught TypeError: this.childDataProvider(...).getSubmittableItems is not a function
Any other ideas?
Is there a way of altering the filterCriterion on ListDataProviderView without replacing the whole data provider which is wrapping it? I was wondering whether this would work so that they next time the data is fetched it will use the new filter?

Christine Lei-Oracle

The original wrapping order will use original data without buffered data information. Don't seem to have a way better than your current one although it's not that 'automatic' unfortunately.

1 - 3

Post Details

Added on Apr 1 2022
67 comments
5,344 views