Ordered query against large data set
Hello --
I'm looking to see if anyone has any idea to solve an issue that I am working.
I need to run a query against a large table (10 billion records daily). I need to limit the amount of results returned(generally 1000), as many will satisfy the predicate. I have no idea what time window will contain enough data for that particular predicate to reach 1000 records. It might be 5 minutes, it might be 21 hours.
I must return the most recent results that satisfy the predicate. So without sorting all of the data in the time window in a sub-select (way too slow) I don't see how to efficiently accomplish this.
I'm looking to see if anyone has any idea to solve an issue that I am working.
I need to run a query against a large table (10 billion records daily). I need to limit the amount of results returned(generally 1000), as many will satisfy the predicate. I have no idea what time window will contain enough data for that particular predicate to reach 1000 records. It might be 5 minutes, it might be 21 hours.
I must return the most recent results that satisfy the predicate. So without sorting all of the data in the time window in a sub-select (way too slow) I don't see how to efficiently accomplish this.
0