Skip to Main Content

SQL Developer

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

mimic netbeans IDE feature---group tabs in rows for SQL worksheet

user7091240Oct 19 2021 — edited Oct 19 2021

professional developers need to work at multiple SQL queries or worksheets, or packages when design, debug, trace, etc.
To help to organize the multiple worksheets better, instead of clicking on the left and right arrow at the end of the single bar, putting the tabs or tab heads into groups, and allow each group to be in a row, and even better allow each row has its own color.
All these have been available in Netbeans IDE for long time, and it is a low hanging sweet fruit to pick up and make SQL developer much more user friendly and more efficient for big projects which involves multiple PL/SQL packages, procedures, functions, triggers, etc.
Thank you very much.

Comments

Paul M.
Did you try an explain plan for your query ? Anyway, if the query can't be revised, then an index on class_id column might help...

How many rows has EVENT_DATA table ?
32685
Hello

Well, what about some data volumes and an execution plan? Also, how long does it take to run and how much faster does it need to be?

The main thing that sticks out like a sore thumb though is that you're representing date information in a NUMBER data type. The fact that you have to apply functions to this column to translate it to a date data type is going to prevent oracle from using any indexes. What I would suggest is to reverse the calculations i.e. set up a range based on SYSDATE that resolves a "utimestamp" value rather than the way you are doing it now.

Also, have you considered creating a composit index on even_data, perhaps on (class_id, agent_id) - and if you sort out the range properly, you could include utimestamp?

Without the bare minumum of an execution plan though, this could be a lot of wasted effort as the problem might not even be related to the indexing or utimestamp column.

HTH

David
438181
David, Paul:

2. I have about 500 million records. The number of records this query applies to is about 30-60 records and these records were inserted within the last 30 days.

3. The query takes about 1 hour to complete
4. Unfortunatley I cannot change the date field to a different type as it is created by the backend application. This field utimestamp hold the time value in Unix epoch format and I am using this sql to create monthly reports. I have no way of knowing the start and end times in Unix epoch format for the previous month assuming I am running the report on 1st of every month.

Thanks
Ravi
cd_2
Would this lead in the right direction?
SELECT   event_data.TYPE, COUNT (*)
    FROM event_data
   WHERE event_data.class_id = 910713
     AND event_data.agent_id = 160
     AND event_data.utimestamp >= TRUNC(SYSDATE - 2, 'MM') - TO_DATE('19700101', 'yyyymmdd')) * 86400 
GROUP BY event_data.TYPE;
not tested

Forgot the 'MM' in the TRUNC statement.

C.

Message was edited by:
cd
32685
Hello

My point was that if you can convert from a unix epoch date to a "normal" date, then you can do the same in reverse:
WHERE
	EVENT_DATA.utimestamp >=(TRUNC(SYSDATE, 'MM') - TO_DATE('01/01/1970','MM/DD/YYYY')) * (24*60*60)
AND
	EVENT_DATA.utimestamp < (TRUNC(LAST_DAY(SYSDATE) + 1 , 'MM') - TO_DATE('01/01/1970','MM/DD/YYYY')) * (24*60*60)
This sets up a range from the start of the current month to the end of the current month. You can of course substitute SYSDATE with a bind variable if this date is selectable.

What is the partition key for the table?

HTH

David

p.s.

To preserve the formatting of your posts, use [pre] and [/pre] to enclose your code or execution plan. it will make it much easier to read that way.

Message was edited by:
David Tyler


Didn't see your post cd...both on the same track though which is promising :-)
cd_2
Didn't see your post cd...both on the same track
though which is promising :-)
Yes, I'd say that this should give the OP an idea. And I forgot 'MM' in TRUNC ... ;-)

C.
438181

David, cd:
1. The partition key for the table is utime, the EVENT_DATA table is partitioned on weekly basis and the analysis is performed once every week. The last analysis was done on Friday. The table is indexed on utime and agent_id.

select last_analyzed, NUM_ROWS 
from user_tables 
where 
table_name='EVENT_DATA';

LAST_ANAL NUM_ROWS

--------- ----------

22-SEP-06 576809400

2. I changed the statement to include actual utime values and even this one takes about 90 minutes.

select EVENT_DATA.TYPE,COUNT(*)
from EVENT_DATA
where
EVENT_DATA.utime >= 1157083700 AND
EVENT_DATA.utime < 1159678400 AND
EVENT_DATA.agent_id = 160 AND
EVENT_DATA.class_id=910713
group by EVENT_DATA.TYPE;

Any thoughts?
Thanks for the help.
Ravi

cd_2
Yes, how about another explain plan?

C.
32685
....and the output from that query?
438181

cd: here is the explain plan. It's rather long. I am still learning how to interpret this.

SQL> explain plan set statement_id = 'nsuser' into plan_table for
  2  select /*+ PARALLEL (EVENT_DATA,2) */ EVENT_DATA.TYPE,COUNT(*)
  3  from ns.EVENT_DATA
  4  where
  5  EVENT_DATA.utime >= 1157083700 AND
  6  EVENT_DATA.utime < 1159678400 AND
  7  EVENT_DATA.agent_id = 160 AND
  8  EVENT_DATA.class_id=910713
  9  group by EVENT_DATA.TYPE;

Explained.

SQL> col id		 format 99
SQL> col parent_id	 format 999	heading 'Par|ID'
SQL> col position	 format 999	heading 'Pos'
SQL> col options	 format a14
SQL> col part_id	 format 9999	heading 'Compute|by step'
SQL> col cardinality	 format 9999999999999 heading 'Cardi-|nality'
SQL> col bytes		 format 9999999999999 heading 'Bytes'
SQL> col partition_start format 99999	heading 'Part|Start'
SQL> col partition_stop  format 99999	heading 'Part|Stop'
SQL> 
SQL> select substr(optimizer,1,8)	 optimizer,
  2  	    id,
  3  	    parent_id,
  4  	    position,
  5  	    substr(operation,1,16)	 operations,
  6  	    substr(object_name,1,10)	 obj_name,
  7  	    substr(options,1,12)	 options,
  8  	    substr(object_type,1,6)	 obj_type,
  9  	    cardinality,
 10  	    bytes,
 11  	    substr(partition_start,7,4)  partition_start,
 12  	    substr(partition_stop,7,4)	 partition_stop,
 13  	    partition_id		 part_id,
 14  	    substr(other_tag,1,23)	 other_tag
 15  from   plan_table
 16  order by id;

              Par                                                                Cardi-                Part Part Compute                                                                                                                                                                                    
OPTIMIZE  ID   ID  Pos OPERATIONS       OBJ_NAME   OPTIONS        OBJ_TY         nality          Bytes Star Stop by step OTHER_TAG                                                                                                                                                                          
-------- --- ---- ---- ---------------- ---------- -------------- ------ -------------- -------------- ---- ---- ------- -----------------------                                                                                                                                                            
CHOOSE     0      #### SELECT STATEMENT                                              63           1449                                                                                                                                                                                                      
           1    0    1 SORT                        GROUP BY                          63           1449                   PARALLEL_TO_SERIAL                                                                                                                                                                 
           2    1    1 SORT                        GROUP BY                          63           1449                   PARALLEL_TO_PARALLEL                                                                                                                                                               
           3    2    1 PARTITION RANGE             ITERATOR                                                            3 PARALLEL_COMBINED_WITH_                                                                                                                                                            
ANALYZED   4    3    1 TABLE ACCESS     EVENT_DATA FULL                           62817        1444791                 3 PARALLEL_COMBINED_WITH_                                                                                                                                                            

SQL> 
SQL> select substr(optimizer,1,8)	 optimizer,
  2  	    id,
  3  	    parent_id,
  4  	    position,
  5  	    substr(operation,1,16)	 operations,
  6  	    substr(object_name,1,30)	 obj_name,
  7  	    substr(options,1,12)	 options,
  8  	    substr(object_type,1,6)	 obj_type,
  9  	    cardinality,
 10  	    bytes,
 11  	    substr(partition_start,7,4)  partition_start,
 12  	    substr(partition_stop,7,4)	 partition_stop,
 13  	    partition_id		 part_id
 14  from   plan_table
 15  order by id;

              Par                                                                                    Cardi-                Part Part Compute                                                                                                                                                                
OPTIMIZE  ID   ID  Pos OPERATIONS       OBJ_NAME                       OPTIONS        OBJ_TY         nality          Bytes Star Stop by step                                                                                                                                                                
-------- --- ---- ---- ---------------- ------------------------------ -------------- ------ -------------- -------------- ---- ---- -------                                                                                                                                                                
CHOOSE     0      #### SELECT STATEMENT                                                                  63           1449                                                                                                                                                                                  
           1    0    1 SORT                                            GROUP BY                          63           1449                                                                                                                                                                                  
           2    1    1 SORT                                            GROUP BY                          63           1449                                                                                                                                                                                  
           3    2    1 PARTITION RANGE                                 ITERATOR                                                            3                                                                                                                                                                
ANALYZED   4    3    1 TABLE ACCESS     EVENT_DATA                     FULL                           62817        1444791                 3                                                                                                                                                                

SQL> 
SQL> 
SQL> select other transformed_statement
  2  from   plan_table
  3  where  id = (select max(id) from plan_table);

TRANSFORMED_STATEMENT                                                                                                                                                                                                                                                                                       
--------------------------------------------------------------------------------                                                                                                                                                                                                                            
                                                                                                                                                                                                                                                                                                            

SQL> 
SQL> 
SQL> 
SQL> col level	   format 99	heading 'LVL'
SQL> -- col operation format a62
SQL> col operation format a76
SQL> col options   format a20
SQL> col object    format a30
SQL> 
SQL> select level,
  2  	    lpad(' ',2*level) || to_char(rownum-1) || '-' || operation || decode(id,
  3  0, ' Cost= ' || position) operation,
  4  	    options,
  5  	    object_name object
  6  from   plan_table
  7  connect by prior id = parent_id
  8  start with id = 0
  9  order by id;

LVL OPERATION                                                                    OPTIONS              OBJECT                                                                                                                                                                                                
--- ---------------------------------------------------------------------------- -------------------- ------------------------------                                                                                                                                                                        
  1   0-SELECT STATEMENT Cost= 574405                                                                                                                                                                                                                                                                       
  2     1-SORT                                                                   GROUP BY                                                                                                                                                                                                                   
  3       2-SORT                                                                 GROUP BY                                                                                                                                                                                                                   
  4         3-PARTITION RANGE                                                    ITERATOR                                                                                                                                                                                                                   
  5           4-TABLE ACCESS                                                     FULL                 EVENT_DATA                                                                                                                                                                                            

SQL> 
SQL> col operation format a70
SQL> col cost	   format 999999999999
SQL> col kbytes    format 999999999999
SQL> select level,
  2  	    lpad(' ',2*level) || to_char(rownum-1) ||'-'|| operation ||' '|| options
  3  || decode(id, 0, substr(optimizer,1,6) || ' Cost='||to_char(cost)) operation,
  4  	    object_name object,
  5  	    cost,
  6  	    cardinality,
  7  	    round(bytes / 1024) kbytes
  8  from   plan_table
  9  connect by prior id = parent_id
 10  start with id = 0
 11  order by id;

                                                                                                                                Cardi-                                                                                                                                                                      
LVL OPERATION                                                              OBJECT                                  COST         nality        KBYTES                                                                                                                                                        
--- ---------------------------------------------------------------------- ------------------------------ ------------- -------------- -------------                                                                                                                                                        
  1   0-SELECT STATEMENT CHOOSE Cost=574405                                                                      574405             63             1                                                                                                                                                        
  2     1-SORT GROUP BY                                                                                          574405             63             1                                                                                                                                                        
  3       2-SORT GROUP BY                                                                                        574405             63             1                                                                                                                                                        
  4         3-PARTITION RANGE ITERATOR                                                                                                                                                                                                                                                                      
  5           4-TABLE ACCESS FULL                                          EVENT_DATA                            574355          62817          1411                                                                                                                                                        

SQL> spool off
438181

output

select EVENT_DATA.TYPE,COUNT(*)
from EVENT_DATA
where
EVENT_DATA.utime >= 1157083700 AND
EVENT_DATA.utime < 1159678400 AND
EVENT_DATA.agent_id = 160 AND
EVENT_DATA.class_id=910713
group by EVENT_DATA.TYPE;


TYPE
--------------------------------------------------------------------------------
  COUNT(*)
----------
http
	12

udp
	 2

unauthenticated_tcp
	 6


TYPE
--------------------------------------------------------------------------------
  COUNT(*)
----------
dns(udp)
	 4

tcp_connections
	 2

tcp_incoming
	27


TYPE
--------------------------------------------------------------------------------
  COUNT(*)
----------
tcp_outgoing
	12

tcp_ratio
	 3


8 rows selected.
32685
Hello

Can I make a suggestion with the execution plan?....instead of using these scripts, just use DBMS_XPLAN as it gives you a fair bit more info and it presents the plan in nicely formatted way:
SQL> EXPLAIN PLAN FOR
  2  select * from dual;

Explained.

SQL> SELECT * FROM TABLE(dbms_xplan.display);

PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------
Plan hash value: 272002086

--------------------------------------------------------------------------
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |     1 |     2 |     2   (0)| 00:00:01 |
|   1 |  TABLE ACCESS FULL| DUAL |     1 |     2 |     2   (0)| 00:00:01 |
--------------------------------------------------------------------------

8 rows selected.
Also, how were the stats gathered i.e what options were used?

David
438181

David:

SQL> SELECT * FROM TABLE(dbms_xplan.display);

PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-----------------------------------------------------------------------------------------------------------------------
| Id  | Operation                  |  Name       | Rows  | Bytes | Cost  | Pstart| Pstop |  TQ    |IN-OUT| PQ Distrib |
-----------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT           |             |    63 |  1449 |   574K|       |       |        |      |            |
|   1 |  SORT GROUP BY             |             |    63 |  1449 |   574K|       |       | 72,01  | P->S | QC (RAND)  |
|   2 |   SORT GROUP BY            |             |    63 |  1449 |   574K|       |       | 72,00  | P->P | HASH       |
|   3 |    PARTITION RANGE ITERATOR|             |       |       |       |   244 |   249 | 72,00  | PCWP |            |
|*  4 |     TABLE ACCESS FULL      | EVENT_DATA  | 62817 |  1410K|   574K|   244 |   249 | 72,00  | PCWP |            |
-----------------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   4 - filter("EVENT_DATA"."UTIME">=1157083700 AND "EVENT_DATA"."UTIME"<1159678400 AND "EVENT_DATA"."AGENT_ID"=160 AND
              "EVENT_DATA"."CLASS_ID"=910713)

Note: cpu costing is off

18 rows selected.

SQL>
438181

David, cd: another thing I notice is the following

if I donot use the count(*), the explain plan shows it's using the EVENT_DATA_AGENT_IDX index. When I add the count, it tries to access the full table.

SQL> explain plan set statement_id = 'nsuser' into plan_table for
  2  select /*+ FIRST_ROWS */ EVENT_DATA.TYPE
  3  from ns.EVENT_DATA
  4  where
  5  EVENT_DATA.utime >= 1157083700 AND
  6  EVENT_DATA.utime < 1159678400 AND
  7  EVENT_DATA.agent_id = 160 AND
  8  EVENT_DATA.class_id=910713;

Explained.

SQL> SELECT * FROM TABLE(dbms_xplan.display);

PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------
| Id  | Operation                          |  Name                 | Rows  | Bytes | Cost  | Pstart| Pstop |
------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                   |                       | 62817 |  1410K|  1510K|       |       |
|*  1 |  TABLE ACCESS BY GLOBAL INDEX ROWID| EVENT_DATA            | 62817 |  1410K|  1510K| ROWID | ROW L |
|*  2 |   INDEX RANGE SCAN                 | EVENT_DATA_AGENT_IDX  |  6071K|       | 30698 |       |       |
------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter("EVENT_DATA"."UTIME">=1157083700 AND "EVENT_DATA"."UTIME"<1159678400 AND
              "EVENT_DATA"."CLASS_ID"=910713)
   2 - access("EVENT_DATA"."AGENT_ID"=160)

Note: cpu costing is off

17 rows selected.

SQL>
SQL> explain plan set statement_id = 'nsuser' into plan_table for
  2  select /*+ FIRST_ROWS */ EVENT_DATA.TYPE, COUNT(*)
  3  from ns.EVENT_DATA
  4  where
  5  EVENT_DATA.utime >= 1157083700 AND
  6  EVENT_DATA.utime < 1159678400 AND
  7  EVENT_DATA.agent_id = 160 AND
  8  EVENT_DATA.class_id=910713
  9  GROUP BY EVENT_DATA.TYPE;

Explained.

SQL> SELECT * FROM TABLE(dbms_xplan.display);

PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-----------------------------------------------------------------------------------------------------------------------
| Id  | Operation                  |  Name       | Rows  | Bytes | Cost  | Pstart| Pstop |  TQ    |IN-OUT| PQ Distrib |
-----------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT           |             |    63 |  1449 |   287K|       |       |        |      |            |
|   1 |  SORT GROUP BY             |             |    63 |  1449 |   287K|       |       | 77,01  | P->S | QC (RAND)  |
|   2 |   SORT GROUP BY            |             |    63 |  1449 |   287K|       |       | 77,00  | P->P | HASH       |
|   3 |    PARTITION RANGE ITERATOR|             |       |       |       |   244 |   249 | 77,00  | PCWP |            |
|*  4 |     TABLE ACCESS FULL      | EVENT_DATA  | 62817 |  1410K|   287K|   244 |   249 | 77,00  | PCWP |            |
-----------------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   4 - filter("EVENT_DATA"."UTIME">=1157083700 AND "EVENT_DATA"."UTIME"<1159678400 AND "EVENT_DATA"."AGENT_ID"=160 AND
              "EVENT_DATA"."CLASS_ID"=910713)

Note: cpu costing is off

18 rows selected.
438181
Just wondering if there is a way to force the 2nd SQL statement to use a index?
32685
hello

You can't force the optimiser to use an index. You can pass a hint which may make it choose to use that index but it will not force it to do so.

I think I may be missing something here but in the stats, the cardinality for the partition table scan is saying 62k rows but you stated that there were billions of rows in the table as a whole. The query that you ran also returns very few rows indeed so are the stats definately accurate? If the number of rows is so low, and you have global indexes giving you access to those rows, why the need for parallel query?

David
536027
For the hint you may try

select /*+INDEX(a,EVENT_DATA_PART_IDX)*/ a.TYPE,COUNT(0)
from ns.EVENT_DATA a
where
a.utime >= 1157083700 AND
a.utime < 1159678400 AND
a.agent_id = 160 AND
a.class_id=910713
group by a.TYPE;

If it tries a full index scan you may want to add a cardinality hint( CARDINALITY(A,50) )
to indicate the number of rows you are expecting.
1 - 17

Post Details

Added on Oct 19 2021
14 comments
161 views