Lessons Learned Doing Large Parallel Inserts and Index Builds on Exadata
Wanted to share something new I learned while doing some large inserts and subsequently index builds on some large Siebel tables. We had to insert data from around 10 archive tables back into the base tables they came from. After that we had to build the indexes that were dropped to make the inserts run faster. The number of records inserted varies from around 200M to 2B.
Things were pretty fast except for one table S_EVT_MAIL which contains a CLOB column and suffered from a lot of IO wait time which the other tables did not. For this table IO wait accounted for over 80% of the total time where for most of the other tables IO wait was well under 10% even other tables that also had CLOB columns. Total time for this table was nearly 8 hours and it was not even one of the biggest tables with only 244M rows to be inserted.