This discussion is archived
2 Replies Latest reply: Sep 25, 2007 3:03 PM by 807605 RSS

Performance

807605 Newbie
Currently Being Moderated
I have 2 Excel files to compare, both files have about 40,000 rows and 34 columns, I have to pick up about 10 values from each row and compare.
I sort the entries and run the code, what happens is upto about 15,000 entries performance is good, but it slows down dramatically once the number reaches 17,000 and it takes for ever.

any suggestions?
  • 1. Re: Performance
    807605 Newbie
    Currently Being Moderated
    mkapadia wrote:
    I have 2 Excel files to compare, both files have about 40,000 rows and 34 columns, I have to pick up about 10 values from each row and compare.
    I sort the entries and run the code, what happens is upto about 15,000 entries performance is good, but it slows down dramatically once the number reaches 17,000 and it takes for ever.

    any suggestions?
    Is it the sorting that makes it slow down? You make it sound like the slow down happens after they're sorted. How much data are you loading into memory at once?
  • 2. Re: Performance
    807605 Newbie
    Currently Being Moderated
    Hmm lemme see...
    I have 2 Excel files to compare, both files have about 40,000 rows and 34 >columns, I have to pick up about 10 values from each row and compare.
    I sort the entries and run the code, what happens is upto about 15,000 >entries performance is good, but it slows down dramatically once the >number reaches >17,000 and it takes for ever.
    yes, I am using AraayList.
    I read from the file, add to ArrayList, cast the ArrayList object to String and >compare.
    there are 34 columns on the each row,
    1. I read a line form the FIRST file
    2. Tokenize the line, and I get 34 tokens
    3. Add them into the ArrayList
    4. Iread a line form the FIRST file
    5. Tokenize the line, and I get 34 tokens
    6. Add them into the ArrayList
    7. Compare
    
    while(ReadLine != null) //FIRST File
    {
    int match =0;
    tokenize
    Add to ArryList
    while(ReadLine != null and match ==0 ) //Second File
    {
    tokenize
    Add to ArryList
    compare
    {
    Pick up the required fields and compare till it finds the match
    }
    }
    if not found
    write the output to the file.
    } 
    (General head scratching)
    How about a real code example? Without a real example can't tell.
    Are you making a new Tokenizer for every line for example???

    (T)