This discussion is archived
1 Reply Latest reply: Nov 2, 2011 6:00 AM by Charles Lamb RSS

Is there a good way to import a bulk of records into a kvstore?

897380 Newbie
Currently Being Moderated
I have one billion records.
I wrote a Java program and call store.put(), but the performance is very poor.

Is there a better way?
  • 1. Re: Is there a good way to import a bulk of records into a kvstore?
    Charles Lamb Pro
    Currently Being Moderated
    894377 wrote:
    I have one billion records.
    I wrote a Java program and call store.put(), but the performance is very poor.
    Writing one billion records (in any system) is going to require some tuning. You didn't say anything about your configuration (number of machines, size of machines, amount of memory, size of cache, amount of disk, type of disk, etc). You may be using kvlite. If so, kvlite is not intended for any kind of serious performance -- it is only meant to be a way for someone to use the API in a relatively "small" environment.

    I suggest that you start by reading Chapter 2 in the Admin Guide, [Planning Your Installation|http://download.oracle.com/docs/cd/NOSQL/html/AdminGuide/installplanning.html]. It contains information about sizing your NoSQL Database properly.

    Charles

    Edited by: Charles Lamb on Nov 2, 2011 8:59 AM

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points