This discussion is archived
1 2 Previous Next 25 Replies Latest reply: Oct 20, 2009 7:56 PM by ChaoHuang Go to original post RSS
  • 15. Re: How can I optimize  insert(put) performance?
    greybird Expert
    Currently Being Moderated
    What does
    TxnNoSync false
    mean? This seems like the opposite of what you want, which TxnNoSync=true.

    --mark                                                                                                                                                                                                                                                           
  • 16. Re: How can I optimize  insert(put) performance?
    Charles Lamb Pro
    Currently Being Moderated
    Please compile and run the following program with args

    java SyncTest raf notdirect sync 10000 512

    and

    java SyncTest raf notdirect nosync 10000 512

    and tell me the results.
    import java.io.File;
    import java.io.FileDescriptor;
    import java.io.FileOutputStream;
    import java.io.IOException;
    import java.io.RandomAccessFile;
    import java.nio.ByteBuffer;
    import java.nio.channels.FileChannel;
    
    /*
     * Measure the cost of write with and without fsync. The test can vary
     * these factors:
     *   java package used: nio = java.nio
     *                      raf = java.io.RandomAccessFile
     *                      fos = java.io.FileOutputStream
     *   direct/notdirect: applies only to NIO use, determines whether direct
                            buffers are used
     *   sync/nosync:      if sync, fsyncs are executed after ever write
     *   numWrites:        number of writes
     *   writeSize:        (in bytes)
     *
     * In JE.1.7.1 and 2.0, the parameters to mimic default commits are:
     *     java SyncTest nio notdirect sync <numWrites> <writeSize>
     */
    public class SyncTest {
        private RandomAccessFile file;
        private int numWrites;
        private int writeSize;
    
        public static void main(String[] argv) {
    
            try {
                if (argv.length != 5) {
                    System.out.println("Usage: SyncTest <nio|raf|fos>" +
                                       " <direct|notdirect>" +
                                       " <sync|nosync)>" +
                                       "<numWrites> <writeSize>");
                } else {
                    boolean useNio = argv[0].equalsIgnoreCase("nio");
                    boolean useRaf = argv[0].equalsIgnoreCase("raf");
                    boolean useDirect = argv[1].equalsIgnoreCase("direct");
                    boolean doSync = argv[2].equalsIgnoreCase("sync");
                    int numWrites = Integer.parseInt(argv[3]);
                    int writeSize = Integer.parseInt(argv[4]);
    
                    SyncTest test = new SyncTest(numWrites, writeSize);
    
                    System.out.println("sync=" + doSync);
                    System.out.println("nio=" + useNio);
                    System.out.println("raf=" + useRaf);
                    System.out.println("fos=" + (!useRaf && !useNio));
                    System.out.println("numWrites=" + numWrites);
                    System.out.println("writeSize=" + writeSize);
    
                    if (useNio) {
                        test.nioWrite(useDirect, doSync);
                    } else if (useRaf){
                        test.rafWrite(doSync);
                    } else {
                        test.fosWrite(doSync);
                    }
                }
            } catch (Exception e) {
                e.printStackTrace();
                System.exit(-1);
            }
        }
    
        SyncTest(int numWrites, int writeSize)
            throws IOException  {
    
            File testFile= new File("syncTest.data");
            if (testFile.exists()) {
                testFile.delete();
            }
    
            file = new RandomAccessFile("syncTest.data", "rw");
            this.numWrites = numWrites;
            this.writeSize = writeSize;
        }
    
        void nioWrite(boolean useDirect, boolean doSync)
            throws Exception {
    
            /* Setup file/channel */
    
            FileChannel channel = file.getChannel();
            ByteBuffer buffer = null;
            if (useDirect) {
                buffer = ByteBuffer.allocateDirect(writeSize);
            } else {
                buffer = ByteBuffer.allocate(writeSize);
            }
    
            /* Setup data. */
            for (int i = 0; i < writeSize; i++) {
                buffer.put((byte) 1);
            }
    
            System.out.println("direct=" + buffer.isDirect());
    
            int numWritten = 0;
            int position = 0;
    
            /* write and sync */
            long start = System.currentTimeMillis();
            for (int i = 0; i < numWrites; i++, position += numWritten) {
                buffer.flip();
                numWritten = channel.write(buffer, position);
                if (numWritten != writeSize) {
                    throw new Exception("Record " + i + " only " +
                                        numWritten + " bytes written");
                }
                if (doSync) {
                    channel.force(false);
                }
            }
            long end = System.currentTimeMillis();
    
            reportStats(start, end);
        }
    
        void rafWrite(boolean doSync)
            throws Exception {
    
            /* Setup file */
            RandomAccessFile file = new RandomAccessFile("syncTest.data", "rw");
    
            /* Setup data. */
            byte[] data = new byte[writeSize];
            for (int i = 0; i < writeSize; i++) {
                data[i] = (byte) 1;
            }
    
            /* write and sync */
            long start = System.currentTimeMillis();
            FileDescriptor fd = file.getFD();
            for (int i = 0; i < numWrites; i++) {
                file.write(data);
    
                if (doSync) {
                    fd.sync();
                }
            }
            long end = System.currentTimeMillis();
    
            reportStats(start, end);
        }
        void fosWrite(boolean doSync)
            throws Exception {
    
            /* Setup file */
            File outputFile = new File("syncTest.data");
            outputFile.createNewFile();
            FileOutputStream file = new FileOutputStream(outputFile, true);
    
            /* Setup data. */
            byte[] data = new byte[writeSize];
            for (int i = 0; i < writeSize; i++) {
                data[i] = (byte) 1;
            }
    
            /* write and sync */
            long start = System.currentTimeMillis();
            FileDescriptor fd = file.getFD();
            for (int i = 0; i < numWrites; i++) {
                file.write(data);
    
                if (doSync) {
                    fd.sync();
                }
            }
            long end = System.currentTimeMillis();
    
            reportStats(start, end);
        }
    
        private void reportStats(long start, long end) {
            double sec = (end - start)/1e3;
            System.out.println("total time: " + sec + " sec clock");
            System.out.println("throughput: " + (double) numWrites/sec +
                               " (write+sync)/sec\t");
        }
    
    }
    Edited by: Charles Lamb on Oct 19, 2009 7:11 AM

    Edited by: Charles Lamb on Oct 22, 2009 9:12 AM
  • 17. Re: How can I optimize  insert(put) performance?
    727712 Newbie
    Currently Being Moderated
    sync=true
    nio=false
    raf=true
    fos=false
    numWrites=10000
    writeSize=512
    total time: 10.187 sec clock
    throughput: 981.6432708353785 (write+sync)/sec     


    sync=false
    nio=false
    raf=true
    fos=false
    numWrites=10000
    writeSize=512
    total time: 0.047 sec clock
    throughput: 212765.95744680852 (write+sync)/sec
  • 18. Re: How can I optimize  insert(put) performance?
    Charles Lamb Pro
    Currently Being Moderated
    So two questions:

    (1) As my colleague Mark Hayes points out, you have txnNoSync=false which seems to be the opposite of what you want. Don't you want txnNoSync=true? Make sure you're not setting any other txnSync options (e.g. txnWriteNoSync) -- only set txnNoSync.

    (2) What happens if you run SyncTest again with your write cache disabled (hdparm -W 0)?

    Charles Lamb
  • 19. Re: How can I optimize  insert(put) performance?
    727712 Newbie
    Currently Being Moderated
    (1) sure i only set txnNoSync false,in other word,all keep default set.

    (2) on my xp home ,write cache enable default,when i disable wirte cache,Significantly reduce the efficiency of.
    Excute TestJEDPLAS Insert 100000 Records total times 50562ms

    default:
    Excute TestJEDPLAS Insert 100000 Records total times 18306ms
  • 20. Re: How can I optimize  insert(put) performance?
    Charles Lamb Pro
    Currently Being Moderated
    Xin,

    Please email me your source code (TestJEDPLAS and all related class files). I am completely confused as to which code we're using (Chao posted on code on his blog site which may be different from what you're running) and on which platform (I've seen results from linux and Windows). It will be easier if I can just run this code on my own machine.

    Thanks.

    Charles Lamb (charles.lamb @ o...le.com)bdbje

    Edited by: Charles Lamb on Oct 22, 2009 9:12 AM
  • 21. Re: How can I optimize  insert(put) performance?
    727712 Newbie
    Currently Being Moderated
    Have to send you e-mail, please check.
  • 22. Re: How can I optimize  insert(put) performance?
    Charles Lamb Pro
    Currently Being Moderated
    Kevin,

    When I run your code I get the following:

    On my Windows XP machine (AMD Athlon 64 X2 Dual Core 5200+ 2.7GHz):

    -Xmx1024m:
    18.781 secs (TxnNoSync true)
    23.172 secs (TxnNoSync false)

    On my Solaris machine I get (2.0GHz Quadcore Intel Nahalem):

    Disk Write Cache Enabled:
    13.382 secs (TxnNoSync false)
    11.883 secs (TxnNoSync true)

    Disk Write Cache Disabled:
    14.207 secs (TxnWriteNoSync false)
    11.801 secs (TxnWriteNoSync true)

    Note that by not setting TxnNoSync to true (i.e by leaving it as the default), you will get slower results as shown above.

    The fact that there is so little difference between the results for write cache enabled and disabled hints that this app is cpu-bound and not IO bound. My guess is that your windows machine has a fairly slow CPU.

    I notice that AuthServiceAccenssor.loadAuthServiceDb does a lot of string concatenations. This is known to take a lot of CPU.

    Charles Lamb
  • 23. Re: How can I optimize  insert(put) performance?
    727712 Newbie
    Currently Being Moderated
    thanks,Charles Lamb.

    This result with our good performance of the machine is almost. 5000+records/sec

    If you hava good idea,Please tell me, thanks a lot!
  • 24. Re: How can I optimize  insert(put) performance?
    Charles Lamb Pro
    Currently Being Moderated
    Kevin,

    The difference is probably cpu speed. If you run it on a faster CPU, I am quite sure you'll see better performance. Also, be sure to set TxnNoSync=true during the bulk load phase. You can make the data durable when you call Environment.close() or Environment.sync() and then set TxnNoSYnc=false when you're doing subsequent CRUD operations.

    I really don't think there is anything else I can do to help you. We have run this on several CPU's and the differences in performance are all directly attributable to cpu clock speed. As we say, "You can't get blood from a stone." If you need better performance, then you need to get a faster cpu.

    Charles Lamb
  • 25. Re: How can I optimize  insert(put) performance?
    ChaoHuang Newbie
    Currently Being Moderated
    Hi Kevin,

    My colleague (Charles Lamb) gets confused with your questions. If you could post your further questions here in Chinese, I can help you explain to the JE development team in English (via internal email).

    Let me try to explain Charles Lamb's answers to your current questions here in Chinese:
    1. 我们怀疑造成你的(3k insert/sec)和我的(9k insert/sec)性能差异的原因是:你的CPU太慢!但暂时不是100%肯定。
    - 你只是提到你XP机器的配置(CPU: 1.73 GHz * 2核, 2G memory, 160G 硬盘 - 是5400转还是7200?)。然后你又用一台Linux机器做了同样测试(你没有提到这台Linux机器的具体信息..)。
    - 对比来看, 我的机器是: CPU: 2.13 GHz * 2核, 2G memory, 250G硬盘 - 7200转,并设置了write cache。

    2. 你的期望的性能是 5K insert/sec/thread,是吗?如果是的话,我们建议你做一些性能的profiling,来确认性能瓶颈是否在CPU。

    3. 如果你想在你目前"我们看来相对慢”的机器上尽量调优你的批量数据加载(纯插入)性能,我们建议你尝试:
    - 设置TxnNoSync = true (用别的办法保证数据不丢失 - 如Charles说的:You can make the data durable when you call Environment.close() or Environment.sync() and then set TxnNoSYnc=false when you're doing subsequent CRUD operations.)
    - 用多个线程来插入
    - 加大JE的cache 大小(比如800M)
    - 加大JE的log file大小(比如设置到50M,默认是10M)
    - 关闭Je 的 cleaner和checkpointer线程
    - 调优你的JDK参数(比如:-server -Xms1024m -Xmx1024m -XX:+UseConcMarkSweepGC -XX:+UseAdaptiveGCBoundary -XX:+DisableExplicitGC)

    欢迎提问,谢谢!

    Chao Huang
    Oracle Berkeley DB

    Edited by: Chao Huang on Oct 21, 2009 10:55 AM
1 2 Previous Next

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points