7 Replies Latest reply on Oct 4, 2017 9:43 AM by 3260775

    copy with "dd" command take long time when specifying block size

    3260775

      Hi All,

       

      on our platform, we used a copy with the command "dd" with the block size specification.

      the result was not good, because the time of the copy was long for example:

       

      root@T5_ldm_app:/# time sh -c " dd if=/dev/zero of=test.img bs=8k count=100000 && sync"

      100000+0 records in
      100000+0 records out
       real    0m19.121s
      user    0m0.172s
      sys     0m1.387s
       
       root@T5_ldm_app:/# time sh -c " dd if=/dev/zero of=test.img bs=128k count=10000 && sync"
      10000+0 records in
      10000+0 records out
       
      real    0m39.056s
      user    0m0.020s
      sys     0m0.686s

       

       

      the platform in question is composed of sparc T5-4 and FS1-2.

      can someone offer me a solution to reduce this time?

       

      noting that the old platform composed of T4-4 is on which there is more I / O the copy is done in less than 8 seconds.

       

      thanks ALL.

        • 1. Re: copy with "dd" command take long time when specifying block size
          Andris Perkons-Oracle

          Hi,

           

          make sure you are comparing apples with apples. Did the T4-4 use the same storage? Same disks with the same disk layout? Same filesystem with exactly the same mount options?

          What happens to the run times if you leave out the "sync" command?

           

          Andris

          • 2. Re: copy with "dd" command take long time when specifying block size
            Andris Perkons-Oracle

            And when running this from within guest domains:

             

            Is the storage attached the same way on both the T4-4 and the T5? How is it attached - does the domain access the storage directly or is the storage virtualized (vdisk or vhba)?

             

            Are the control domains sized equally (CPU, RAM)?

            • 3. Re: copy with "dd" command take long time when specifying block size
              3260775

              Hi Andris

               

              Thank you for your return.

               

              I will return with the answers as soon as possible

              • 4. Re: copy with "dd" command take long time when specifying block size
                3260775

                Sorry for delay

                below the specification of the two platforms

                 

                 

                T4-4

                T5-4

                File System

                vxfs

                zfs

                Storage array

                Oracle  STK 2540 M2

                Oracle FS1-2

                Connection type between storage array and server

                Directly

                Via switch Brocade

                                      

                 

                T4-4

                T5-4

                File System

                vxfs

                zfs

                Storage array

                Oracle    STK 2540 M2

                Oracle FS1-2

                Connection type between storage array and server

                Directly

                Via switch Brocade

                • 5. Re: copy with "dd" command take long time when specifying block size
                  Andris Perkons-Oracle

                  Hi,

                   

                  well, apples and oranges...

                  Unless your primary use case is to copy data with "dd", your comparison is useless.

                   

                  I would bet that the vxfs has no mountoptions, which translates to "delaylog", meaning that you have a –3 second window where your data is not necessarily persistent. Try setting your mountoptions to "log,convosync=direct" and you'll see that your vxfs performance will drop dramatically.

                   

                  Then, your workload is just small enough (800 MB resp 1,2GB) to fit into the STK 2540's cache. I don't know how large the cache for the FS1-2 is, but I guess that with larger workloads, the performance differences will level out.

                  The STK uses 15k RPM disks. How fast are the disks inside the FS1? Then, you don't mention how many disks make up your file systems.

                   

                  The connection of the storage arrays also has different latencies. A direct connection should be slightly faster than a connection via a switch. Not much faster, but "100000 x not much faster" adds up.

                   

                  Then on the topic of ZFS: What recordsize are you using? How is the zpool set up? Mirrored or Raid-Z-something? How full is the file system?

                   

                  Andris

                  • 6. Re: copy with "dd" command take long time when specifying block size
                    Nik

                    Hi.

                    For compare need select base. This can allow say that some feature or hardware is better or not.

                     

                    You have whole different.

                    Different  servers, different arrays ( with different disks), different FS.

                    Any from this can be point of the problem.

                     

                    You test:

                     

                    root@T5_ldm_app:/# time sh -c " dd if=/dev/zero of=test.img bs=8k count=100000 && sync"

                    You copy 8k * 100 000 ~ 800 MB   Speed ~ 41 MB/s

                    root@T5_ldm_app:/# time sh -c " dd if=/dev/zero of=test.img bs=128k count=10000 && sync"

                    You copy 128k * 10 000  ~1280 MB Speed ~ 32 MB/s

                     

                     

                    But you sync all file systems. So it's not clear do you have any other activity on other file systems.

                    According  man:

                         Unlike fsync(3C), which  completes  the  writing  before  it

                         returns,  sync() schedules but does not necessarily complete

                         the writing before returning.

                     

                    So when you run only one sync, you can not be sure that buffers is real  flushed.

                     

                    Regards,

                    Nik.

                    • 7. Re: copy with "dd" command take long time when specifying block size
                      3260775

                      Hi All,

                       

                      We use a delaylog mount option and raid 10 for T4-4.

                      disk used on STK 2540 ( 7200 rpm)

                       

                      Regards.