0 Replies Latest reply on May 8, 2014 3:21 PM by msncookie

    ZFS SA 7320 NFS throughput seems very slow

    msncookie

      We have a clustered 7320, 2xCPU (2.40GHz), 96GB RAM, one large 2T mirrored pool (18 disks on DE2-24P) and one project to be used for Oracle DB, NFS shares.

      Two Solaris T4-1 hosts direct/x-over connected to 7320's via 10GbE fiber.

       

      We haven't done enough testing with Oracle DB yet (which will be using dNFS) however when I do a simple large file copy from the hosts to an NFS share on the 7320, I'm getting no more than 20MB/s write speed (at least according to rsync --progress) and ~54MB/s read.  For a 10GbE connection (negotiated 10,000-Full, no netstat errors, collisions, etc) this seems awfully slow. 

       

      We are using the Oracle DB recommended mount options, i.e.:

       

      rw,bg,hard,nointr,rsize=1048576,wsize=1048576,proto=tcp,noac,forcedirectio,vers=3,suid

       

      Tried removing forcedirectio and that made throughput plummet to about 2MB/s!  I've also tried following Doc ID 1213725.1 and gave this a try in /etc/system:

       

      set rpcmod:clnt_max_conns = 8

       

      ...with no noticeable change.  The only thing I'm curious about in this doc is our lack of logzillas - our installer did not deem it necessary I guess.  Our DB load will be more read heavy, but my main concern right now is the paltry 20MB/s write speed and I would like to focus on getting that corrected before worrying about Oracle DB performance.

       

      Thanks!