5 Replies Latest reply: Apr 29, 2013 6:10 PM by 1006219 RSS

    ZFS arc size

    847658
      I have Solaris server running with 32G memory and i set to limit ZFS cache to 4G on /etc/system. But when i run memstat|mdb, zfs file data showing zfs cache usage as 5G? Why its growing more than the value that i set on /etc/system?


      Page Summary Pages MB %Tot

      ZFS File Data 673674 5263 16%

      Edited by: Solaris72 on Mar 21, 2012 7:48 AM
        • 1. Re: ZFS arc size
          SteveS
          You did reboot after adding the entry to /etc/system right?

          Can you provide the zfs_arc_max entry from your /etc/system file please. I just want to make sure you have the correct value (in bytes or hex)

          What values do you see in kmastat for "zfs_file_data"

          # echo "::kmastat" | mdb -k | grep -i zfs_file_data

          You can look through the kmastat output in more detail by removing the grep.

          The following will display further details about your ARC

          # echo "::arc" | mdb -k
          # kstat -m zfs -p
          • 2. Re: ZFS arc size
            847658
            I don't think my entry is wrong on /etc/system as i see available memory 28+G with no application running

            set zfs:zfs_arc_max=4294967296

            Based on my initial post, do my arc usage exceeds 4G or I'm reading that wrong (zfs_file_data)?

            Edited by: Solaris72 on Mar 21, 2012 1:08 PM
            • 3. Re: ZFS arc size
              SteveS
              Solaris72 wrote:
              I don't think my entry is wrong on /etc/system as i see available memory 28+G with no application running

              set zfs:zfs_arc_max=4294967296
              That value is correct for 4G. It's worth confirming because I see quite a lot of people making mistakes with the numbers and then wonder why things don't add up or work as expected. The value is only read at boot time as part of the call to arc_init() and can't be changed on the fly.
              Based on my initial post, do my arc usage exceeds 4G or I'm reading that wrong (zfs_file_data)?
              "ZFS File Data" isn't the size of the ARC. Using the kstat or mdb commands I provided in the previous post you can check the ARC in more detail:

              $ kstat -p zfs:0:arcstats
              ...
              zfs:0:arcstats:c     1005826786 <--------------- Target size of the ARC
              zfs:0:arcstats:c_max     1073741824 <-------- Maximum size of the ARC. Tunable in /etc/system using 'zfs_arc_min'
              zfs:0:arcstats:c_min     67108864 <--------- Minimum size of the ARC. Tunable in /etc/system using 'zfs_arc_max'
              ...
              zfs:0:arcstats:p     1005826786 <--------------- Target size of the MRU (Most Recently Used) cache within the ARC
              ...
              zfs:0:arcstats:size     999490496 <---------------- Current size of the ARC
              ...
              • 4. Re: ZFS arc size
                847658
                Thanks Steve. That answers my question. Here is my kstat output

                zfs:0:arcstats:c 4294967296 --> 4G
                zfs:0:arcstats:c_max 4294967296 --> 4G
                zfs:0:arcstats:c_min 1054042368 --> 1G
                zfs:0:arcstats:p 1821141504 --> 1G
                zfs:0:arcstats:size 4294643120 -->4G

                I'm planning to increase zfs cache to 8G, to make sure.
                • 5. Re: ZFS arc size
                  1006219
                  Hi Steve

                  I keep bumping into this thread so I'll just step in and ask.

                  When you say Zfs file data isn't Arc I understand since I cleary see it on my server using kstat.

                  The problem is that is "eats" 5GB of memory I wan't to have, which means I have an arc max size of 8GB and a Zfs file data of 13GB reported by memstat...

                  And I just can't understand what does it represent or find in piles of documentation, so I can try to limit it.

                  What is it?

                  Fyi, I'm running Solaris 11.1

                  Thanks
                  Nuno Abrantes