3 Replies Latest reply: Jan 16, 2013 4:25 AM by 800381 RSS

    Long pause at the end of DTrace script.

    calebtng
      Anyone else experience this? When my DTrace scripts end, it is a very long time -- around 5 minutes -- before I get my command prompt back. During this time, other users find their commands hang as well. What is going on? I would like to use DTrace to determine what DTrace is doing, but I fear a collapse of the space-time continuum.

      This does not happen on all our server, just some of the larger Oracle database servers.
        • 1. Re: Long pause at the end of DTrace script.
          800381
          While that's happening, run this DTrace script:
          #!/usr/sbin/dtrace -s
          
          #pragma D option quiet
          
          profile:::profile-1001hz
          / arg0 /
          {
              @addr[ arg0 ] = count();
          }
          
          dtrace:::BEGIN
          {
              printf( "Begin\n" );
          }
          
          dtrace:::END
          {
              printa( "%@u %a\n", @addr );
          }
          Start that right before you end your other DTrace script. You might want to redirect the output of that script to a file. Let that run for a good 20 or 30 seconds, break it. If you can't break it, don't worry because it'll just be collecting more data about what's going on and you can break it when your system starts responding again.

          FWIW, I suspect a memory paging issue of some sort. So, it would help to see the output from:
          echo ::memstat | mdb -k
          • 2. Re: Long pause at the end of DTrace script.
            sir_toby
            You can also go for the full stack...

            profile:::profile-1001hz
            /arg0/
            {
            @astack[stack()] = count();
            }

            ...and print it with

            printa(@astack);

            Oracle DBMS? DSGA, DISM? Up to several minutes? Solaris 10 or Solaris 11.0? I guess it's page coalescing of small pages in order to form large ones.
            • 3. Re: Long pause at the end of DTrace script.
              800381
              sir_toby wrote:
              You can also go for the full stack...

              profile:::profile-1001hz
              /arg0/
              {
              @astack[stack()] = count();
              }

              ...and print it with

              printa(@astack);

              Oracle DBMS? DSGA, DISM? Up to several minutes? Solaris 10 or Solaris 11.0? I guess it's page coalescing of small pages in order to form large ones.
              That's what I was thinking, although seemingly getting triggered by stopping a DTrace script does make for other possibilities. I've seen similar problems before when literally hundreds of Oracle parallel jobs processes would all time out and exit simultaneously.