Forum Stats

  • 3,825,934 Users
  • 2,260,581 Discussions
  • 7,896,738 Comments

Discussions

DTrace stuck when listing probes with "-P" and "-p"

GregoryG.
GregoryG. Member Posts: 36
edited Sep 11, 2013 3:50PM in Oracle Linux and UEK Preview

Hello,

To provide some other feedback with UEK3/Dtrace 0.4 Beta1

1/- I've built a small USDT sample:

demo_probes.d

provider demo {
 probe progress__counter(int);
};

demo.c

#include <sys/sdt.h>

/*
 ** USDT probes with DTrace and UEK3 beta1
 **/

int main(int argc, char *argv[]) {
   int i=0;

   while (1) {
      sleep(1); 
      i++; 
      DTRACE_PROBE1(demo, progress__counter, i);
   }
}

2/- When I run it and use -p and -l together, it works well:

dtrace -p 7276 -l |grep demo
  803   demo7276              demo                              main progress-counter

3/- When I use -p -P and -l together, the command get stuck:

dtrace -p 7276 -P 'demo7276' -l

In this case, running strace on dtrace shows:

strace -p 9018
Process 9018 attached - interrupt to quit
futex(0x101d3dc, FUTEX_WAIT_PRIVATE, 1, NULL

Best Regards

Gregory

Best Answer

  • NickAlcock
    NickAlcock Member Posts: 4
    Answer ✓

    The testcases revealed another bug (not the same bug, though its symptoms are outwardly identical): dtrace -p $some_pid -P a-provider-without-wildcards hung.

    That needs a rather more invasive fix, but it cleans up the code a good bit and fixes a bunch of other possible hangs at the same time. (And it's done, and waiting for the next release.)

Answers

  • GregoryG.
    GregoryG. Member Posts: 36

    Hello,

    Sorry, -p doesn't seem useful when used in conjunction -l. I'm not sure if that's expected of not.

    However, the -P blocks the -l command which looks like a wrong behavior:

    dtrace -P demo7276 -l

    Simple doesn't return

    Gregory

  • Note that dtrace -P 'demo*' -l works fine, as does dtrace -m demo -l: it's only non-wildcarded lookups by provider that block forever. Even -P 'demo7276*' works, i.e. a wildcard that expands only to this one thing. (I note that all the tests for this case in the DTrace testsuite we imported from Solaris are intended to be manually executed: I should automate them to ensure this never happens again.)

    Looking into the cause now.

  • Diagnosed: we attempt to explicitly lock the dpr_lock() when creating pid and usdt probes, which is prohibited since it confuses the machinery by which we make a recursive lock out of a non-recursive one that we can use in condition variables. Thus we fail to realize the lock is already taken and soon end up double-locking a non-recursive mutex, and a hang ensues.

    Fix trivial (already done, but coming up with some proper testcases for this too before releasing).

  • GregoryG.
    GregoryG. Member Posts: 36

    Nick,

    This wasn't that obvious for me ; despite the source code ;-).

    Thank you for taking the feedback into account...

    Gregory

  • NickAlcock
    NickAlcock Member Posts: 4
    Answer ✓

    The testcases revealed another bug (not the same bug, though its symptoms are outwardly identical): dtrace -p $some_pid -P a-provider-without-wildcards hung.

    That needs a rather more invasive fix, but it cleans up the code a good bit and fixes a bunch of other possible hangs at the same time. (And it's done, and waiting for the next release.)

This discussion has been closed.