1 Reply Latest reply on Oct 22, 2011 8:44 PM by 895592

    Recording real mouse


      We are developing an application that records the screen and voice of a remote user while being in a site, using Java Web Start. The way is set up now, the java web start launches the site to be analyzed in a browser window (not inside the Java window, because we need to show the browser bar and functionality).

      The problem is that we cannot record the mouse to see:
      - how the user moves the cursor in the screen
      - where the user clicks
      - be able to notice when something is clickable (the mouse changes from arrow to hand)
      - count the number of clicks

      1. Do you know if there is any way to accomplish this, and record all this mouse information even if the website is not launched inside the Java window?
      2. If we cannot record the real mouse, can we track the clicks?
      3. http://java.sun.com/developer/technicalArticles/GUI/translucent_shaped_windows/ Can we use a full transparent window and track the mouse clicks while the user is surfing the website?
      4. Any other suggestion to accomplish this?

      Thank you very much for your help.
        • 1. Re: Recording real mouse
          If I understand the requirements correctly, you want to track mouse movement inside a browser window. One possible approach I can think of would be to think about looking at tracking mouse movements in the browser instead.

          Here is the first kind of approach that popped into my head:
          1.) Embed a web server into your app (such as Jetty, or something even smaller, simpler - use the built in Sun one).
          2.) Inject Javascript into the webpage content (onmouseover on 'body' tag I think should do it) to track mouse movement.
          3.) Have the Javascript make calls (say for example by inserting an image tag in the DOM that makes a call to your embedded server <img src="track.gif?x=142&y=399">

          There might be a better way to do this of course, but the point is tracking mouse movements with JavaScript in the browser and communicating the position to you Java process 'might' be the easiest approach.