Does java.sound add delay to "live" audio streams (vs. native code)?
845492 Mar 23, 2011 11:15 PMFirst, before things get off track: the question isn't about (nor am I implying) Java being slow.
I am developing an application that listens to a "live" audio feed (by "live", I mean it is not stored, but being streamed in real time - as opposed to being a Clip or File that I can manage directly) and performing some processing, driving something like a visualization. I've measured (using eyes, ears, and stopwatch) that there is a constant ~ 0.65sec delay from the "sound event" to the "displayed event". I presumed it was my code being slow. It is unacceptably distracting for the intended use.
I've spent quite some time analyzing, refining, and restructuring my code, without significant improvement. I did a quick "instrumentation" of my code using System.nanoTime() for reference. I realize it's not dead-on accurate, but it should give some good ideas of where time is being spent. The FFT is taking about 20nSec. The graphical rendering (the "paint" method) takes about 1uSec. Loading the process down by throwing about 500 different operations at my processing code results in a total delay (including FFT, painting the window, and my processing) of less than 5 mSec. That delay would be fine.
I was about to give up, thinking that I would not be able to succeed when I found a (commercial) application that does something similar to what I'm doing. I can run it at the same time as my app, listening to the same audio line, and with side-by-side visual comparison I see that the response of the other app is without noticeable delay compared to the audio source, but at least 0.5 sec "ahead" of my grahical output. I don't have source code for the other app, and I don't think it's written in Java (probably native Win32 I'm guessing). However, it does demonstrate that the goal is achievable.
At this point, there are two (java.sound) explanations I can come up with (and one Swing-related explanation):
1) To provide the very simple stream interface (just filling an array from a TargetDataLine and then processing it) java.sound is adding some pipeline delay in the audio stream.
2) I should be listening to some other Line (although there is no other Line that I can find that makes sense - unless I can listen directly to a Port).
3) The delay isn't actually in the audio section, but in the graphic rendering section. I realize that's beyond the scope of this sub-forum. All I can think of is that Swing is introducing an enormous delay between the time I (indirectly - through a <customJPanel>.repaint() call for a redraw of my window. I can measure the actual time required for the my custom paint() method to complete, but not the latency between the time the repaint() call is made and the Swing thread dispatches that call to the actual paint() method which does the work.
I would appreciate any help others might be able to share with this. Thanks.
[Addendum: I just tried replacing the JPanel.repaint() call (deferred to the Swing EDT) with JPanel.paintImmediately() and there was no change in the delay.]
Edited by: ags on Mar 23, 2011 4:14 PM
Edited by: ags on Mar 23, 2011 4:15 PM
I am developing an application that listens to a "live" audio feed (by "live", I mean it is not stored, but being streamed in real time - as opposed to being a Clip or File that I can manage directly) and performing some processing, driving something like a visualization. I've measured (using eyes, ears, and stopwatch) that there is a constant ~ 0.65sec delay from the "sound event" to the "displayed event". I presumed it was my code being slow. It is unacceptably distracting for the intended use.
I've spent quite some time analyzing, refining, and restructuring my code, without significant improvement. I did a quick "instrumentation" of my code using System.nanoTime() for reference. I realize it's not dead-on accurate, but it should give some good ideas of where time is being spent. The FFT is taking about 20nSec. The graphical rendering (the "paint" method) takes about 1uSec. Loading the process down by throwing about 500 different operations at my processing code results in a total delay (including FFT, painting the window, and my processing) of less than 5 mSec. That delay would be fine.
I was about to give up, thinking that I would not be able to succeed when I found a (commercial) application that does something similar to what I'm doing. I can run it at the same time as my app, listening to the same audio line, and with side-by-side visual comparison I see that the response of the other app is without noticeable delay compared to the audio source, but at least 0.5 sec "ahead" of my grahical output. I don't have source code for the other app, and I don't think it's written in Java (probably native Win32 I'm guessing). However, it does demonstrate that the goal is achievable.
At this point, there are two (java.sound) explanations I can come up with (and one Swing-related explanation):
1) To provide the very simple stream interface (just filling an array from a TargetDataLine and then processing it) java.sound is adding some pipeline delay in the audio stream.
2) I should be listening to some other Line (although there is no other Line that I can find that makes sense - unless I can listen directly to a Port).
3) The delay isn't actually in the audio section, but in the graphic rendering section. I realize that's beyond the scope of this sub-forum. All I can think of is that Swing is introducing an enormous delay between the time I (indirectly - through a <customJPanel>.repaint() call for a redraw of my window. I can measure the actual time required for the my custom paint() method to complete, but not the latency between the time the repaint() call is made and the Swing thread dispatches that call to the actual paint() method which does the work.
I would appreciate any help others might be able to share with this. Thanks.
[Addendum: I just tried replacing the JPanel.repaint() call (deferred to the Swing EDT) with JPanel.paintImmediately() and there was no change in the delay.]
Edited by: ags on Mar 23, 2011 4:14 PM
Edited by: ags on Mar 23, 2011 4:15 PM