9 Replies Latest reply: Mar 21, 2011 1:37 PM by 845492 RSS

    How do I select ports for capturing audio?

    845492
      Summary:

      I am working on a project that needs to programmatically query the sound system and present the user with a list of choices of input lines (Port) to be captured (just one input at any time), with the ability for the user to control the volume of the selected input line (Port).

      It seems that either:
      a) I'm not querying the sound system correctly to get the information needed to do what I want
      b) I don't understand the way the system is intended to present a logical description of the sound system
      c) My system is bizarre
      d) The (only) way I can think of to make this work is actually the correct way - and will be subject to failure on different machines.

      I hope it's either option a) or b) - I can fix that; c) is a nuisance, but d) is a worry.

      Discussion:

      What I've found is that all 3 source Ports reported by AudioSystem queries are associated with Mixers that do not support TargetDataLines (each such Mixer presents a description of "Port Mixer"). It appears for each of the three (source) Port Mixers on my system, there is another logically distinct Mixer (each such Mixer presents a description of "Direct Audio Device: DirectSound Capture") that is somehow connected to the source Port Mixer. The only thing that I can find that indicates any connection between the two Mixers is the naming of the two Mixers: the Port Mixer is named "Port <someName>" and the Direct Audio Mixer is named "<someName>" (without the "Port" prefix). I confirm this logical connection between the two mixers by noting that when I read from a TargetDataLine on the Direct Audio Mixer I am capturing the audio present on the source Port of the similarly-named Port Mixer.

      I'd be happy to just use the TargetDataLine from the Direct Audio Mixer and ignore the Port Mixer, but there are no controls on the former and I need to present a volume/gain control to the user, which only exists on the Ports of the latter.

      Question:

      Is it correct, and portable, to assume that there is this apparent 1:1 relationship between a mixer named "<someName>" and another mixer named "Port <someName>"? I have read the entire javax.sound.sampled tutorial, several times. I have read the often-recommended article at http://www.vsj.co.uk/java/display.asp?id=370
      several times and while I believe I understand the concepts being presented, the configuration I have doesn't seem to match what is described there.

      I've written code to query the audio system on my machine (Windows Vista) and the (shortened) output is below. Java version info:

      java version "1.6.0_24"
      Java(TM) SE Runtime Environment (build 1.6.0_24-b07)
      Java HotSpot(TM) Client VM (build 19.1-b02, mixed mode, sharing)

      If there is some way to make this question better so as not to require an unreasonable amount of effort from anyone kind enough to answer just let me know.

      Thanks.

      -----
      AudioSystem Info::
      
        Source Ports: 3
           LineInfo: Master Volume source port
             Port is a SOURCE
             Controls: 1
           LineInfo: LINE_IN source port
             Port is a SOURCE
             Controls: 1
           LineInfo: Master Volume source port
             Port is a SOURCE
             Controls: 1
      
        Target Ports: 5
           LineInfo: SPEAKER target port
             Port is a TARGET
             Controls: 4
           LineInfo: Master Volume target port
             Port is a TARGET
             Controls: 4
           LineInfo: Master Volume target port
             Port is a TARGET
             Controls: 4
           LineInfo: Master Volume target port
             Port is a TARGET
             Controls: 4
           LineInfo: Master Volume target port
             Port is a TARGET
             Controls: 4
      
      Mixer Info::
      
      Found 13 mixers
      Mixer Name: Primary Sound Driver
        Description: Direct Audio Device: DirectSound Playback
        Controls: NONE
        Source Lines: 2
           LineInfo: interface SourceDataLine supporting 8 audio formats, and buffers of at least 32 bytes
             Controls: 4
           LineInfo: interface Clip supporting 8 audio formats, and buffers of at least 32 bytes
             Controls: NONE
        Target Lines: NONE
      
      Mixer Name: Speakers (VIA High Definition Audio)
        Description: Direct Audio Device: DirectSound Playback
        Controls: NONE
        Source Lines: 2
           LineInfo: interface SourceDataLine supporting 8 audio formats, and buffers of at least 32 bytes
             Controls: 4
           LineInfo: interface Clip supporting 8 audio formats, and buffers of at least 32 bytes
             Controls: NONE
        Target Lines: NONE
      
      Mixer Name: SPDIF Interface (VIA High Definition Audio)
        Description: Direct Audio Device: DirectSound Playback
        Controls: NONE
        Source Lines: 2
           LineInfo: interface SourceDataLine supporting 8 audio formats, and buffers of at least 32 bytes
             Controls: 4
           LineInfo: interface Clip supporting 8 audio formats, and buffers of at least 32 bytes
             Controls: NONE
        Target Lines: NONE
      
      Mixer Name: Primary Sound Capture Driver
        Description: Direct Audio Device: DirectSound Capture
        Controls: NONE
        Source Lines: NONE
        Target Lines: 1
           LineInfo: interface TargetDataLine supporting 8 audio formats, and buffers of at least 32 bytes
             Controls: NONE
      
      Mixer Name: Line In (VIA High Definition Au
        Description: Direct Audio Device: DirectSound Capture
        Controls: NONE
        Source Lines: NONE
        Target Lines: 1
           LineInfo: interface TargetDataLine supporting 8 audio formats, and buffers of at least 32 bytes
             Controls: NONE
      
      Mixer Name: Stereo Mix (VIA High Definition
        Description: Direct Audio Device: DirectSound Capture
        Controls: NONE
        Source Lines: NONE
        Target Lines: 1
           LineInfo: interface TargetDataLine supporting 8 audio formats, and buffers of at least 32 bytes
             Controls: NONE
      
      Mixer Name: CD Audio (VIA High Definition A
        Description: Direct Audio Device: DirectSound Capture
        Controls: NONE
        Source Lines: NONE
        Target Lines: 1
           LineInfo: interface TargetDataLine supporting 8 audio formats, and buffers of at least 32 bytes
             Controls: NONE
      
      Mixer Name: Java Sound Audio Engine
        Description: Software mixer and synthesizer
        Controls: 1
        Source Lines: 2
           LineInfo: interface SourceDataLine supporting 8 audio formats
             Controls: 4
           LineInfo: interface Clip supporting 8 audio formats, and buffers of 0 to 4194304 bytes
             Controls: 4
        Target Lines: NONE
      
      Mixer Name: Port Speakers (VIA High Definition A
        Description: Port Mixer
        Controls: NONE
        Source Lines: NONE
        Target Lines: 1
           LineInfo: SPEAKER target port
             Port is a TARGET
             Controls: 4
      
      Mixer Name: Port SPDIF Interface (VIA High Defin
        Description: Port Mixer
        Controls: NONE
        Source Lines: NONE
        Target Lines: 1
           LineInfo: Master Volume target port
             Port is a TARGET
             Controls: 4
      
      Mixer Name: Port Stereo Mix (VIA High Definition
        Description: Port Mixer
        Controls: NONE
        Source Lines: 1
           LineInfo: Master Volume source port
             Port is a SOURCE
             Controls: 1
        Target Lines: 1
           LineInfo: Master Volume target port
             Port is a TARGET
             Controls: 2
      
      Mixer Name: Port Line In (VIA High Definition Au
        Description: Port Mixer
        Controls: NONE
        Source Lines: 1
           LineInfo: LINE_IN source port
             Port is a SOURCE
             Controls: 1
        Target Lines: 1
           LineInfo: Master Volume target port
             Port is a TARGET
             Controls: 2
      
      Mixer Name: Port CD Audio (VIA High Definition A
        Description: Port Mixer
        Controls: NONE
        Source Lines: 1
           LineInfo: Master Volume source port
             Port is a SOURCE
             Controls: 1
        Target Lines: 1
           LineInfo: Master Volume target port
             Port is a TARGET
             Controls: 2
      Edited by: user8620608 on Mar 7, 2011 9:51 PM

      Edited by: ags on Mar 8, 2011 9:55 PM
      Shortened code output. Added Exec summary. Trying to make it easy to help.
        • 1. Re: How do I select ports for capturing audio?
          captfoss
          It seems that either:
          a) I'm not querying the sound system correctly to get the information needed to do what I want
          b) I don't understand the way the system is intended to present a logical description of the sound system
          c) My system is bizarre
          d) The (only) way I can think of to make this work is actually the correct way - and will be subject to failure on different machines.

          I hope it's either option a) or b) - I can fix that; c) is a nuisance, but d) is a worry.
          Working through this logically...

          SourceDataLines are things the user can read from, and TargetDataLines are things the user can write to...

          You have SourceDataLines associated with the Speakers and the SPDIF Interface... both of those are outputs, but I think it's safe to assume those two "mixers" are actually loopback interfaces to monitor the outputs.

          You have TargetDataLines associated with the CD Audio and Port Line In... both of those are inputs, and writing to them doesn't really make any sense...

          It looks like your system is "being helpful" by providing a loopback interface (Master Volume target port) with each output port...

          My best guess is that all of the ports with just source lines feed into a common area ("Master Volume") and then all feed out of that common area into a series of mixers that have a loopback SourceDataLine you can monitor the output on and a TargetDataLine that you can write to... and then they leave the system through whatever hardware they are associated with.

          If that's not the case, then it's the case that you have multiple parallel paths through your Sound System, and you can "write" to things that should be inputs because those inputs have their own mixers.

          Is it a safe bet that it's a 1:1 mapping? Well, I think it's a safe bet that either that's the case, or, it doesn't matter. You could run into problems down the line doing it that way, but, I'd recommend you:
          1) Handle a normal system (system with a single port mixer) in the normal way
          2) Handle a funky system like yours by assuming "Mixer <Source>" will give you the output of "Port <Source>"...

          Because, frankly, I think that "Mixer <Source>" should give you the output of all of the "Port <Source>" that are enabled... or it'll just give you the output of the one Port. Either way, you should be fine.
          • 2. Re: How do I select ports for capturing audio?
            captfoss
            As a sidenote... another thread poped up
            Re: Loopback recording in Windows7 and Vista

            giving the link
            http://en.wikipedia.org/wiki/Sound_card_mixer

            which talks about the way Windows Vista / 7 handles loopbacks. Apparently the new way of doing it is "every playback device can be opened in loopback recording mode, so you can record what you hear from your speakers, headphones, digital output etc"...

            So, that kind of ties both of our theories together... every port object on Windows Vista/7 should have its own associated Mixer... but that's only going to be true sometimes... and I'd expect the output of one to be the the output of the others under normal circumstances, but I suppose it's now possible to do things like, mix your voice into an audio conference call on the headphones but not on the speakers...
            • 3. Re: How do I select ports for capturing audio?
              845492
              captfoss wrote:SourceDataLines are things the user can read from, and TargetDataLines are things the user can write to...
              Isn't Java Sound terminology "backwards" - at least from the point of view of the user (but not from the point of view of the mixer)?

              My understanding is that a SourceDataLine is an input line to the mixer - typically an output from an application. I would write to a SourceDataLine to send audio to a speaker for playback, for instance. Likewise, a TargetDataLine is the output of a mixer - and the input to an application that wants to capture audio. I'm currently capturing audio from a TargetDataLine in my app.

              If my understanding is correct, then the loopback mixers/interfaces aren't needed to explain that part of the mystery - although that concept may still be a factor somewhere.
              • 4. Re: How do I select ports for capturing audio?
                captfoss
                ags wrote:
                captfoss wrote:SourceDataLines are things the user can read from, and TargetDataLines are things the user can write to...
                Isn't Java Sound terminology "backwards" - at least from the point of view of the user (but not from the point of view of the mixer)?

                My understanding is that a SourceDataLine is an input line to the mixer - typically an output from an application. I would write to a SourceDataLine to send audio to a speaker for playback, for instance. Likewise, a TargetDataLine is the output of a mixer - and the input to an application that wants to capture audio. I'm currently capturing audio from a TargetDataLine in my app.
                Yeah, my explanation was backwards. I always do that if I don't explicily look it up, because it is backwards from the perspective of the user.
                If my understanding is correct, then the loopback mixers/interfaces aren't needed to explain that part of the mystery - although that concept may still be a factor somewhere.
                So yeah, everything I said was backwards. It makes sense that you'd have SDLs for your outputs and TDLs for your inputs...

                But I think the "Mixer" objects you're seeing is a result of the new way Windows Vista / 7 handles the loopbacks for all of the devices... so I'd think you'd be safe to handle normal situations normally (single port mixer) and then the special situations (multiple port mixers) using the names...
                • 5. Re: How do I select ports for capturing audio?
                  845492
                  captfoss wrote:
                  But I think the "Mixer" objects you're seeing is a result of the new way Windows Vista / 7 handles the loopbacks for all of the devices... so I'd think you'd be safe to handle normal situations normally (single port mixer) and then the special situations (multiple port mixers) using the names...
                  My problem is that I seem to have "multiple-mixer ports" rather than "multiple-port mixers". My end goal is to query and find all source Ports on the system (with (VOLUME) control) and present that list to users for selection of audio source. So I need to end up with a list of source Ports, their VOLUME control, and a TargetDataLine from which I can read the audio.

                  As it turns out, my system has several Mixers (each named "Port <someName>") that have exactly one source Port and one target Port. Not a TargetDataLine, but a target Port. I can't read from a target Port. For each of these Mixers with a source Port, there is another unique Mixer (named "<something>" - without the "Port" prefix - but otherwise it's name is identical to it's "sibling") that has no source lines of any kind, and exactly one TargetDataLine. I found that I can read the audio data from that TargetDataLine that corresponds to the audio source that I know is present on the source Port of the other (seemingly unconnected/unrelated/unassociated - other than the similarity in the naming convention) Mixer.

                  That seems really strange to me and was ultimately the reason I reached out for some help with this. I can't even just give myself an easy out with "it must be a corner-case weird system and no application would work properly with it" - because I have a generic app that I've been testing (I don't know which language it's written in) that happily does just what I've set out to do, with no problem.

                  Thanks.
                  • 6. Re: How do I select ports for capturing audio?
                    captfoss
                    My problem is that I seem to have "multiple-mixer ports" rather than "multiple-port mixers".
                    Each input port (Port Line In, Port CD Audio, Port Stereo Mix) has a source port and a target port...

                    Each output port (Port Speakers, Port SPDIF Interface) has just a target port...

                    Notice that each of the target ports on your inputs is listed as the Master Volume target port...

                    So it seems to be like each input gives you a place to read from the port (the master volume target port) and a place to input mix the port (the associated source ports, aka the loop forward)...

                    And each output port is giving you a place to read from the port (the loop back)...

                    So your inputs all have loop forwards, and your outputs all have loop backs. Am I misinterpriting something here?
                    • 7. Re: How do I select ports for capturing audio?
                      845492
                      captfoss wrote:

                      Each input port (Port Line In, Port CD Audio, Port Stereo Mix) has a source port and a target port...

                      Each output port (Port Speakers, Port SPDIF Interface) has just a target port...

                      Notice that each of the target ports on your inputs is listed as the Master Volume target port...

                      So it seems to be like each input gives you a place to read from the port (the master volume target port) and a place to input mix the port (the associated source ports, aka the loop forward)...

                      And each output port is giving you a place to read from the port (the loop back)...

                      So your inputs all have loop forwards, and your outputs all have loop backs. Am I misinterpriting something here?
                      Yes, I see what you are saying. However, unless I'm missing something, I still am stuck with having to make the assumption that a mixer named <someName> is logically connected to a port mixer named "Port <someName>" based solely on their names.

                      You correctly note that the input port Mixers (e.g. "Port Line In") have a source Port (in this case the actual analog line-in line, I presume) and a target Port. The problem is that I can't capture audio from a target Port, I need a TargetDataLine. If I understand correctly, while everything (Mixer,Port,DataLine,SourceDataLine,TargetDataLine) derives from the Line class, digital capture can only be accomplished using a TargetDataLine. So while there is indeed a target Port on the "Port Line In" Port Mixer, there is no directly-attached TargetDataLine on that Mixer from which I can fill a buffer with the digitized audio content.

                      If I'm missing something (which I hope is the case) I'd welcome being corrected. Otherwise, the only thing I've been able to do so far to make this work is associate the two Mixers ("Port Line In" and "Line In") based on their names - which causes me concern for lack of portability and robustness.

                      Thanks.
                      • 8. Re: How do I select ports for capturing audio?
                        captfoss
                        I still am stuck with having to make the assumption that a mixer named <someName> is logically connected to a port mixer named "Port <someName>" based solely on their names.
                        Yeah, I suppose that's true. I mean, you could always run a quick correlation test between the mixer you can read from ("Line In") and the port mixer you think is associated ("Port Line In"). Disable the source port on the port mixer, and your TDL should either start spitting out zeros or run out of data...
                        Otherwise, the only thing I've been able to do so far to make this work is associate the two Mixers ("Port Line In" and "Line In") based on their names - which causes me concern for lack of portability and robustness.
                        The lack of robustness could be handled by the above correlation test...

                        The lack of portability is handled by considering this scenerio a fallback sitations.

                        Is this a normal situation? Handle it...
                        Is this a freaky THING / PORT THING situation? Handle it...

                        But yeah, you're probably not going to ever look at your code and be like "Glad I handled that the *right* way..."
                        • 9. Re: How do I select ports for capturing audio?
                          845492
                          captfoss wrote:
                          I still am stuck with having to make the assumption that a mixer named <someName> is logically connected to a port mixer named "Port <someName>" based solely on their names.
                          Yeah, I suppose that's true. I mean, you could always run a quick correlation test between the mixer you can read from ("Line In") and the port mixer you think is associated ("Port Line In"). Disable the source port on the port mixer, and your TDL should either start spitting out zeros or run out of data...
                          Otherwise, the only thing I've been able to do so far to make this work is associate the two Mixers ("Port Line In" and "Line In") based on their names - which causes me concern for lack of portability and robustness.
                          The lack of robustness could be handled by the above correlation test...

                          The lack of portability is handled by considering this scenerio a fallback sitations.

                          Is this a normal situation? Handle it...
                          Is this a freaky THING / PORT THING situation? Handle it...

                          But yeah, you're probably not going to ever look at your code and be like "Glad I handled that the *right* way..."
                          Ug. Well, thanks for the responses. I was hoping that this couldn't possibly be the "correct" way to handle this case (even if it's uncommon).

                          You are correct that I doubt I'll look at this code and like it - but I'll feel better that the experts here checked my work and couldn't offer a better solution.

                          Thanks.