2 Replies Latest reply: Aug 18, 2010 2:56 AM by 843802 RSS

    JSAPI Recognition won't run twice

    843802
      I'm working on an implementation of JSAPI's Recognition. 'listen' method is invoked through a web service and it runs fine for the first time. When I run it for a second time it looks like I've got two engines allocated (though I deallocate it) because everything is displayed 2 times. Then the app crashes and blocks a port. When it is run for the third time I displays everything three times and an exception is thrown due to a blocked port. I tried to use both the same Recognizer object for every 'listen' method call and creating a new object for every invocation (the output below is the second case). It doesn't help either when I use different ports for other invocations. When I restart the server it again works only once. So what I think the problem is I am not deallocating or closing sources properly. I think it is JSAPI problem but could also be RTP. Please help.

      I use Cloud Garden's JSAPI implementation.

      Here is the code:
      package asr;
      
      import javax.speech.*;
      import javax.speech.recognition.*;
      
      import com.cloudgarden.audio.*;
      import com.cloudgarden.speech.*;
      
      import javax.media.protocol.*;
      import javax.media.*;
      import javax.media.format.*;
      
      public class RecognitionEngine {
      
           public RecognitionEngine(String recognitionEngineName) {
      
           }
      
           /**
            * TODO: synchronizacja result?
            * 
            * @return
            */
           public String listen(String destination, String port) {
                ++counter;
                System.out.println("Times called: " + counter);
                
                result = null;
      
                try {
                     String rtpRecoUrl = "rtp://" + destination + ":" + port + "/audio";
                     reco = Central.createRecognizer(new EngineModeDesc(null, null,
                               java.util.Locale.ENGLISH, null));
      
                     reco.addEngineListener(new TestEngineListener());
      
                     reco.allocate();
                     reco.waitEngineState(reco.ALLOCATED);
      
                     CGAudioManager recoAudioMan = (CGAudioManager) reco.getAudioManager();
                     recoAudioMan.addAudioListener(new TestAudioListener());
                     recoAudioMan.addTransferListener(new TransferListener() {
                          public void bytesTransferred(TransferEvent e) {
                               System.out.print("R" + e.getLength() + " ");
                          }
                     });
      
                     boolean gramExists = false;
                     for (RuleGrammar r : reco.listRuleGrammars()) {
                          if (r.getName().equals("gram1"))
                               gramExists = true;
                     }
                     if (!gramExists) {
                          RuleGrammar gram = reco.newRuleGrammar("gram1");
                          Rule r = gram
                                    .ruleForJSGF("all your base are belong to us {BASE} | hello world {HELLO} | goodbye computer {QUIT} | what time is it {TIME} | what date is it {DATE}");
                          gram.setRule("rule1", r, true);
                          gram.setEnabled(true);
                     }
      
                     // must be called before recognizer is resumed (for SAPI5 engines)!
                     reco.getRecognizerProperties().setResultAudioProvided(true);
      
                     recoDataSink = recoAudioMan.getDataSink();
                     reco.commitChanges();
                     reco.requestFocus();
                     reco.resume();
                     reco.waitEngineState(reco.LISTENING);
                     reco.addResultListener(new ResultAdapter() {
                          public void resultUpdated(ResultEvent ev) {
                               System.out.println("updated " + ev);
                          }
      
                          public void resultAccepted(ResultEvent ev) {
                               System.out.println("accepted " + ev);
                               FinalResult res1 = (FinalResult) ev.getSource();
                               FinalRuleResult frr = (FinalRuleResult) res1;
                               String[] tags = frr.getTags();
                               if (tags == null)
                                    return;
                               String tag = tags[0];
      
                               // Uncomment this to hear what recognizer heard.
                               // try {
                               // reco.pause();
                               // frr.getAudio().play();
                               // reco.resume();
                               // } catch (Exception e) {
                               // e.printStackTrace();
                               // }
                               //
      
                               if (tag.equals("QUIT")) {
                                    System.out.println("Someone said goodbye computer");
                               } else if (tag.equals("HELLO")) {
                                    System.out.println("Someone said hello world");
                               } else if (tag.equals("TIME")) {
                                    System.out.println("Someone said what time is it");
                               } else if (tag.equals("DATE")) {
                                    System.out.println("Someone said what date is it");
                               } else if (tag.equals("BASE")) {
                                    System.out.println("ALL YOUR BASE ARE BELONG TO US!!");
                               }
                               result = tag;
                               close();
                          }
                     });
      
                     javax.sound.sampled.AudioFormat fmt = recoAudioMan.getAudioFormat();
      
                     FileTypeDescriptor cd = new FileTypeDescriptor(FileTypeDescriptor.RAW);
                     Format[] outFormats = { new AudioFormat(AudioFormat.LINEAR,
                               fmt.getFrameRate(), fmt.getSampleSizeInBits(),
                               fmt.getChannels(), AudioFormat.LITTLE_ENDIAN,
                               AudioFormat.SIGNED) };
                     System.out.println("DataSink = " + recoDataSink + " rtpRecoUrl="
                               + rtpRecoUrl);
      
                     ProcessorModel pm = new ProcessorModel(
                               new MediaLocator(rtpRecoUrl), outFormats, cd);
      
                     listening = true;
      
                     recoProc = Manager.createRealizedProcessor(pm);
                     System.out.println("RTP connection established");
      
                     recoDataSink.setSource(recoProc.getDataOutput());
                     recoProc.start();
      
                     recoDataSink.open();
                     recoDataSink.start();
      
                     System.out.println("listening");
      
                     reco.waitEngineState(reco.DEALLOCATED);
      
                     System.out.println("EXITING");
                } catch (Exception e) {
                     e.printStackTrace();
                } finally {
                     listening = false;
                }
                return result;
           }
      
           public void close() {
                System.out.println("closing");
                try {
                     recoProc.stop();
                     recoProc.close(); //mine //unblocks port
                     //recoProc.deallocate(); //mine
                     //recoDataSink.stop(); //mine
                     recoDataSink.close();
      
                     reco.deallocate();
                     reco.waitEngineState(Engine.DEALLOCATED);
      
                     Thread.currentThread().sleep(2000);
                } catch (Exception e2) {
                     e2.printStackTrace();
                }
                System.out.println("closed!");
      
           }
      
           public boolean isListening() {
                return listening;
           }
      
      
      
           private int counter = 0;
           private boolean listening = false;
           private String result;
           private Recognizer reco;
           private Processor recoProc;
           private DataSink recoDataSink;
      }
        • 1. Re: JSAPI Recognition won't run twice
          843802
          output:
          ...
          //server starting
          INFO: Server startup in 6153 ms
          
          INIT!
          Recognizer jest null!
          Times called: 1
          CloudGarden's JSAPI1.0 implementation
          Version 1.7.0
          Implementation contained in files cgjsapi.jar and cgjsapi170.dll
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineAllocatingResources
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineAllocated
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
          com.cloudgarden.speech.CGRecognizer@1fbc355 changesCommitted
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
          com.cloudgarden.speech.CGRecognizer@1fbc355 focusGained
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
          DataSink = com.cloudgarden.audio.CGDataSink@800aa1 rtpRecoUrl=rtp://192.168.56.1:12346/audio
          RTP connection established
          listening
          R1536 R6000 R144 R2304 R2304 R2304 R2304 R2304 R2304 Speech started
          R2304 R2304 R2304 R2304 com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerProcessing
          updated javax.speech.recognition.ResultEvent[source=Grammar:gram1, Conf:3, Tokens{}]
          R2304 R2304 R2304 R2304 R2304 R2304 R2304 R2304 R2304 R2304 updated javax.speech.recognition.ResultEvent[source=Grammar:gram1, Conf:3, Tokens{hello,world}]
          R2304 R2304 Speech stopped
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
          updated javax.speech.recognition.ResultEvent[source=Grammar:gram1, Conf:3, Tokens{hello,world}, Tags{HELLO}]
          accepted javax.speech.recognition.ResultEvent[source=Grammar:gram1, Conf:3, Tokens{hello,world}, Tags{HELLO}]
          Someone said hello world
          closing
          R0 R-1 com.cloudgarden.speech.CGRecognizer@1fbc355 focusLost
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineDeallocatingResources
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineDeallocated
          EXITING
          closed!
          Speech started
          Speech stopped
          
          INIT!
          Recognizer jest null!
          Times called: 1
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineAllocatingResources
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineAllocatingResources
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineAllocated
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineAllocated
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
          com.cloudgarden.speech.CGRecognizer@1fbc355 changesCommitted
          com.cloudgarden.speech.CGRecognizer@1fbc355 changesCommitted
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
          com.cloudgarden.speech.CGRecognizer@1fbc355 focusGained
          com.cloudgarden.speech.CGRecognizer@1fbc355 focusGained
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
          DataSink = com.cloudgarden.audio.CGDataSink@800aa1 rtpRecoUrl=rtp://192.168.56.1:12346/audio
          RTP connection established
          listening
          R-1 R-1 
          
          INIT!
          Recognizer jest null!
          Times called: 1
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
          com.cloudgarden.speech.CGRecognizer@1fbc355 changesCommitted
          com.cloudgarden.speech.CGRecognizer@1fbc355 changesCommitted
          com.cloudgarden.speech.CGRecognizer@1fbc355 changesCommitted
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
          com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
          com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
          DataSink = com.cloudgarden.audio.CGDataSink@800aa1 rtpRecoUrl=rtp://192.168.56.1:12346/audio
          Cannot create the RTP Session: Can't open local data port: 12346
          javax.media.CannotRealizeException
               at javax.media.Manager.blockingCall(Manager.java:2005)
               at javax.media.Manager.createRealizedProcessor(Manager.java:794)
               at asr.RecognitionEngine.listen(RecognitionEngine.java:124)
               at service.IVRService.recognize(IVRService.java:190)
               at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
               at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
               at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
               at java.lang.reflect.Method.invoke(Unknown Source)
               at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:194)
               at org.apache.axis2.rpc.receivers.RPCMessageReceiver.invokeBusinessLogic(RPCMessageReceiver.java:102)
               at org.apache.axis2.receivers.AbstractInOutMessageReceiver.invokeBusinessLogic(AbstractInOutMessageReceiver.java:40)
               at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:100)
               at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:176)
               at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:275)
               at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:133)
               at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
               at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
               at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
               at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
               at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
               at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
               at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
               at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
               at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
               at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
               at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
               at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
               at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
               at java.lang.Thread.run(Unknown Source)
          Edited by: mr_b on Aug 17, 2010 3:11 AM
          • 2. Re: JSAPI Recognition won't run twice
            843802
            I found out that this Thread is still running even after close() method. Don't know how to get rid of it though.

            Daemon Thread [Thread-12] (Suspended)     
                 CGRecognizer.enterMainEventLoop() line: not available [native method]     
                 CGRecognizer.access$10() line: not available     
                 CGRecognizer$5.run() line: not available     
                 Thread.run() line: not available