This discussion is archived
2 Replies Latest reply: Aug 18, 2010 12:56 AM by 843802 RSS

JSAPI Recognition won't run twice

843802 Newbie
Currently Being Moderated
I'm working on an implementation of JSAPI's Recognition. 'listen' method is invoked through a web service and it runs fine for the first time. When I run it for a second time it looks like I've got two engines allocated (though I deallocate it) because everything is displayed 2 times. Then the app crashes and blocks a port. When it is run for the third time I displays everything three times and an exception is thrown due to a blocked port. I tried to use both the same Recognizer object for every 'listen' method call and creating a new object for every invocation (the output below is the second case). It doesn't help either when I use different ports for other invocations. When I restart the server it again works only once. So what I think the problem is I am not deallocating or closing sources properly. I think it is JSAPI problem but could also be RTP. Please help.

I use Cloud Garden's JSAPI implementation.

Here is the code:
package asr;

import javax.speech.*;
import javax.speech.recognition.*;

import com.cloudgarden.audio.*;
import com.cloudgarden.speech.*;

import javax.media.protocol.*;
import javax.media.*;
import javax.media.format.*;

public class RecognitionEngine {

     public RecognitionEngine(String recognitionEngineName) {

     }

     /**
      * TODO: synchronizacja result?
      * 
      * @return
      */
     public String listen(String destination, String port) {
          ++counter;
          System.out.println("Times called: " + counter);
          
          result = null;

          try {
               String rtpRecoUrl = "rtp://" + destination + ":" + port + "/audio";
               reco = Central.createRecognizer(new EngineModeDesc(null, null,
                         java.util.Locale.ENGLISH, null));

               reco.addEngineListener(new TestEngineListener());

               reco.allocate();
               reco.waitEngineState(reco.ALLOCATED);

               CGAudioManager recoAudioMan = (CGAudioManager) reco.getAudioManager();
               recoAudioMan.addAudioListener(new TestAudioListener());
               recoAudioMan.addTransferListener(new TransferListener() {
                    public void bytesTransferred(TransferEvent e) {
                         System.out.print("R" + e.getLength() + " ");
                    }
               });

               boolean gramExists = false;
               for (RuleGrammar r : reco.listRuleGrammars()) {
                    if (r.getName().equals("gram1"))
                         gramExists = true;
               }
               if (!gramExists) {
                    RuleGrammar gram = reco.newRuleGrammar("gram1");
                    Rule r = gram
                              .ruleForJSGF("all your base are belong to us {BASE} | hello world {HELLO} | goodbye computer {QUIT} | what time is it {TIME} | what date is it {DATE}");
                    gram.setRule("rule1", r, true);
                    gram.setEnabled(true);
               }

               // must be called before recognizer is resumed (for SAPI5 engines)!
               reco.getRecognizerProperties().setResultAudioProvided(true);

               recoDataSink = recoAudioMan.getDataSink();
               reco.commitChanges();
               reco.requestFocus();
               reco.resume();
               reco.waitEngineState(reco.LISTENING);
               reco.addResultListener(new ResultAdapter() {
                    public void resultUpdated(ResultEvent ev) {
                         System.out.println("updated " + ev);
                    }

                    public void resultAccepted(ResultEvent ev) {
                         System.out.println("accepted " + ev);
                         FinalResult res1 = (FinalResult) ev.getSource();
                         FinalRuleResult frr = (FinalRuleResult) res1;
                         String[] tags = frr.getTags();
                         if (tags == null)
                              return;
                         String tag = tags[0];

                         // Uncomment this to hear what recognizer heard.
                         // try {
                         // reco.pause();
                         // frr.getAudio().play();
                         // reco.resume();
                         // } catch (Exception e) {
                         // e.printStackTrace();
                         // }
                         //

                         if (tag.equals("QUIT")) {
                              System.out.println("Someone said goodbye computer");
                         } else if (tag.equals("HELLO")) {
                              System.out.println("Someone said hello world");
                         } else if (tag.equals("TIME")) {
                              System.out.println("Someone said what time is it");
                         } else if (tag.equals("DATE")) {
                              System.out.println("Someone said what date is it");
                         } else if (tag.equals("BASE")) {
                              System.out.println("ALL YOUR BASE ARE BELONG TO US!!");
                         }
                         result = tag;
                         close();
                    }
               });

               javax.sound.sampled.AudioFormat fmt = recoAudioMan.getAudioFormat();

               FileTypeDescriptor cd = new FileTypeDescriptor(FileTypeDescriptor.RAW);
               Format[] outFormats = { new AudioFormat(AudioFormat.LINEAR,
                         fmt.getFrameRate(), fmt.getSampleSizeInBits(),
                         fmt.getChannels(), AudioFormat.LITTLE_ENDIAN,
                         AudioFormat.SIGNED) };
               System.out.println("DataSink = " + recoDataSink + " rtpRecoUrl="
                         + rtpRecoUrl);

               ProcessorModel pm = new ProcessorModel(
                         new MediaLocator(rtpRecoUrl), outFormats, cd);

               listening = true;

               recoProc = Manager.createRealizedProcessor(pm);
               System.out.println("RTP connection established");

               recoDataSink.setSource(recoProc.getDataOutput());
               recoProc.start();

               recoDataSink.open();
               recoDataSink.start();

               System.out.println("listening");

               reco.waitEngineState(reco.DEALLOCATED);

               System.out.println("EXITING");
          } catch (Exception e) {
               e.printStackTrace();
          } finally {
               listening = false;
          }
          return result;
     }

     public void close() {
          System.out.println("closing");
          try {
               recoProc.stop();
               recoProc.close(); //mine //unblocks port
               //recoProc.deallocate(); //mine
               //recoDataSink.stop(); //mine
               recoDataSink.close();

               reco.deallocate();
               reco.waitEngineState(Engine.DEALLOCATED);

               Thread.currentThread().sleep(2000);
          } catch (Exception e2) {
               e2.printStackTrace();
          }
          System.out.println("closed!");

     }

     public boolean isListening() {
          return listening;
     }



     private int counter = 0;
     private boolean listening = false;
     private String result;
     private Recognizer reco;
     private Processor recoProc;
     private DataSink recoDataSink;
}
  • 1. Re: JSAPI Recognition won't run twice
    843802 Newbie
    Currently Being Moderated
    output:
    ...
    //server starting
    INFO: Server startup in 6153 ms
    
    INIT!
    Recognizer jest null!
    Times called: 1
    CloudGarden's JSAPI1.0 implementation
    Version 1.7.0
    Implementation contained in files cgjsapi.jar and cgjsapi170.dll
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineAllocatingResources
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineAllocated
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
    com.cloudgarden.speech.CGRecognizer@1fbc355 changesCommitted
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
    com.cloudgarden.speech.CGRecognizer@1fbc355 focusGained
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
    DataSink = com.cloudgarden.audio.CGDataSink@800aa1 rtpRecoUrl=rtp://192.168.56.1:12346/audio
    RTP connection established
    listening
    R1536 R6000 R144 R2304 R2304 R2304 R2304 R2304 R2304 Speech started
    R2304 R2304 R2304 R2304 com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerProcessing
    updated javax.speech.recognition.ResultEvent[source=Grammar:gram1, Conf:3, Tokens{}]
    R2304 R2304 R2304 R2304 R2304 R2304 R2304 R2304 R2304 R2304 updated javax.speech.recognition.ResultEvent[source=Grammar:gram1, Conf:3, Tokens{hello,world}]
    R2304 R2304 Speech stopped
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
    updated javax.speech.recognition.ResultEvent[source=Grammar:gram1, Conf:3, Tokens{hello,world}, Tags{HELLO}]
    accepted javax.speech.recognition.ResultEvent[source=Grammar:gram1, Conf:3, Tokens{hello,world}, Tags{HELLO}]
    Someone said hello world
    closing
    R0 R-1 com.cloudgarden.speech.CGRecognizer@1fbc355 focusLost
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineDeallocatingResources
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineDeallocated
    EXITING
    closed!
    Speech started
    Speech stopped
    
    INIT!
    Recognizer jest null!
    Times called: 1
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineAllocatingResources
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineAllocatingResources
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineAllocated
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineAllocated
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
    com.cloudgarden.speech.CGRecognizer@1fbc355 changesCommitted
    com.cloudgarden.speech.CGRecognizer@1fbc355 changesCommitted
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
    com.cloudgarden.speech.CGRecognizer@1fbc355 focusGained
    com.cloudgarden.speech.CGRecognizer@1fbc355 focusGained
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
    DataSink = com.cloudgarden.audio.CGDataSink@800aa1 rtpRecoUrl=rtp://192.168.56.1:12346/audio
    RTP connection established
    listening
    R-1 R-1 
    
    INIT!
    Recognizer jest null!
    Times called: 1
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
    com.cloudgarden.speech.CGRecognizer@1fbc355 changesCommitted
    com.cloudgarden.speech.CGRecognizer@1fbc355 changesCommitted
    com.cloudgarden.speech.CGRecognizer@1fbc355 changesCommitted
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerSuspended
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
    com.cloudgarden.speech.CGRecognizer@1fbc355 engineResumed...
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
    com.cloudgarden.speech.CGRecognizer@1fbc355 recognizerListening
    DataSink = com.cloudgarden.audio.CGDataSink@800aa1 rtpRecoUrl=rtp://192.168.56.1:12346/audio
    Cannot create the RTP Session: Can't open local data port: 12346
    javax.media.CannotRealizeException
         at javax.media.Manager.blockingCall(Manager.java:2005)
         at javax.media.Manager.createRealizedProcessor(Manager.java:794)
         at asr.RecognitionEngine.listen(RecognitionEngine.java:124)
         at service.IVRService.recognize(IVRService.java:190)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
         at java.lang.reflect.Method.invoke(Unknown Source)
         at org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:194)
         at org.apache.axis2.rpc.receivers.RPCMessageReceiver.invokeBusinessLogic(RPCMessageReceiver.java:102)
         at org.apache.axis2.receivers.AbstractInOutMessageReceiver.invokeBusinessLogic(AbstractInOutMessageReceiver.java:40)
         at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:100)
         at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:176)
         at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:275)
         at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:133)
         at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
         at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
         at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
         at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
         at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
         at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
         at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
         at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
         at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
         at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
         at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
         at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
         at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
         at java.lang.Thread.run(Unknown Source)
    Edited by: mr_b on Aug 17, 2010 3:11 AM
  • 2. Re: JSAPI Recognition won't run twice
    843802 Newbie
    Currently Being Moderated
    I found out that this Thread is still running even after close() method. Don't know how to get rid of it though.

    Daemon Thread [Thread-12] (Suspended)     
         CGRecognizer.enterMainEventLoop() line: not available [native method]     
         CGRecognizer.access$10() line: not available     
         CGRecognizer$5.run() line: not available     
         Thread.run() line: not available