9 Replies Latest reply on Jan 29, 2020 7:13 AM by 3415172

    GG extract error

    3415172
    3415172 Jan 24, 2020 7:38 AM

      Dear experts,

       

      Am experiencing below issue in one of our environment.

      One of extract(regstered with below command) is STARTING status forever and nothing moving.

      >> ADD EXTRACT <PROC_NAME>, SOURCEISTABLE

       

      Source GG version: 12.2 (Database: 12.1 non-containarized)

      Target GG version: 12.3(Database: 12.2 containerized)

       

      The same environment was working fine earlier without any issues.

       

      Observations:

      -------------

      - No errors in ggserr.log, mgr.rpt, extract process report.

      - Able to telnet the goldengate vip along with target manager & collector port from source.

      - Collector process at target is keep spanning from 50001 to 50004 ports(not sure if its expected behavior)

      - Timeout error at target mgr report file, but no errors.

      - Remote Trail file is getting created when extract started, but not updating(keep showing 0 bytes).

      - There is another process(PM003AP) which is connecting to same target is in RUNNING status; But the problematic extract(PPM006AX-doing full load) is in STARTING state and not updating.

       

      What we tried:

      --------------

      - Restarted XAG goldengate at target

      - Restarted source single node goldengate server(restart manager, jagent)

      - Re-registered process

      - Append "TIMEOUT 600" with RMTHOST parameter in extract

       

      Another important observation: Trails stored under dirdat is configured with DBFS. If I use non-DBFS directory to write the trails from extract(using rmthost) then atleast it turning into RUNNING state for moment and trail sizes increase, but fails with below error finally.

       

      2020-01-21 17:02:48  ERROR   OGG-01224  TCP/IP error 111 (Connection refused), endpoint: 100.78.249.108:7809.

       

      Not much information in suppot or on web, any inputs from you will be helpful in this critical issue.

       

      Thanks in advance.!

       

      --

      Regards,

      KK.

      • 103 Views
      • Tags:
        • 1. Re: GG extract error
          Vikas Panwar
          Vikas Panwar Jan 27, 2020 6:07 PM (in response to 3415172)

          Hi KK,

           

          you are using SOURCEISTABLE. Are you planning to initial load

           

          for initial load there are other alternatives which are sometimes recommended instead of using GG (like expdp and impdp)

           

          if you trying to do IL and you can have those alternatives, go ahead with those. Probably you will by pass your configuration and error all together.

           

          if you like this reply, approach mark it helpful, correct. This will help others too

           

          -VP

          • 2. Re: GG extract error
            3415172
            3415172 Jan 28, 2020 11:45 AM (in response to Vikas Panwar)

            hi . thanks for reply

             

            The solution is already in place and unfortunately can't change anything at this moment.

             

            Also, to add, have observed one thing in extract report.

            first it says:

            2020-01-27 16:43:07  WARNING OGG-01223  TCP/IP error 32 (Broken pipe), endpoint: 100.78.249.108:50002.

             

             

            2020-01-27 16:43:12  INFO    OGG-06569  Remote Collector/Server version 12.3 receiving data is a different version than this Extract 12.2 sending data.

             

            later, 2020-01-27 16:43:17  INFO    OGG-01230  Recovered from TCP error, host 100.78.249.108, port 50002.

             

            here, i see few tables processed and trail size increased at target, but after some moment it says: it failed on same port with below error;

            100.78.249.108:50002.

             

            Can you pls let me know any clue from above.

            • 3. Re: GG extract error
              Vikas Panwar
              Vikas Panwar Jan 28, 2020 4:56 PM (in response to 3415172)

              Hi,

               

              did you try using , format release 12.2 in your extract parameter file

              and also do you have ALLOWDUPDIRS in GLOBALS ?

               

              Please don't forget to mark this helpful, correct appropriately. This will help others too

               

              -VP

              1 person found this helpful
              • 4. Re: GG extract error
                3415172
                3415172 Jan 28, 2020 5:07 PM (in response to Vikas Panwar)

                Thanks vikas for reply..

                 

                it was working fine earlier without mentioning format release 12.2, but will try again with that in problematic process now.

                 

                ALLOWDUPDIRS in globals is pointing to <GG_HOME>/dirdat

                • 5. Re: GG extract error
                  Vikas Panwar
                  Vikas Panwar Jan 28, 2020 5:17 PM (in response to 3415172)

                  yeah that is correct place for allowdupdirs to point to

                   

                  please keep marking the responses helpful or like them.

                   

                  -VP

                  • 6. Re: GG extract error
                    3415172
                    3415172 Jan 28, 2020 7:10 PM (in response to Vikas Panwar)

                    few  qq on above behavior,

                     

                    1.  Does dynamic collector keep assign different ports for each extract process(parameters with RMTHOST)? How will be the behavior of collector process and port allocation, as we observed in MGR.rpt of replicat, where we see different ports for collector.

                     

                    2.  I am thinking the tcp/ip errors because target unable to communicate back to Source extract when all data transferred from source to target through TCP/IP. Can you please let me know on how extract process (when using rmtfile) comes to know all bytes are transferred to target in trail. I think this is the communication extract is missing and failining with reasons.

                     

                    thanks in advance.

                    • 7. Re: GG extract error
                      Vikas Panwar
                      Vikas Panwar Jan 28, 2020 10:50 PM (in response to 3415172)

                      I have to dig deeper but collector is internal to GG its managed by GG automatically.

                      will wait for others, who have more info on this to comment on this.

                      • 8. Re: GG extract error
                        ORASCN
                        ORASCN Jan 29, 2020 6:19 AM (in response to 3415172)

                        Hi ,

                         

                        Collector process uses different ports. It picks the port dynamically / randomly. To avoid such behavior, we will be using the parameter DYNAMICPORTLIST and give some range of ports in the manager process parameter file.

                         

                        Once this is given, the collector process will use the ports within the range listed. This enhances the security of the environment. So that only the range of ports which are given can be opened at the network level.

                         

                        There are two types of Server Collector processes. 1. Dynamic and 2. Static.

                         

                        If you need to know more about collect process, check the below link,

                         

                        https://www.oracle-scn.com/how-to-start-the-oracle-goldengate-collector-process-manually/

                         

                        Regards,

                        Veera

                        1 person found this helpful
                        • 9. Re: GG extract error
                          3415172
                          3415172 Jan 29, 2020 7:13 AM (in response to ORASCN)

                          Thanks Veera for your reply..

                           

                          Its clear now on collector process behavior with the above description.

                          I have already gone through the link you've mentioned and try that as well to make it static port, still we face same issue.

                           

                          Can you pls also clarify 2nd one on tcp/ip. Does collector at target has any mechanism to communicate back to extract such that extract processes knows it and move its checkpoint to process other tables? What i am thinking is: that communication is not going back to extract. Because i tried the load with just one table, the trail updates the incoming data,( as its size increases at target sub-directories) and extract fails exactly after 5 mins(300 secs default timeout) since the last message in view report.

                           

                          the last message is: processing mentioned table at particular timestamp.

                           

                          Thanks in advance.