2 Replies Latest reply on Jan 4, 2012 4:58 PM by Todd Little-Oracle

    Tuxedo.. One node is getting overloaded

      Problem Statement:

      1.     We have two physical nodes (A and B) running some tuxedo services (containing application business logics).

      2.     One particular service (S) is installed in one node only (Node A) that in turn calls other services that reside in both of the nodes.

      3.     It is observed that calls originating from S is calling services in Node A only.

      4.     This is causing an imbalance of load making A heavily loaded than B.

      Can some one help us to tweak tuxedo configuration in such a way that all service calls irrespective of their origin are equally distributed in Node A and B ?
        • 1. Re: Tuxedo.. One node is getting overloaded
          Kristian Ivarsson

          Is a "node" a server built with 'buildserver' ?

          In server A you have service S and service X and in server B you also have a service named X ?

          How is your configuration (ubconfig) done ?

          Normally Tuxedo just find the first available server that has advertised a certain service, but to be honest I have never tried to have the same service advertised in two separate servers (binaries) so there might be some (more or less obvious) explanations for this behaviour

          Have you ever seen a call to service X ended up in server B ?

          I won't question your objective to this "solution", but I'm intrested in what kind of problem you're trying to solve by having the same service in two (or more) different servers (if I have understand correctly)

          • 2. Re: Tuxedo.. One node is getting overloaded
            Todd Little-Oracle

            I'm assuming you are running your Tuxedo application in an MP configuration, i.e., two or more machines clustered together and that you have enabled load balancing (LDBAL) in your UBBCONFIG file. In this case, when Tuxedo looks to chose an IPC queue to place the request for a service on, it adds the value of NETLOAD from the *MACHINES section of the UBBCONFIG file (or from the environment variable TMNETLOAD) to the load value determined for the service.  This tends to cause more requests to be executed locally than remotely depending upon how high the value of NETLOAD is.

            Also, when you say a load imbalance, are you describing work done or work queued? When Tuxedo load balancing is enabled, Tuxedo will choose the first server (actually IPC queue since multiple servers may share an IPC queue in the case of MSSQ operation) that doesn't have any work queued to it. The result is that the first server tends to execute far more requests than the next, and so on down the line. So this would be normal behavior. If on the other hand you are referring to the amount of work queued to each queue varies significantly, this would not be expected as once all queues have work, Tuxedo will choose the queue with the least amount of work queued to place the request on.

            As for attempting to get "equal" distribution of work, this is typically not possible with Tuxedo unless all servers (queues) are busy all the time, and even then in an MP configuration, the local node doesn't have exact load information for remote queues, so the routing decision is approximate and not exact. The local node has its own view of what the load on remote queues is and that is zeroed during each BB scan. In general though, you should see fairly good distribution of load (as opposed to work done.)

            Also, if XA transactions are involved, Tuxedo will attempt to route requests to server groups that are already part of the transaction to provide some transaction affinity and reduce the number of branches to the resource manager.

            If you can provide more details, perhaps we can uncover exactly what is going on.

            Todd Little
            Oracle Tuxedo Chief Architect