This discussion is archived
6 Replies Latest reply: Oct 24, 2013 1:51 AM by tBodur RSS

Cluster Apex MS's across multiple nodes for load balancing?

Oratime Newbie
Currently Being Moderated
Can Cluster Apex MS's across multiple nodes for load balancing? If Yes/No, any supportive doc's will be much appreciated.

Here is what I did ..........
I tried multiple MS's on different nodes for Apex Domain (clustered)
Deployed apex.war, i.war with targets pointing to cluster.
Apex listener techstack is shared filesystem $HOME/apex, where apex-config.xml and bdb are available
Anyone of the MS's whichever started up first is locking up bdb and 500 server shows up when internal workspace is being launched with host specific url, where as it works fine for the host which actually locked up bdb (first MS that was started)
2nd MS shows errors as below in log files ..
-------------
Caused By: com.sleepycat.je.EnvironmentLockedException: (JE 4.0.103)$HOME/apex/bdb The environment cannot be locked for single writer access. ENV_LOCKED: The je.lck file could not be locked. Environment is invalid and must be closed.
-------------
Its expected behavior until here .....
++++ I tried something to get it working ++++
Whilst both MS's are up and running, delete $HOME/apex/bdb but let apex-config,xml be as it is...now the internal workspace urls for both hosts work fine. I logged into host specific urls and navigated, created/edited users.,etc and it is all looking good but it did not create new bdb directory under $HOME/apex ...!!
Having said its working fine, the same issue repeats after the MS's bounce. So I did put a command line to delete bdb dir once the MS comes up to avoid the issue but really not sure if this is the right way at all?

Geek's ...awaiting your advice!!
  • 1. Re: Cluster Apex MS's across multiple nodes for load balancing?
    Udo Guru
    Currently Being Moderated
    Hello,

    I don't know any official document on that task (yet).
    Derived from my personal experience, each APEX Listener instance needs its own home directory and its own config. You can share the images for your APEX instance between your APEX Listener deployments, but not the other two.
    So your deployment can have a shared filesystem with +$HOME/apex_listener_<instance>+ (or no shared file system for that) and +$HOME/apex_images+ (or similar). Note that you should distribute the apex-config.xml manually in that scenario, as I'm not sure how stateful your clustering is when the config is rewritten after a change using listenerAdmin (especially in a situation where not all nodes are online).
    So I did put a command line to delete bdb dir once the MS comes up to avoid the issue but really not sure if this is the right way at all?
    I wouldn't recommend that. You may be able to use APEX Listener anyway as long as you don't use the Resource Templates, because this bdb-file-database is used to store the definitions for them. But I don't think I'd feel fine with that kind of workaround.
    On the other hand, with APEX Listener 2 (currently in EA-phase) the configuration for Resource Templates have been moved into the APEX SQL Workshop (APEX Workspace), so the bdb doesn't exist any longer. If you can live with your workaround until APEX Listener 2 gets production, this might be better than "splitting" the home per instance.

    -Udo
  • 2. Re: Cluster Apex MS's across multiple nodes for load balancing?
    tBodur Newbie
    Currently Being Moderated
    Hi,

    We have the same problem. Due to business requirements we need to deploy apex listener on WLS on one physical machine cluster with two managed servers. We are hitting "$HOME/apex/bdb The environment cannot be locked for single writer access" error and I've tried Oratime's work around without success. Since Apex Listener 2 is in production, I tried to resolve my problem with it. But Udo, sorry for that the bdb does exists... I tired "delete the direcory work around" with version 2 but the same results. dbd directory is still there and deleting does not end up with expected results. We will not be using REST services so there must be some kind of hack or work around to disable this behavior. I would like to hear a good solution from the community.

    Regards,

    Edited by: tBodur on Dec 23, 2012 6:22 PM
  • 3. Re: Cluster Apex MS's across multiple nodes for load balancing?
    tBodur Newbie
    Currently Being Moderated
    Hello,

    I'm still trying to cluster Apex on weblogic MS on single node. Any feedback would be greatly appreciated.

    Regards,
  • 4. Re: Cluster Apex MS's across multiple nodes for load balancing?
    Oratime Newbie
    Currently Being Moderated
    Hey there,

    I managed to fulfill my requirement..

    If its a cluster on same machine or across machines, this should work

    1. Login to machine, cd $DOMAIN_HOME
    2. mkdir -p Apex_lsn_config/AdminServer Apex_lsn_config/<MS1> Apex_lsn_config/<MS2> # MS1 and MS2 are the Managed Server names as appropriate
    #If you are planning for cluster spawning MS's across machines, make sure you create the dir's on step 2 for each machine respectively. (in my case $DOMAIN_HOME is not shared)
    3. Copy apex-config.xml from the /tmp/apex or whatever location you have it currently to Apex_lsn_config/<MS1> Apex_lsn_config/<MS2>
    4. cd $DOMAIN_HOME/bin; cp -p SetDomainEnv.sh SetDomainEnv.sh.orig #Backup the file
    5. Append -Djava.io.tmpdir in SetDomainEnv.sh as below for JAVA_OPTIONS # Do it on both machine if you are not sharing DOMAIN_HOME and planning cluster across machines
    -Djava.io.tmpdir=$DOMAIN_HOME/APEX_CONFIG/${SERVER_NAME}
    Hint: Search for "iterativeDev" and append the same line with -Djava.jo.tmpdir
    6. Modify "java.io.tmpdir" from the web.xml file of apex.war as below and re-deploy the war
    <context-param>
         <param-name>config.dir</param-name>
         <param-value>${java.io.tmpdir}</param-value>
    </context-param>
    7. Bounce Weblogic Admin and Manged Servers. Make sure to tail the Managed Server log to see apex-config.xml is picked from the new location.
    8. Brew a Coffee for yourself :)

    - You find the instructions on creating a cluster from weblogic documentation, the steps mentioned above are only to overcome the bdb locking issue whilst creating a cluster.

    Did it help?

    Edited by: Oratime on Mar 25, 2013 2:44 AM
  • 5. Re: Cluster Apex MS's across multiple nodes for load balancing?
    Oratime Newbie
    Currently Being Moderated
    Have answered my own question.
  • 6. Re: Cluster Apex MS's across multiple nodes for load balancing?
    tBodur Newbie
    Currently Being Moderated

    Great job. Thanks for the answer.!!

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points