Skip navigation

Besides all the exciting stuff about containers I still work sometimes with my hands in the mud . Today I encountered some strange behaviour on starting WebLogic, which I wanted to share with you, because it's hard to find the cause.

If you read the error message, you might think, this is an easy one; lots of blogposts and solutions are written. However, not everything is what it' looks like.

 

Executing the startWebLogic.sh script somehow, creates a file for the AdminServer which is located in the <DOMAIN_HOME>servers/AdminServer/tmp, usually called as <WebLogic Server Name>.lok, like in here AdminServer.lok.

This is file is claimed by the java process which WebLogic uses to start and prevents duplicate startups. If a process is running you might get the above error.  Solution was to stop the duplicate process, remove the file and start again.

However: In this case there was no process running...... The file was created but WebLogic failed to start. A removal of the file did not help, and every time I tried to start the file was created.

So I started to investigate on other startscripts, such as the NodeManager..... same results.

 

Further investigation:

 

The domain home was located on an NFS Share, with a separate admin and managed Server homes. I suspected that it had something to do with NFS.

 

So I created a testfile to see if this was the case, a file called TestLock.java:

 

import java.io.File;

import java.io.FileOutputStream;

import java.io.IOException;

 

public class TestLock

{

  public static void main(String[] args)

  {

   try

   {

    File f = new File(".homelock");

    f.createNewFile();

    FileOutputStream fos = new FileOutputStream(f);

    fos.getChannel().lock();

    }

    catch (IOException e)

    {

        e.printStackTrace();

    }

  }

}

 

 

And compiled it to a runnable class, and run it

This resulted in an error. Testing on other NFS did not gave me this error

Diagnosing with the linux command dmesg gave a lot of output, but the one which was applicable:

 

 

After the storage admin resolved the issues with lockd and statd, file locking was available again and WebLogic could startup normally.