This discussion is archived
8 Replies Latest reply: Jun 3, 2010 7:26 AM by psadminMD RSS

Suddenly not able to start up the appserver? SQL library error

psadminMD Newbie
Currently Being Moderated
Our datacenter went down this past weekend and so my Template 9.1 environments also went down, unfortuantely I have yet to have time to put in start up scripts on those servers. I am trying to bring all 3 tiers up manually and am encountering errors I've never had before. I am assuming that I am doing something wrong because this environment has not been touched and I've never had this problem before.

Templates have not been changed, we accepted all the defaults. We have 3 different servers, 1 for DB, 1 App, 1 Webserver
I have started the database and listener and can reach them via sqlplus no problem. However, when starting up my appserver I am getting the following error: GenMessageBox(200, 0, M): PS General SQL Routines: Missing or invalid version of SQL library libpsora

I have set my Oracle environment (. oraenv) to my database name. And my Oracle home to: /opt/oracle/psft/appbatch/oracle-client/10.2.0.4-64bit. Is this correct? I am starting the appserver as psadm2.

What am I missing? Why am I suddently having library file version errors?

Thanks!!
  • 1. Re: Suddenly not able to start up the appserver? SQL library error
    Nicolas.Gasparotto Oracle ACE
    Currently Being Moderated
    GenMessageBox(200, 0, M): PS General SQL Routines: Missing or invalid version of SQL library libpsora
    Please, provide some more input of the psadmin command error and appserver log file entries.
    Are people and sysadm users' password not expired (even if I would expect an other error) ?
    Verify it from sqlplus :
    select username,account_status,expiry_date,profile
    from   dba_users
    where username in ('PEOPLE','SYSADM');
    From the App/Batch server, within psadm2 user, are you able to connect onto the database ?
    I have set my Oracle environment (. oraenv) to my database name.
    Where are you running it from ? I've never ran it to get it work.

    Nicolas.
  • 2. Re: Suddenly not able to start up the appserver? SQL library error
    psadminMD Newbie
    Currently Being Moderated
    Ran your sql statement, here is the output:

    SQL> select username,account_status,expiry_date,profile
    2 from dba_users
    3 where username in ('PEOPLE','SYSADM');

    USERNAME ACCOUNT_STATUS EXPIRY_DA
    ------------------------------ -------------------------------- ---------
    PROFILE
    ------------------------------
    PEOPLE OPEN 08-JUL-10
    DEFAULT

    SYSADM OPEN 29-JUL-10
    DEFAULT

    Here is more of the appserver error:
    Do you wish to see the error messages in the APPSRV.LOG file? (y/n) [n] :y

    PSADMIN.1656 (0) [06/02/10 14:24:58](0) Begin boot attempt on domain APPDOM
    PSAPPSRV.1671 (0) [06/02/10 14:25:12](0) PeopleTools Release 8.50.02 (Linux) sta
    rting. Tuxedo server is APPSRV(99)/1
    PSAPPSRV.1671 (0) [06/02/10 14:25:12](0) Cache Directory being used: /home/psadm
    2/ps/pt/8.50/appserv/APPDOM/CACHE/PSAPPSRV_1/
    PSAPPSRV.1671 (0) [06/02/10 14:25:12](1) GenMessageBox(200, 0, M): PS General SQ
    L Routines: Missing or invalid version of SQL library libpsora (200,0)
    PSAPPSRV.1671 (0) [06/02/10 14:25:12](1) GenMessageBox(0, 0, M): Database Signon
    : Could not sign on to database PS91DEMO with user PS.
    PSAPPSRV.1671 (0) [06/02/10 14:25:12](0) Server failed to start
    PSADMIN.1656 (0) [06/02/10 14:25:19](0) End boot attempt on domain APPDOM

    Yes I can connect to sqlplus from the app/batch server with the psadm2 user:
    [psadm2@ovm002 appserv]$ sqlplus SYSADM@psdmo

    SQL*Plus: Release 10.2.0.1.0 - Production on Wed Jun 2 14:28:57 2010

    Copyright (c) 1982, 2005, Oracle. All rights reserved.

    Enter password:

    Connected to:
    Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
    With the Partitioning, OLAP, Data Mining and Real Application Testing options

    SQL>

    Thanks.
  • 3. Re: Suddenly not able to start up the appserver? SQL library error
    psadminMD Newbie
    Currently Being Moderated
    Also,
    Here is how I am setting the environment.

    [psadm2@ovm002 ~]$ . oraenv
    ORACLE_SID = [psadm2] ? psdmo
    Cannot locate ORACLE_HOME.
    ORACLE_HOME = [] ? /opt/oracle/psft/appbatch/oracle-client/10.2.0.4-64bit
    [psadm2@ovm002 ~]$
  • 4. Re: Suddenly not able to start up the appserver? SQL library error
    Nicolas.Gasparotto Oracle ACE
    Currently Being Moderated
    pspeon2.0 wrote:
    Also,
    Here is how I am setting the environment.

    [psadm2@ovm002 ~]$ . oraenv
    Why are you doing this ? You don't need it, did you try to disconnect/reconnect and start the appserver without this ?

    Nicolas.
  • 5. Re: Suddenly not able to start up the appserver? SQL library error
    psadminMD Newbie
    Currently Being Moderated
    Out of habit I guess...we do this on our other environments because we sometimes have different Oracle Homes.

    I got completely out and back in and tried it again without setting any Oracle params and I'm still getting the same issue. I've encountered this before and had to add some symbolic links, but that was on my other environments and only after an Oracle version upgrade. Not sure why this is suddenly having a problem with the SQL library files. I'm playing around with symbolic link as per the Oracle document to see if maybe that will fix it, but so far it doesn't look like it is.
  • 6. Re: Suddenly not able to start up the appserver? SQL library error
    Nicolas.Gasparotto Oracle ACE
    Currently Being Moderated
    Here what I have :
    [psadm2@psovmab 10.2.0.4-64bit]$ pwd
    /opt/oracle/psft/appbatch/oracle-client/10.2.0.4-64bit
    [psadm2@psovmab 10.2.0.4-64bit]$ cd lib
    [psadm2@psovmab lib]$ ls -lrt
    ...
    lrwxrwxrwx 1 root root       15 Oct  5  2009 libagtsh.so -> libagtsh.so.1.0
    lrwxrwxrwx 1 root root       15 Oct  5  2009 libocci.so -> libocci.so.10.1
    drwxrwxr-x 2 root root     4096 Oct  5  2009 stubs
    lrwxrwxrwx 1 root root       76 Oct  5  2009 libclntsh.so.9.0 -> /opt/oracle/psft/appbatch/oracle-client/10.2.0.4-64bit/lib/libclntsh.so.10.1
    lrwxrwxrwx 1 root root       76 Oct  5  2009 libclntsh.so -> /opt/oracle/psft/appbatch/oracle-client/10.2.0.4-64bit/lib/libclntsh.so.10.1
    [psadm2@psovmab lib]$ cd ../lib32
    [psadm2@psovmab lib32]$ ls -lrt
    ...
    lrwxrwxrwx 1 root root       17 Oct  5  2009 libclntsh.so -> libclntsh.so.10.1
    lrwxrwxrwx 1 root root       15 Oct  5  2009 libagtsh.so -> libagtsh.so.1.0
    lrwxrwxrwx 1 root root       50 Oct  5  2009 ldflags -> /products/oracle_client/10.2.0.4-64bit/lib/ldflags
    lrwxrwxrwx 1 root root       15 Oct  5  2009 libocci.so -> libocci.so.10.1
    drwxrwxr-x 2 root root     4096 Oct  5  2009 stubs
    Nicolas.
  • 7. Re: Suddenly not able to start up the appserver? SQL library error
    psadminMD Newbie
    Currently Being Moderated
    What do you have under your $PS_HOME/bin directory for links? In the past they had to have matching links between the $PS_HOME/bin and $ORACLE_HOME/lib directories.

    Thanks!
  • 8. Re: Suddenly not able to start up the appserver? SQL library error
    psadminMD Newbie
    Currently Being Moderated
    FIXED IT!
    Found a hint in one of the solutions on My Oracle Support at the very bottom. ID 650409.1.

    I did add the links to $PS_HOME/bin, but what finally fixed it was the path to my $ORACLE_HOME/lib was incorrect in the $PS_HOME/appserv/DomainName/psappsrv.env file for my LD_LIBRARY_PATH. It had my path as /opt/oracle/psft/appbatch/oracle-client/lib instead of /opt/oracle/psft/appbatch/oracle-client/*10.2.0.4-64bit*/lib. Funny how it was fine everywhere else, except this file

    Now as to how in the world that got changed I have no clue. I am the only one that touches these servers and I haven't touched them in months.

    Does this file get rebuilt? Would a crash cause it not to be populated correctly? So weird.

    t.

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points