8 Replies Latest reply: Nov 21, 2012 12:00 PM by user10437903 RSS

    Directory Setup for RAC and Oracle Restart

    user10437903
      I have three database installations that I'd like some insight on the initial directory structure and permissions. The three installations are:
      1) Prod RAC - OEL 5U2, 11.2.0.2 Grid Infrastructure, 11.2.0.2 DB EE with RAC option
      2) Test RAC - OEL 5U2, 11.2.0.2 Grid Infrastructure, 11.2.0.2 DB EE with RAC option
      3) Single Instance - OEL 5U2, 11.2.0.2 Grid Infrastructure Standalone (Oracle Restart), 11.2.0.2 DB EE

      The they all have two users GRID (for Grid Infrastructure Software Owner) and ORACLE (for Oracle Database Software Owner).

      We had a consultant install the databases. It's very weird since we have an inconsistency in directories and permission. I want to understand the inconsistency between RAC installations as well as Oracle Restart Install.

      SEE BELOW: The two RAC servers have different owners for /u01/app/11.2.0.2 directory. The Oracle Restart Installation - Single Instance has the same owner as Prod RAC but different than Test RAC. What the consultant said was that ORACLE_BASE would be /u01/app/oracle for both Grid Infrastructure and Database installations. This was to apply to TEST/PROD RAC (3 node, 3 instances, 1 instance/node) and Oracle Restart installation. The GRID_HOME would be /u01/app/11.2.0.2/grid and ORACLE_HOME would be /u01/app/11.2.0.2/db_1. It seems like the owners aren't setup correctly. Why? I don't want problems with patching or upgrading. If this setup is technically incorrect what is the best way to setup so there's no owner/permission issues. Is it necessary to have two separate ORACLE_BASE's like /u01/app/grid?

      PRODUCTION REAL APPLICATION CLUSTER (Displaying only one Node details)

      $ cd app
      drwxr-xr-x 6 root oinstall 4096 Feb 26 2011 .
      drwxr-xr-x 3 root oinstall 4096 Apr 21 2009 ..
      drwxrwxr-x 4 grid oinstall 4096 Feb 26 2011 11.2.0.2
      drwxrwxr-x 2 emagent oinstall 4096 Feb 26 2011 emagent
      drwxrwxr-x 9 oracle oinstall 4096 May 16 2011 oracle
      drwxrwx--- 7 grid oinstall 4096 Feb 26 2011 oraInventory
      ............................
      $ cd 11.2.0.2
      drwxrwxr-x 4 grid oinstall 4096 Feb 26 2011 .
      drwxr-xr-x 6 root oinstall 4096 Feb 26 2011 ..
      drwxr-xr-x 79 oracle oinstall 4096 Mar 29 2011 db_1
      drwxr-xr-x 69 root oinstall 4096 Feb 26 2011 grid
      ..................................
      $ cd ..
      $ cd oracle
      9 oracle oinstall 4096 May 16 2011 .
      drwxr-xr-x 6 root oinstall 4096 Feb 26 2011 ..
      drwxr-xr-x 4 oracle oinstall 4096 Feb 26 2011 admin
      drwxr-xr-x 3 oracle oinstall 4096 Feb 26 2011 cfgtoollogs
      drwxrwxr-x 2 grid oinstall 4096 Feb 26 2011 Clusterware
      drwxrwxr-x 11 grid oinstall 4096 Feb 26 2011 diag
      drwxrwxr-x 3 grid oinstall 4096 Feb 26 2011 hqrac03
      drwxr----- 3 oracle oinstall 4096 May 16 2011 oradiag_oracle
      drwxr-xr-x 3 root root 4096 Mar 13 2011 oradiag_root

      TEST REAL APPLICATION CLUSTER (Displaying only one Node details)

      $ cd app
      drwxr-xr-x 6 root oinstall 4096 Jan 26 2011 ./
      drwxr-xr-x 3 root oinstall 4096 Jan 26 2009 ../
      drwxrwxr-x 4 root oinstall 4096 Jan 26 2011 11.2.0.2/
      drwxrwxr-x 2 emagent oinstall 4096 Mar 1 2011 emagent/
      drwxrwxr-x 10 grid oinstall 4096 May 16 2011 oracle/
      drwxrwx--- 7 grid oinstall 4096 Jan 27 2011 oraInventory/
      ..................................
      $ cd 11.2.0.2
      drwxrwxr-x 4 root oinstall 4096 Jan 26 2011 ./
      drwxr-xr-x 6 root oinstall 4096 Jan 26 2011 ../
      drwxr-xr-x 80 oracle oinstall 4096 Mar 17 2011 db_1/
      drwxr-xr-x 70 root oinstall 4096 Jan 27 2011 grid/
      .................................
      $ cd ..
      $ cd oracle
      drwxrwxr-x 10 grid oinstall 4096 May 16 2011 ./
      drwxr-xr-x 6 root oinstall 4096 Jan 26 2011 ../
      drwxr-xr-x 4 oracle oinstall 4096 Feb 2 2011 admin/
      drwxr-xr-x 4 oracle oinstall 4096 Feb 2 2011 cfgtoollogs/
      drwxr-xr-x 2 grid oinstall 4096 Jan 26 2011 Clusterware/
      drwxrwxr-x 11 grid oinstall 4096 Jan 26 2011 diag/
      drwxr-xr-x 3 grid oinstall 4096 Jan 26 2011 hqract03/
      drwxr-xr-x 3 grid oinstall 4096 Mar 23 2011 oradiag_grid/
      drwxr----- 3 oracle oinstall 4096 May 16 2011 oradiag_oracle/
      drwxr-xr-x 3 root root 4096 Jan 27 2011 oradiag_root/

      SINGLE INSTANCE DATABASE (ORACLE RESTART)

      $ cd app
      drwxrwxr-x 6 503 oinstall 4096 Feb 26 2011 .
      drwxr-xr-x 3 root root 4096 Apr 16 2009 ..
      drwxrwxr-x 4 grid oinstall 4096 Feb 26 2011 11.2.0.2
      drwxrwxr-x 8 grid oinstall 4096 Feb 26 2011 oracle
      drwxrwx--- 6 grid oinstall 4096 Feb 26 2011 oraInventory
      .....................
      $ cd 11.2.0.2
      drwxrwxr-x 4 grid oinstall 4096 Feb 26 2011 .
      drwxrwxr-x 6 503 oinstall 4096 Feb 26 2011 ..
      drwxr-xr-x 76 oracle oinstall 4096 Apr 6 2011 db_1
      drwxr-x--- 69 root oinstall 4096 Feb 26 2011 grid
      .....................
      $ cd ..
      $ cd oracle
      drwxrwxr-x 8 grid oinstall 4096 Feb 26 2011 .
      drwxrwxr-x 6 503 oinstall 4096 Feb 26 2011 ..
      drwxrwxr-x 5 grid oinstall 4096 Feb 26 2011 admin
      drwxrwxr-x 7 grid oinstall 4096 Feb 26 2011 cfgtoollogs
      drwxrwxr-x 2 grid oinstall 4096 Feb 26 2011 checkpoints
      drwxrwxr-x 2 grid oinstall 4096 Feb 26 2011 Clusterware
      drwxrwxr-x 11 grid oinstall 4096 Feb 26 2011 diag
      drwxrwxr-x 3 grid oinstall 4096 Feb 26 2011 hqods01


      Thanks so much.
        • 1. Re: Directory Setup for RAC and Oracle Restart
          alvaromiranda
          Hello

          I did only check prod, as that is the important one.

          i am afraid that setup is not clean. I won't use the work is wrong, but need to be normalized.

          grid oracle base need to be separate from oracle oracle base, and the grid oracle home, MUST be outside of oracle base/oracle home of the oracle user.

          In a multi user setup, the structure should be as follow.

          grid user, group oinstall, asm* groups
          oracle user, group oinstall, dba and db* groups

          The GRID install, have, grid home and oracle base

          the oracle base for grid should be, /u01/app/grid
          the grid home should be, /u01/app/11.2.0.3/grid

          the Oracle user have several homes, and oracle base

          the oracle base for oracle should be, /u01/oracle

          so, for permissions, when I create this, i do as root

          mkdir -p /u01/app/11.2.0.3/grid
          mkdir -p /u01/app/grid
          mkdir -p /u01/app/oracle
          mkdir -p /u01/app/oraInventory

          chown -R grid:oinstall /u01
          chown -R oracle:oinstall /u01/app/oracle

          then, you install grid .. you have so say oracle home is /u01/app/grid and oracle home is /u01/app/11.2.0.3/grid

          at the end, the root.sh as part of the install will take care of the permissions for oraInventory, and the rest for grid.

          when grid is installed, you can move to the oracle db install you use oracle base as /u01/app/oracle and oracle home as /u01/app/oracle/product/11.2.0.3/dbhome_1

          that will give you

          2 4 drwxr-xr-x 6 root oinstall 4096 Sep 21 03:20 /u01

          5996545 4 drwxr-xr-x 6 root oinstall 4096 Sep 18 22:58 /u01/app
          5996546 4 drwxr-xr-x 8 grid oinstall 4096 Sep 18 22:50 /u01/app/grid
          5996547 4 drwxr-xr-x 3 root oinstall 4096 Sep 18 22:30 /u01/app/11.2.0.3
          5996548 4 drwxr-xr-x 69 root oinstall 4096 Sep 21 10:52 /u01/app/11.2.0.3/grid
          5996549 4 drwxrwx--- 5 grid oinstall 4096 Oct 30 22:08 /u01/app/oraInventory
          7406840 4 drwxr-xr-x 7 oracle oinstall 4096 Oct 30 22:09 /u01/app/oracle
          7406841 4 drwxr-xr-x 3 oracle oinstall 4096 Sep 18 22:58 /u01/app/oracle/product
          7406842 4 drwxr-xr-x 6 oracle oinstall 4096 Oct 30 22:04 /u01/app/oracle/product/11.2.0.3
          7766052 4 drwxr-xr-x 77 oracle oinstall 4096 Nov 2 01:20 /u01/app/oracle/product/11.2.0.3/dbhome_1

          Each user have to use his own oracle_base, as each oracle_base will store files and folders created for logs, admin, diagnostic with different permissions and different usages, for a reason you have role separation.

          example of the content of each oracle base

          $ find /u01/app/oracle/ -maxdepth 1 -ls
          7406840 4 drwxr-xr-x 7 oracle oinstall 4096 Oct 30 22:09 /u01/app/oracle/
          7406841 4 drwxr-xr-x 3 oracle oinstall 4096 Sep 18 22:58 /u01/app/oracle/product
          8749076 4 drwxrwxr-x 3 oracle asmadmin 4096 Sep 18 23:03 /u01/app/oracle/diag
          8749064 4 drwxr-x--- 6 oracle oinstall 4096 Oct 30 22:09 /u01/app/oracle/admin
          8716289 4 drwxr-xr-x 2 oracle oinstall 4096 Oct 30 22:18 /u01/app/oracle/checkpoints
          8749061 4 drwxr-x--- 5 oracle oinstall 4096 Sep 18 23:12 /u01/app/oracle/cfgtoollogs

          $ find /u01/app/grid/ -maxdepth 1 -ls
          5996546 4 drwxr-xr-x 8 grid oinstall 4096 Sep 18 22:50 /u01/app/grid/
          7307540 4 drwxrwxr-x 4 grid oinstall 4096 Sep 18 22:49 /u01/app/grid/diag
          7307474 4 drwxr-xr-x 2 grid oinstall 4096 Sep 18 22:33 /u01/app/grid/Clusterware
          7307265 4 drwxr-xr-x 2 grid oinstall 4096 Sep 18 22:33 /u01/app/grid/checkpoints
          7307532 4 drwxr-x--- 4 grid oinstall 4096 Sep 18 22:49 /u01/app/grid/cfgtoollogs
          7406796 4 drwxr-xr-x 3 root root 4096 Sep 18 22:50 /u01/app/grid/oradiag_root

          Take care,
          Alvaro.
          • 2. Re: Directory Setup for RAC and Oracle Restart
            rcc50886
            grid oracle base need to be separate from oracle oracle base
            This is not a "MUST" do thing. If we use one ORACLE_BASE for the GI and DB's as it will provide centralize diagnostic directory.

            Find below are the simple and popular directory structure
            GI_HOME=/u01/app/grid/product/11.2.0.2
            GI_BASE=/u01/app/grid
            
            ORACLE_HOME=/u01/app/oracle/product/11.2.0.2
            ORACLE_BASE=/u01/app/oracle
            
            Central Inventory = /u01/app/oraInventory
            -Thanks
            • 3. Re: Directory Setup for RAC and Oracle Restart
              alvaromiranda
              i am sorry, but when you have role separation, each user need to have his own oracle_base.

              is on the manuals if you want to check.

              thanks
              • 4. Re: Directory Setup for RAC and Oracle Restart
                user10437903
                The installation above was done by an Oracle Consultant. He said that this was the best practice in the field. The installation in this case was done for convenience sake. The same person does Clusterware, ASM and DB administration. The separate users and permission are so environment related issues don't arise. I have read the directory permissions requirements and do understand that this configuration doesn't follow the requirements. I am looking for some answers on how to correct without re-installation and what the ramifications of the current directory structures pose.
                Question 1: Other than role separation, what are the problems with having GRID and ORACLE user have the same ORACLE_BASE.

                It seems that the permission regarding the GRID_HOME and ORACLE_HOME seem to be screwed up.
                Question 2: Given that the HOMES for GRID and ORACLE share the same directory (/u01/app/11.2.0.2/...), what should the permission look like GIVEN MY installation directory structure, like Prod, Test or something else? Question 3: What is wrong with having the HOMES share the same directory path? Upgrade issues? Patching issues?

                Thanks.
                • 5. Re: Directory Setup for RAC and Oracle Restart
                  rcc50886
                  Question 1: Other than role separation, what are the problems with having GRID and ORACLE user have the same ORACLE_BASE.
                  absolutely no impact, we have been using the same ORACLE_BASE over the years and no problems at all.
                  what made you think it causes the problem ?
                  Question 2: Given that the HOMES for GRID and ORACLE share the same directory (/u01/app/11.2.0.2/...)
                  before GI installation the GI_HOME will be owned by "grid" user, but when you run the "root.sh" as part of installation then the script changes the necessary permission changes (some directories and files may owned by "root")

                  for RBMS HOME, it always owned by oracle and by running "root.sh" script it will make necessary changes on permissions.
                   Question 3: What is wrong with having the HOMES share the same directory path?
                  You mean ORACLE_HOME=/u01/app/11.2.0.2/oracle
                  GIRD_HOME=/u01/app/11.2.0.2/grid

                  then there will not be any problems at all as long as permissions on /u01/app/11.2.0.2/oracle and /u01/app/11.2.0.2/grid not changed.
                  • 6. Re: Directory Setup for RAC and Oracle Restart
                    user10437903
                    Nov 13, 2012 1:00 PM Answer 1:
                    absolutely no impact, we have been using the same ORACLE_BASE over the years and no problems at all.
                    what made you think it causes the problem ?

                    Nov 14, 2012 8:57 am Question 1:
                    As previously mentioned, Oracle recommends that if separate user are create for Grid Infrastructure and Oracle RDBMS Installations that there be two separate ORACLE_BASE's. If your using two separate user for role separation then what is the advantage/disadvantage of having the same ORACLE_BASE? According to your input, you been using this for years with this setup, so there must be no shared directories in ORACLE_BASE otherwise, you would know what software laid the file GI or RDBMS, right?

                    Nov 13, 2012 1:00 PM Answer 2:
                    before GI installation the GI_HOME will be owned by "grid" user, but when you run the "root.sh" as part of installation then the script changes the necessary permission changes (some directories and files may owned by "root") for RBMS HOME, it always owned by oracle and by running "root.sh" script it will make necessary changes on permissions.

                    Nov 14, 2012 8:57 am Question 2:
                    Can you tell me the owner and permissions for the following directories before and after root.sh for Grid Infrastructure and Oracle RDBMS installation?
                    GI before GI after RDBMS before RDBMS after
                    /u01
                    /u01/app
                    /u01/11.2.0.2

                    Nov 13, 2012 1:00 PM Answer 3:
                    You mean ORACLE_HOME=/u01/app/11.2.0.2/oracle
                    GIRD_HOME=/u01/app/11.2.0.2/grid
                    then there will not be any problems at all as long as permissions on /u01/app/11.2.0.2/oracle and /u01/app/11.2.0.2/grid not changed.

                    Nov 14, 2012 8:57 am Question 3:
                    Yes. I do mean both Oracle_Home and Grid_Home.

                    The reason I am concerned is that the GRID_HOME and ORACLE_HOME share the same path and I am uncertain at what level in directory structure the permission our changed and the impact to out of place updates and upgrades. For example, if 11.2.0.2 has the owner modified by GI is it ok for RDBMS install to change owner. I don't know if this scenario is possible so I looking for someone who knows what the expected behavior is on the directories.

                    Caution: For Oracle Grid Infrastructure for a Cluster installations, note the following restrictions for the Oracle Grid Infrastructure binary home (Grid home):
                    ■ It must not be placed under one of the Oracle base directories, including the Oracle base directory of the Oracle Grid Infrastructure installation owner.
                    ■ It must not be placed in the home directory of an installation owner
                    During installation, ownership of the path to the Grid home is changed to root. This change causes permission errors for other installations.

                    I will been updating HELPFUL and CORRECT after this answer.

                    Thanks.
                    • 7. Re: Directory Setup for RAC and Oracle Restart
                      user10437903
                      Nov 13, 2012 1:00 PM Answer 1:
                      absolutely no impact, we have been using the same ORACLE_BASE over the years and no problems at all.
                      what made you think it causes the problem ?

                      Nov 14, 2012 8:57 am Question 1:
                      As previously mentioned, Oracle recommends that if separate user are create for Grid Infrastructure and Oracle RDBMS Installations that there be two separate ORACLE_BASE's. If your using two separate user for role separation then what is the advantage/disadvantage of having the same ORACLE_BASE? According to your input, you been using this for years with this setup, so there must be no shared directories in ORACLE_BASE otherwise, you would know what software laid the file GI or RDBMS, right?

                      Nov 13, 2012 1:00 PM Answer 2:
                      before GI installation the GI_HOME will be owned by "grid" user, but when you run the "root.sh" as part of installation then the script changes the necessary permission changes (some directories and files may owned by "root") for RBMS HOME, it always owned by oracle and by running "root.sh" script it will make necessary changes on permissions.

                      Nov 14, 2012 8:57 am Question 2:
                      Can you tell me the owner and permissions for the following directories before and after root.sh for Grid Infrastructure and Oracle RDBMS installation?
                      GI before GI after RDBMS before RDBMS after
                      /u01
                      /u01/app
                      /u01/11.2.0.2

                      Nov 13, 2012 1:00 PM Answer 3:
                      You mean ORACLE_HOME=/u01/app/11.2.0.2/oracle
                      GIRD_HOME=/u01/app/11.2.0.2/grid
                      then there will not be any problems at all as long as permissions on /u01/app/11.2.0.2/oracle and /u01/app/11.2.0.2/grid not changed.

                      Nov 14, 2012 8:57 am Question 3:
                      Yes. I do mean both Oracle_Home and Grid_Home.

                      The reason I am concerned is that the GRID_HOME and ORACLE_HOME share the same path and I am uncertain at what level in directory structure the permission our changed and the impact to out of place updates and upgrades. For example, if 11.2.0.2 has the owner modified by GI is it ok for RDBMS install to change owner. I don't know if this scenario is possible so I looking for someone who knows what the expected behavior is on the directories.

                      Caution: For Oracle Grid Infrastructure for a Cluster installations, note the following restrictions for the Oracle Grid Infrastructure binary home (Grid home):
                      ■ It must not be placed under one of the Oracle base directories, including the Oracle base directory of the Oracle Grid Infrastructure installation owner.
                      ■ It must not be placed in the home directory of an installation owner
                      During installation, ownership of the path to the Grid home is changed to root. This change causes permission errors for other installations.

                      I will been updating HELPFUL and CORRECT after this answer.

                      Thanks.
                      • 8. Re: Directory Setup for RAC and Oracle Restart
                        user10437903
                        Nov 13, 2012 1:00 PM Answer 1:
                        absolutely no impact, we have been using the same ORACLE_BASE over the years and no problems at all.
                        what made you think it causes the problem ?

                        Nov 14, 2012 8:57 am Question 1:
                        As previously mentioned, Oracle recommends that if separate user are create for Grid Infrastructure and Oracle RDBMS Installations that there be two separate ORACLE_BASE's. If your using two separate user for role separation then what is the advantage/disadvantage of having the same ORACLE_BASE? According to your input, you been using this for years with this setup, so there must be no shared directories in ORACLE_BASE otherwise, you would know what software laid the file GI or RDBMS, right?

                        Nov 13, 2012 1:00 PM Answer 2:
                        before GI installation the GI_HOME will be owned by "grid" user, but when you run the "root.sh" as part of installation then the script changes the necessary permission changes (some directories and files may owned by "root") for RBMS HOME, it always owned by oracle and by running "root.sh" script it will make necessary changes on permissions.

                        Nov 14, 2012 8:57 am Question 2:
                        Can you tell me the owner and permissions for the following directories before and after root.sh for Grid Infrastructure and Oracle RDBMS installation?
                        GI before GI after RDBMS before RDBMS after
                        /u01
                        /u01/app
                        /u01/11.2.0.2

                        Nov 13, 2012 1:00 PM Answer 3:
                        You mean ORACLE_HOME=/u01/app/11.2.0.2/oracle
                        GIRD_HOME=/u01/app/11.2.0.2/grid
                        then there will not be any problems at all as long as permissions on /u01/app/11.2.0.2/oracle and /u01/app/11.2.0.2/grid not changed.

                        Nov 14, 2012 8:57 am Question 3:
                        Yes. I do mean both Oracle_Home and Grid_Home.

                        The reason I am concerned is that the GRID_HOME and ORACLE_HOME share the same path and I am uncertain at what level in directory structure the permission our changed and the impact to out of place updates and upgrades. For example, if 11.2.0.2 has the owner modified by GI is it ok for RDBMS install to change owner. I don't know if this scenario is possible so I looking for someone who knows what the expected behavior is on the directories.

                        Caution: For Oracle Grid Infrastructure for a Cluster installations, note the following restrictions for the Oracle Grid Infrastructure binary home (Grid home):
                        ■ It must not be placed under one of the Oracle base directories, including the Oracle base directory of the Oracle Grid Infrastructure installation owner.
                        ■ It must not be placed in the home directory of an installation owner
                        During installation, ownership of the path to the Grid home is changed to root. This change causes permission errors for other installations.

                        I will been updating HELPFUL and CORRECT after this answer.

                        Thanks.