9 Replies Latest reply: Mar 3, 2010 12:36 AM by 756561 RSS

    ASM DG creation fails on sles11_x86_64 with 11gR1

    756561
      Hi,

      I have sles11x86_64 installed and using Veritas Volume manager raw disks for ASM but ASM dg creation fails with error mentioned below;

      ls /dev/vx/rdsk/asmdg/asmvol
      /dev/vx/rdsk/asmdg/asmvol
      test:~ # ls -l /dev/vx/rdsk/asmdg/asmvol
      crw-rw---- 1 oracle oinstall 199, 65533 Feb 26 12:08 /dev/vx/rdsk/asmdg/asmvol
      test:~ # su - oracle
      oracle@test:~> bash
      oracle@test:~> export ORACLE_SID=+ASM
      oracle@test:~> export ORACLE_HOME=/oracle/orabin/product/11.1.0/db_1
      oracle@test:~> cd /oracle/orabin/product/11.1.0/db_1/bin


      cat /etc/oratab
      #

      # This file is used by ORACLE utilities. It is created by root.sh
      # and updated by the Database Configuration Assistant when creating
      # a database.

      # A colon, ':', is used as the field terminator. A new line terminates
      # the entry. Lines beginning with a pound sign, '#', are comments.
      #
      # Entries are of the form:
      # $ORACLE_SID:$ORACLE_HOME:<N|Y>:
      #
      # The first and second fields are the system identifier and home
      # directory of the database respectively. The third filed indicates
      # to the dbstart utility that the database should , "Y", or should not,
      # "N", be brought up at system boot time.
      #
      # Multiple entries with the same $ORACLE_SID are not allowed.
      #
      #
      +ASM:/oracle/orabin/product/11.1.0/db_1:N

      test./sqlplus /nolog

      SQL*Plus: Release 11.1.0.6.0 - Production on Fri Feb 26 12:25:32 2010

      Copyright (c) 1982, 2007, Oracle. All rights reserved.

      SQL> connect /as sysdba
      Connected.
      SQL> select name from v$asm_diskgroup;

      no rows selected

      SQL> select path from v$asm_disk;

      PATH
      --------------------------------------------------------------------------------
      /dev/vx/rdsk/asmdg/asmvol

      SQL> create diskgroup asm_dg external redundancy disk '/dev/vx/rdsk/asmdg/asmvol';
      create diskgroup asm_dg external redundancy disk '/dev/vx/rdsk/asmdg/asmvol'
      *
      ERROR at line 1:
      ORA-15018: diskgroup cannot be created
      ORA-15201: disk /dev/vx/rdsk/asmdg/asmvol contains a valid RDBMS file


      SQL> select name from v$asm_diskgroup;

      no rows selected

      SQL> create diskgroup asm_dg external redundancy disk '/dev/vx/rdsk/asmdg/asmvol' -f;
      create diskgroup asm_dg external redundancy disk '/dev/vx/rdsk/asmdg/asmvol' -f
      *
      ERROR at line 1:
      ORA-00933: SQL command not properly ended


      SQL> create diskgroup asm_dg external redundancy disk '/dev/vx/rdsk/asmdg/asmvol' force;
      create diskgroup asm_dg external redundancy disk '/dev/vx/rdsk/asmdg/asmvol' force
      *
      ERROR at line 1:
      ORA-15018: diskgroup cannot be created
      ORA-15080: synchronous I/O operation to a disk failed
      ORA-15080: synchronous I/O operation to a disk failed
      ORA-15080: synchronous I/O operation to a disk failed
      ORA-15080: synchronous I/O operation to a disk failed


      SQL> create diskgroup asm_dg external redundancy disk '/dev/vx/rdsk/asmdg/asmvol';
      create diskgroup asm_dg external redundancy disk '/dev/vx/rdsk/asmdg/asmvol'
      *
      ERROR at line 1:
      ORA-15018: diskgroup cannot be created
      ORA-15201: disk /dev/vx/rdsk/asmdg/asmvol contains a valid RDBMS file


      SQL> create diskgroup asm_dg external redundancy disk '/dev/vx/rdsk/asmdg/asmvol' force;
      create diskgroup asm_dg external redundancy disk '/dev/vx/rdsk/asmdg/asmvol' force
      *
      ERROR at line 1:
      ORA-15018: diskgroup cannot be created
      ORA-15080: synchronous I/O operation to a disk failed
      ORA-15080: synchronous I/O operation to a disk failed
      ORA-15080: synchronous I/O operation to a disk failed
      ORA-15080: synchronous I/O operation to a disk failed

      SQL>

      can some body help on this.

      Also wanted to know the background process during ASM creation.

      ~Dhananjay

      Edited by: user9075978 on Feb 26, 2010 4:18 AM
        • 1. Re: ASM DG creation fails on sles11_x86_64 with 11gR1
          KeshavM
          Force the partition table on the second node to be reread

          # partprobe

          Initialize all shared disks on one node (execute as user root)

          dd if=/dev/zero of=/dev/sdc1 bs=1M count=10
          dd if=/dev/zero of=/dev/sdc2 bs=1M count=10
          dd if=/dev/zero of=/dev/sdc3 bs=1M count=10
          dd if=/dev/zero of=/dev/sdc4 bs=1M count=10
          dd if=/dev/zero of=/dev/sdd1 bs=1M count=10
          dd if=/dev/zero of=/dev/sdd2 bs=1M count=10
          dd if=/dev/zero of=/dev/sdd3 bs=1M count=10
          dd if=/dev/zero of=/dev/sdd4 bs=1M count=10
          dd if=/dev/zero of=/dev/sde1 bs=1M count=10
          dd if=/dev/zero of=/dev/sde2 bs=1M count=10
          dd if=/dev/zero of=/dev/sde3 bs=1M count=10
          dd if=/dev/zero of=/dev/sde4 bs=1M count=10
          dd if=/dev/zero of=/dev/sdf3 bs=1M count=10
          dd if=/dev/zero of=/dev/sdf4 bs=1M count=10


          Install ASM lib (execute as user root) on all cluster nodes

               rpm -i /ptc/oms/asmlib/*
               
          Configure ASM lib (set driver owner to oracle and group to dba) on all cluster nodes

          /etc/init.d/oracleasm configure

          Label ASM lib disks on one node (execute as user root)

          /etc/init.d/oracleasm createdisk data01 /dev/sdc3
          /etc/init.d/oracleasm createdisk data02 /dev/sdc4
          /etc/init.d/oracleasm createdisk flash01 /dev/sdd3
          /etc/init.d/oracleasm createdisk flash02 /dev/sdd4
          /etc/init.d/oracleasm createdisk flash03 /dev/sde3
          /etc/init.d/oracleasm createdisk flash04 /dev/sde4
          /etc/init.d/oracleasm createdisk flash05 /dev/sdf3
          /etc/init.d/oracleasm createdisk flash06 /dev/sdf4

          Scan and list the disks on the other node (execute as user root)

          /etc/init.d/oracleasm scandisks
          /etc/init.d/oracleasm listdisks

          Set up udev to configure block device permissions (as user root)

          vi /etc/udev/permissions.d/49-oracle.permissions

          sdc1:root:oinstall:0640
          sdd1:root:oinstall:0640
          # Voting
          sdc2:crs:oinstall:0640
          sdd2:crs:oinstall:0640
          sde2:crs:oinstall:0640
          # ASM spfile
          sde1:asm:oinstall:0640
          # ASM devices
          sdc3:asm:oinstall:0640
          sdc4:asm:oinstall:0640
          sdd3:asm:oinstall:0640
          sdd4:asm:oinstall:0640
          sde3:asm:oinstall:0640
          sde4:asm:oinstall:0640
          sdf3:asm:oinstall:0640
          sdf4:asm:oinstall:0640


          Restart the udev service (execute as user root)

          /sbin/udevstart
          • 2. Re: ASM DG creation fails on sles11_x86_64 with 11gR1
            756561
            I believe this if for OS raw devices which work fine but it fails if we specify the device which are under Veritas control.

            we specify disk path as '/dev/vx/rdsk/asmdg/asmvol' during DBCA in ASM section.

            where it fails.
            • 3. Re: ASM DG creation fails on sles11_x86_64 with 11gR1
              Ronny Egner
              we specify disk path as '/dev/vx/rdsk/asmdg/asmvol' during DBCA in ASM section.
              Veritas VM is not supported. In addition to that: You simple dont need it.

              Why doing the things ASM does for you twice?

              Get rid of Veritas and your errors will be gone.

              --
              Ronny Egner

              My Blog: http://blog.ronnyegner-consulting.de
              • 4. Re: ASM DG creation fails on sles11_x86_64 with 11gR1
                756561
                I believe Veritas VM is supported as I had done in past and currently on x86_64 PPC platform(sles10_sp3).

                but hitting this issue only on sles11_x86_64 platform.
                • 5. Re: ASM DG creation fails on sles11_x86_64 with 11gR1
                  Ronny Egner
                  I believe Veritas VM is supported as I had done in past and currently on x86_64 PPC platform(sles10_sp3).
                  What makes you think so beside "it worked in the past" ? On Metalink all i found was a certification for "Vertitas Cluster File System". So i would recommend you to double-check for a certification against your platform and software stack. This will save a lot of trouble in support cases.

                  Btw: I may be completely wrong but i dont know a platform named "x86_64 PPC". I guess it is a type and you meant "it works on x86_64 and PPC64".

                  --
                  Ronny Egner

                  My Blog: http://blog.ronnyegner-consulting.de

                  Edited by: Ronny Egner on Mar 1, 2010 7:16 AM
                  • 6. Re: ASM DG creation fails on sles11_x86_64 with 11gR1
                    756561
                    Hi,

                    Thanks for the continuous support on this.

                    Actually I mean I have done it two weeks back o sles10_sp3_ppc platform and rhel5_u4 on x86_64 platform.

                    But it fails only on sles11_x86_64 platfrom with below errors.

                    We use raw device not creating Veritas file systems.

                    The disks(raw device are only under Volume Manager disks)no Veritas or any other file system on it.

                    e,g taken one disk under VM,created VM diskgroup, set permission for oracle and specify raw device path"/dev/vx/rdsk/asmdg/asmvol"

                    ----------------error

                    SQL> create diskgroup asm_dg external redundancy disk
                    SQL> '/dev/vx/rdsk/asmdg/asmvol' force;
                    create diskgroup asm_dg external redundancy disk '/dev/vx/rdsk/asmdg/asmvol' force
                    *
                    ERROR at line 1:
                    ORA-15018: diskgroup cannot be created
                    ORA-15034: disk '/dev/vx/rdsk/asmdg/asmvol' does not require the FORCE option


                    SQL> create diskgroup asm_dg external redundancy disk
                    SQL> '/dev/vx/rdsk/asmdg/asmvol';
                    create diskgroup asm_dg external redundancy disk '/dev/vx/rdsk/asmdg/asmvol'
                    *
                    ERROR at line 1:
                    ORA-15018: diskgroup cannot be created
                    Linux-x86_64 Error: 14: Bad address
                    Additional information: -1
                    Additional information: 655360
                    Linux-x86_64 Error: 14: Bad address
                    Additional information: -1
                    Additional information: 655360
                    Linux-x86_64 Error: 14: Bad address
                    Additional information: -1
                    Additional information: 655360
                    Linux-x86_64 Error: 14: Bad address
                    Additional information: -1
                    Additional information: 655360
                    Linux-x86_64 Error: 14: Bad address
                    Additional information: -1
                    Additional information: 655360
                    Linux-x86_64 Error: 14: Bad address
                    Additional information: -1
                    Additional information: 655360
                    Linux-x86_64 Error: 14: Bad address
                    Additional information: -1
                    Additional information: 655360
                    Linux-x86_64 Error: 14: Bad address
                    Additional information: -1
                    Additional information: 655360
                    Linux-x86_64 Error: 14: Bad address
                    Additional information: -1
                    Additional information: 655360
                    Linux-x86_64 Error: 14: Bad address
                    Additional information: -1
                    Additional information: 655360
                    Linux-x86_64 Error: 14: Bad address
                    Addition
                    ============error=====

                    Any thoughts?

                    Thanks,
                    Dhananjay
                    • 7. Re: ASM DG creation fails on sles11_x86_64 with 11gR1
                      Ronny Egner
                      Any thoughts?
                      Actually I mean I have done it two weeks back o sles10_sp3_ppc platform and rhel5_u4 on x86_64 platform.
                      But it fails only on sles11_x86_64 platfrom with below errors.
                      If it works with RHEL5 on x86_64 and you are on the same platform but with SLES 11 x86_64 i would
                      assume there are some possibilities: for instance problems in conjunction SLES 11 + VxVM. Or misconfigured
                      Volumes and so on. Or even: Not supported.

                      My recommendation is to open an Service Request at Oracle or/and Symantec.

                      --
                      Ronny Egner

                      My Blog: http://blog.ronnyegner-consulting.de
                      • 8. Re: ASM DG creation fails on sles11_x86_64 with 11gR1
                        756561
                        Thanks for your inputs....
                        • 9. Re: ASM DG creation fails on sles11_x86_64 with 11gR1
                          756561
                          Hi ,

                          I am able to create the ASM diskgroup now as there are some OS specific Code changes by Veitas for VM.

                          Anyway,

                          How Should I to set I/O mode synchronous in Oracle...

                          Edited by: user9075978 on Mar 2, 2010 10:36 PM