1 2 Previous Next 15 Replies Latest reply: Sep 4, 2013 9:35 AM by Billy~Verreynne RSS

    ASM disk missing, how to find which disk?

    John-MK

      Hello,

      In the ASM alert log, I only see its name as in v$asm_disk NAME column. I have to tell Linux guys which disk is broken. Could you please guide me in this regard?

      DB=11.2.0.3 on RHL5

       

      WARNING: PST-initiated drop of 1 disk(s) in group 1(.261677061))

      SQL> alter diskgroup DATA01 drop disk F1DATA02  force /* ASM SERVER */

      Also, the following statement returns no rows selected: Although I am getting asm disk mirroring/rebalance alarms..

       

      select INST_ID, OPERATION, STATE, POWER, SOFAR, EST_WORK, EST_RATE, EST_MINUTES from GV$ASM_OPERATION;

       

      asm_power_limit is 1.

       

      How can I manually do disk rebalance quickly? instead of increasing the asm_power_limit

       

      Thanks a lot.

       

      Best Regards.

        • 1. Re: ASM disk missing, how to find which disk?
          Javier Francisco Ruiz

          Is your asm configured with asmlib if so you can do a listdisk.

           

           

          To change the rebalance power on an on going rebalance operation you can do the following

          alter diskgroup <diskgroup_name> rebalance power 6;

          • 2. Re: ASM disk missing, how to find which disk?
            John-MK

            Hello,

             

            Yes, asm is configured with asm libraries.

            /etc/init.d/oracleasm listdisks

             

            But listdisks lists the disks according to the NAME given in ASM. It do not lists any disk which is missing. Please correct me if I am wrong, or missing something.

             

            I want to know the the missing DISK info according to the multipath -l command. Fo example, I can tell that dm-40 is missing on the machine or perhaps finding it with any LUN, etc?

             

            Thank you.

            Best Regards

            • 3. Re: ASM disk missing, how to find which disk?
              Javier Francisco Ruiz

              Execute the following this will output the path to the disk then your storage admin can get what he needs.

               

              -------------------------------------------------------------------------------------------------------------

              etc/init.d/oracleasm querydisk -d `/etc/init.d/oracleasm listdisks -d` | \

              cut -f2,10,11 -d" " | \

              perl -pe 's/"(.*)".*\[(.*), *(.*)\]/$1 $2 $3/g;' | \

              while read v_asmdisk v_minor v_major

              do

              v_device=`ls -la /dev | grep " $v_minor, *$v_major " | awk '{print $10}'`

              echo "ASM disk $v_asmdisk based on /dev/$v_device [$v_minor, $v_major]"

              done

              -------------------------------------------------------------------------------------------

              Example output

              ASM disk DG_DISK1A based on /dev/sdm1 [8, 193]

              ASM disk DG_DISK1B based on /dev/sdp1 [8, 241]

              ASM disk DG_DISK2A based on /dev/sdn1 [8, 209]

              ASM disk DG_DISK2B based on /dev/sdq1 [65, 1]

              ASM disk DG_DISK3A based on /dev/sdo1 [8, 225]

              ASM disk DG_DISK3B based on /dev/sdr1 [65, 17]

              ASM disk DISK1 based on /dev/sdg1 [8, 97]

              ASM disk DISK2 based on /dev/sdh1 [8, 113]

              ASM disk DISK3 based on /dev/sdi1 [8, 129]

              ASM disk DISK4 based on /dev/sdj1 [8, 145]

              ASM disk FRA1 based on /dev/sdk1 [8, 161]

              ASM disk FRA_DISK1A based on /dev/sds1 [65, 33]

              ASM disk FRA_DISK1B based on /dev/sdt1 [65, 49]

              • 4. Re: ASM disk missing, how to find which disk?
                John-MK

                Thanks a lot.

                 

                The output looks very nice though, but unfortunately It can not be said from this out put which disk is missing.

                 

                The example output from the total of over 30 disks. As said before, it only lists the NAME given in ASM and dm-64, dm-67 but they are NOT missing of course...

                ASM disk  F1DATA01 based on /dev/dm-64 [652, 64]

                ASM disk F1DATA03 based on /dev/dm-67 [652, 67]

                 

                The other way for admin could be that check all the hundred disks and find which one is missing :-(

                 

                Best Regards

                • 5. Re: ASM disk missing, how to find which disk?
                  Javier Francisco Ruiz

                  What are you trying to do are you trying to mount a diskgroup and it is failing because of missing disk? The alert log will show you the name of the disk that is missing which then you can use that name to get the path of the disk. You can then give the path location and the disk admin can verify the disk.

                  • 6. Re: ASM disk missing, how to find which disk?
                    John-MK

                    ORA-16198 occurred in the Primary database alert log. As soon as this error occurred I got the monitoring alarm about the ASM-Mirroring issue. I noticed in the Standby database that disk02 is missing.. ASM-Mirring / rebalance is automatically fine after half an hour or so...

                     

                    But the linux-admin(s) wants to know the name of the disk so that they find it and repair, and then I again allocate that disk to ASM instance.

                     

                    I am not doing anything at instance level at the moment. Just wanted to know the name of the disk at OS level not at Oracle level. The NAME,PATH from the v$asm_disk does not tell about missing disks...Also the path is something like ORC:FDATA01, and I dont think this helps to locate the disk..

                     

                    Best Regards,

                    • 7. Re: ASM disk missing, how to find which disk?
                      Catch-22

                      How about logging on to your ASM instance to query the disks in your disk groups?

                       

                      See example 6.3 at http://docs.oracle.com/cd/E16338_01/server.112/e10500/asmviews.htm

                       

                      Then, when you found the broken ASM disk, use the the following OS command to query the underlying physical device:

                       

                      /etc/init.d/oracleasm querydisk -p  ASMVOL

                      • 8. Re: ASM disk missing, how to find which disk?
                        John-MK

                        Thank you.

                         

                        I have queried all the v$asm views but nobody shows the information about any disk that is broken, nor /etc/init.d/oracleasm listdisks and querydisks very output is already suggested above by JavierFranciscoRuiz

                         

                        Best Regards

                        • 9. Re: ASM disk missing, how to find which disk?
                          Javier Francisco Ruiz

                          In the asm alert log it will tell you the ASM disk name that has failed

                           

                          Then with the command I showed you can get the path for the disk

                           

                          For example if the alert log was stating that the disk F1DATA01 has gone offline due to some issue you would give the /dev/dm-64 path to the disk admin then they can get the rest.

                           

                          ASM disk  F1DATA01 based on /dev/dm-64 [652, 64]

                          • 10. Re: ASM disk missing, how to find which disk?
                            John-MK

                            I am little confused here now :-)

                             

                            That is the point. From where I get this /dev/dm-* path...  of the missing disk?

                             

                            I lose hope :-(

                             

                            One more time:

                             

                            Alert log will only tell the name of the disk according to v$asm_disk...for example FDISK02 ..nothing else.....

                             

                            listdisks,,,,querydisks of asmlib features only lists ONLINE disks not OFFLINE... The disk which I mentioned is ONLINE its not BROKEN that dm-64

                            • 11. Re: ASM disk missing, how to find which disk?
                              Javier Francisco Ruiz

                              Use the oracleasm querydisk to list all disk defined for ASM even if they are offline. That will give you which disk that was defined.

                              • 12. Re: ASM disk missing, how to find which disk?
                                Billy~Verreynne

                                John-MK wrote:

                                 

                                That is the point. From where I get this /dev/dm-* path...  of the missing disk?

                                Look at disk labels.

                                 

                                I wrote the following little hack and it seems to work for me - find-disks.sh <asm_diskgroup>:

                                 

                                [root@dev1 ~]# cat ./find-disks.sh 
                                #!/bin/bash

                                 

                                [ "$1" = "" ] && echo "$0 <ASM_DISKGROUP>" && exit 1
                                TMPFILE=/tmp/~$$

                                 

                                for DISK in /dev/dm-*
                                do
                                        od --read-bytes=128 --format=a $DISK | sed 's/nul/   /g'  | sed 's/ //g' > $TMPFILE

                                 

                                        grep ORCLDISK $TMPFILE >/dev/null
                                        if [ "$?" -eq 0 ]; then
                                                grep $1 $TMPFILE >/dev/null
                                                [ "$?" -eq 0 ]  && echo "Disk $DISK has an ASM signature and text '$1' in the disk header." && od --read-bytes=128 --format=a $DISK && echo ""
                                        fi
                                done

                                 

                                [ -f $TMPFILE ] && rm $TMPFILE

                                 

                                #eof
                                [root@dev1 ~]#
                                [root@dev1 ~]# ./find-disks.sh  ACFS
                                Disk /dev/dm-11 has an ASM signature and text 'ACFS' in the disk header.
                                0000000 soh stx soh soh nul nul nul nul nul nul nul nul dc4   -   r   S
                                0000020 nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul
                                0000040   O   R   C   L   D   I   S   K nul nul nul nul nul nul nul nul
                                0000060 nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul
                                0000100 nul nul  sp  vt nul nul soh etx   A   C   F   S   1   _   0   0
                                0000120   0   0 nul nul nul nul nul nul nul nul nul nul nul nul nul nul
                                0000140 nul nul nul nul nul nul nul nul   A   C   F   S   1 nul nul nul
                                0000160 nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul
                                0000200

                                 

                                Disk /dev/dm-27 has an ASM signature and text 'ACFS' in the disk header.
                                0000000 soh stx soh soh nul nul nul nul soh nul nul nul syn   -   r   S
                                0000020 nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul
                                0000040   O   R   C   L   D   I   S   K nul nul nul nul nul nul nul nul
                                0000060 nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul
                                0000100 nul nul  sp  vt soh nul soh etx   A   C   F   S   1   _   0   0
                                0000120   0   1 nul nul nul nul nul nul nul nul nul nul nul nul nul nul
                                0000140 nul nul nul nul nul nul nul nul   A   C   F   S   1 nul nul nul
                                0000160 nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul
                                0000200

                                 

                                [root@dev1 ~]#

                                 

                                It dumps the 1st 128 bytes of the disk to temp file, checks for the string ORCLDISK (all ASM disks have this in their headers), and if successful, uses grep to check if the search string (ASM diskgroup name or ASM diskgroup disk name) exists in the header.

                                • 14. Re: ASM disk missing, how to find which disk?
                                  Catch-22

                                  This is actually not necessary if you have Oracle ASMLib installed, which can query the disks using the scandisk and listdisk commands. The OP has or had a different problem though, which is not being able to find out which device of his diskgroups is unavailable and offline.

                                  1 2 Previous Next