This discussion is archived
5 Replies Latest reply: Jul 12, 2013 8:02 AM by 509894 RSS

Required_mirror_free_mb calculation on Exadata

Saurabh Gupta-OC Newbie
Currently Being Moderated

Hi Folks,

 

We have Exadata mixed configuration of HP and HC disks. I have some confusion on Required_mirror_free_mb size accumulated by ASM in normal redundancy.

 

if I look at current configuration we have, the required_mirror_free_mb is something different, would appreciate if you can look into this and clarify my doubts.

 

Disk size allocations:

 

SQL> select failgroup,sum(total_mb/1024/1024) TOTAL_TB from v$asm_disk group by failgroup order by failgroup;

 

FAILGROUPTOTAL_TB
DW01CEL016.47998
DW01CEL026.47998
DW01CEL036.47998
DW01CEL046.47998
DW01CEL056.47998
DW01CEL066.47998
EX01ECEL0132.67508
EX01ECEL0232.67508
EX01ECEL0332.67508
EX01ECEL0432.67508

 

SQL> select name,total_mb/1024 Total_GB,free_mb/1024 Free_GB,cold_used_mb/1024 Cold_GB,required_mirror_free_mb/1024 Req_GB,usable_file_mb/1024 Usable_GB from v$asm_diskgroup;

 

NAMETOTAL_GBFREE_GBCOLD_GBREQ_GBUSABLE_GB
DATA309606151.31624808.685160495.6582
SYSTEMDG1747.51745.8911.609375291.25727.3203
DATA_EXP7680058168.6318631.371920019484.32
RECO_EXP35905.519704.3716201.1372006252.184
DATA_DEV1920018492.1707.902348006846.049
RECO_DEV7685.257576.875108.3751921.3132827.781
DBFS_EXP1351.8751351.2110.664063337.9688506.6211

 

Here I see that the required_free_mb space is too high and consuming almost 38 TB of storage. We have mixed disk types in our configuration (600 GB and 3 TBs). Diskgroups which suffixes with EXP or DEV are having 3 TBs disks.

 

What I understand from Oracle documents that required_mirror_free_mb size would be one disk size on each storage server, hence in this case we have 6 storage cells are having 600 GB disks each and 4 cells are of 3 TBS disks. So the complete required_mirror_free_mb would be 600*6+4*3072=15888 GB but here it is around 38 TB.

 

ASMCMD>lsdg

 

NameTotal_MBFree_MBReq_mirror_free_mbUsable_file_MB
DATA/3170304062989565283840507558
DATA_DEV/196608001893590849152007010354
DATA_EXP/78643200595646801966080019951940
DBFS_EXP/13843201383640346080518780
RECO_DEV/7869696775872019674242895648
RECO_EXP/367672322376423673728008195718
SYSTEMDG/17894401787792298240744776

 

Thanks in advance.

 

 

Regards

Saurabh

  • 1. Re: Required_mirror_free_mb calculation on Exadata
    tychos Expert
    Currently Being Moderated

    Hi Saurabh,

    Your 38T is the result of 32T (HC) and 6TB (HP) disks which is actually not the correct way to calculate as you have asm disk groups separated for HC and HP.

    So your groups on EX01 (*dev and *exp) have a different bigger required free then the groups on DW01 (data and systemdg).

    Also your figures are all showing raw disk sizes which doesn't tell the net amount of storage you can use in a disk group.

    Can you run the following sql and post the output.

    ..

    select name,

           round(TOTAL_MB/decode(type,'NORMAL',2,'HIGH',3,1)/1024) total_GB,

           round(usable_FILE_mb/1024) usable_free_GB,

           round((HOT_USED_MB+COLD_USED_MB)/decode(type,'NORMAL',2,'HIGH',3,1)/1024) data_use_GB,

           round(REQUIRED_MIRROR_FREE_MB/decode(type,'NORMAL',2,'HIGH',3,1)/1024) Req_free_redun_GB,

           type

    from v$asm_diskgroup

    /

    ..

    The first column is the sum of the next free columns.

    The usable_free_GB can actually become negative meaning you used space needed to be able to restore redundancy after the loss of a failure group.

    Regards,

    Tycho

  • 2. Re: Required_mirror_free_mb calculation on Exadata
    Saurabh Gupta-OC Newbie
    Currently Being Moderated

    Thanks Tycho for your reply.

     

    Please find below output of the query you given:

     

    NAMETOTAL_GBUSABLE_FREE_GBDATA_USE_GBREQ_FREE_REDUN_GBTYPE
    DATA15480496124042580NORMAL
    SYSTEMDG8747271146NORMAL
    DATA_EXP384001948493169600NORMAL
    RECO_EXP17953801063423600NORMAL
    DATA_DEV960068463542400NORMAL
    RECO_DEV3843282854961NORMAL
    DBFS_EXP6765070169NORMAL

     

    As per Oracle DOC 798458.1, that value for REQUIRED_MIRROR_FREE_MB is the size of just only 1 disk.

     

    Couple of doubts:

     

    While in calculation why Req_mirror_free_mb is calculated for redundancy level (2) in normal redundancy when we require just one disk in Normal redundancy.

    Secondly, I have checked and found that once the usable space gets full then it shows negative value even though there is space available in req_mirror_free_mb.

     

    Also, as per current system details we have:

     

    No. of failure groups : 10

    Redundancy : Normal

    Disk Sizes in failure groups : 600 GB / 3 TB

    No. of fail groups having 3 TB Size Disks : 4

    No. of fail groups having 600 GB Size Disks : 6

     

     

    So as per Oracle documents that value for REQUIRED_MIRROR_FREE_MB is the size of just only 1 disk

    Hence in this case,

     

     

    Required_mirror_free_mb = 4*3000 + 6*600=15,600 GB (One disk in each fail group)

     

    But as per system currently req_mirror_free_mb is 38 TB and if we divide it by 2 in normal redundancy case then again it would be 19 TB (But dont know why we need to divide it by 2 as we are not storing any data in req_mirror_free_mb place, as it is only required for redundancy restore purpose when there is a disk failure)

     

    Kindly check and clear my doubts.

     

    Your suggestion and effort is highly appreciated.

     

    Regards.

  • 3. Re: Required_mirror_free_mb calculation on Exadata
    tychos Expert
    Currently Being Moderated

    Hi,

     

    As per Oracle DOC 798458.1, that value for REQUIRED_MIRROR_FREE_MB is the size of just only 1 disk.

     

    The note is about oracle 10g using asm on physical disks not Exadata and storage cells.

    On Exadata this should read: required_mirror_free_mb in a disk group is the size of just only 1 grid disk.

     

     

    No. of failure groups :10

     

    Actually you have a division in 6 failure groups (dw01) and 4 failure groups (ex01) which have no dependency. For example the full loss of dw01cel01 has only effect on data and systemdg but not on the other disk groups as hey run on ex01.

     

    Regards,

     

    Tycho

  • 4. Re: Required_mirror_free_mb calculation on Exadata
    robinsc Explorer
    Currently Being Moderated

    On exadata the way I understand it the required_mirror_free_mb is the buffer amount of space you will still have left to maintain your current level of redundancy ( high or normal) in case of failure of any one failgroup.

    As in exadata one whole cell is taken to be one faailgroup  the required_miror_free_mb is at least the size of one cell ( 12 disks).

  • 5. Re: Required_mirror_free_mb calculation on Exadata
    509894 Explorer
    Currently Being Moderated

    I explained the USABLE_FILE_MB in my blog also you can find the REQUIRED_MIRROR_FREE_MB explanation there.

     

    It is a practical way of showing how things work. I wrote it last night.

     

     

    “usable_file_mb” explained for Exadata asm structure | Mehmet Eser's Oracle Blog

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points