Exadata Storage questions — oracle-tech

    Forum Stats

  • 3,715,749 Users
  • 2,242,853 Discussions
  • 7,845,542 Comments

Discussions

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Exadata Storage questions

Arturo Gutierrez
Arturo Gutierrez Member Posts: 331 Blue Ribbon
edited May 2019 in Exadata

Hello,

I've some questions regarding storage objects:

1. Is about this output:

CellCLI>  list physicaldisk

         252:0           RB8ETV                  normal

         252:1           RBBSNV                  normal

         252:2           RBG8AV                  normal

         252:3           RBJ5UV                  normal

         252:4           RBD0LV                  normal

         252:5           RBGGZV                  normal

         252:6           RB6Y2V                  normal

         252:7           RBJ79V                  normal

         252:8           RBB4GV                  normal

         252:9           RB90JV                  normal

         252:10          RBAAGV                  normal

         252:11          RBDDDV                  normal

         FLASH_10_1      PHLE816600ZE6P4BGN-1    normal

         FLASH_10_2      PHLE816600ZE6P4BGN-2    normal

         FLASH_4_1       PHLE816601BS6P4BGN-1    normal

         FLASH_4_2       PHLE816601BS6P4BGN-2    normal

         FLASH_5_1       PHLE816601MW6P4BGN-1    normal

         FLASH_5_2       PHLE816601MW6P4BGN-2    normal

         FLASH_6_1       PHLE816600ZD6P4BGN-1    normal

         FLASH_6_2       PHLE816600ZD6P4BGN-2    normal

         M2_SYS_0        PHDW816601Q4150A        normal

         M2_SYS_1        PHDW816601T0150A        normal

Why for  every Flash Card create two phy disks?

2. list celldisk detail

         name:                   CD_00_ro09celadm01

         comment:

         creationTime:           2018-11-05T15:15:18+01:00

         deviceName:             /dev/sdc

         devicePartition:        /dev/sdc

         diskType:               HardDisk

         errorCount:             0

         freeSpace:              192M

         id:                     14b48577-3eb7-43f9-b048-420bf9c17e65

         physicalDisk:           RB8ETV

         size:                   8.9094085693359375T

         status:                 normal

Why every celldisk has 192 MB of free space ? Has any special use?

Is not too much of course.

Thanks

Arturo

Answers

  • Nishant Baurai
    Nishant Baurai Member Posts: 215
    edited April 2019

    Arturo,

    What is exadata version X6/X7 ?

    What is rack size ?

    Regards,

    Nishant

  • Arturo Gutierrez
    Arturo Gutierrez Member Posts: 331 Blue Ribbon
    edited April 2019

    Hi Nishant,

    This is a X7-2 QR

    Thanks

    Arturo

  • Kazuhiro
    Kazuhiro Member Posts: 20 Blue Ribbon
    edited April 2019

    Hello Arturo,

    >Why for  every Flash Card create two phy disks?

    I had the same question when I first looked at our X7-2 storage server as follows:

    I'm not quite sure but it seems to me that there are two controllers for each flash card.

    That is probably why cell servers can identify as two separate disk (2.9TB) / card.

    [[email protected] ~]# cellcli -e list physicaldisk where disktype=flashdisk

             FLASH_10_1      PHLE742200W96P4BGN-1    normal

             FLASH_10_2      PHLE742200W96P4BGN-2    normal

             FLASH_4_1       PHLE7422017B6P4BGN-1    normal

             FLASH_4_2       PHLE7422017B6P4BGN-2    normal

             FLASH_5_1       PHLE742200KW6P4BGN-1    normal

             FLASH_5_2       PHLE742200KW6P4BGN-2    normal

             FLASH_6_1       PHLE730300FQ6P4BGN-1    normal

             FLASH_6_2       PHLE730300FQ6P4BGN-2    normal

    [[email protected] ~]# cellcli -e list physicaldisk where disktype=flashdisk detail

             name:                   FLASH_10_1

             deviceName:             /dev/nvme2n1

             diskType:               FlashDisk

             luns:                   10_0

             makeModel:              "Oracle Flash Accelerator F640 PCIe Card"

             physicalFirmware:       QDV1RD24

             physicalInsertTime:     2018-04-12T10:59:21+09:00

             physicalSerial:         PHLE742200W96P4BGN-1

             physicalSize:           2.910957656800746917724609375T

             slotNumber:             "PCI Slot: 10; FDOM: 1"

             status:                 normal

             name:                   FLASH_10_2

             deviceName:             /dev/nvme3n1

             diskType:               FlashDisk

             luns:                   10_0

             makeModel:              "Oracle Flash Accelerator F640 PCIe Card"

             physicalFirmware:       QDV1RD24

             physicalInsertTime:     2018-04-12T10:59:21+09:00

             physicalSerial:         PHLE742200W96P4BGN-2

             physicalSize:           2.910957656800746917724609375T

             slotNumber:             "PCI Slot: 10; FDOM: 2"

             status:                 normal

    https://docs.oracle.com/cd/E87231_01/pdf/E87233.pdf

    p.14

    Best regards,

    Kazuhiro

    Arturo Gutierrez
  • robinsc
    robinsc Member Posts: 518 Silver Badge
    edited May 2019

    I understand that every flash card has multiple independant modules that can be replaced independantly .For the X2-2 there were 4 modules per card so with 4 flash cards we used to have 16 devices. What this means is that if one flash module errors out only that module needs to be replaced rather than all the flash on the card.

    flash1.jpg

    flash2.jpg

    Arturo Gutierrez
  • Kazuhiro
    Kazuhiro Member Posts: 20 Blue Ribbon
    edited May 2019

    Hello robinsc,

    >What this means is that if one flash module errors out only that module needs to be replaced rather than all the flash on the card.

    I just wonder if a CE can replace only a damaged module in a flash card in a data center,

    because I always thought that they replace a whole flash card even if a part of the modules is damaged.

    Technically, it will be possible to identify which module (FDOM) is damaged by sundiag, but I'm not sure

    what they are doing exactly in data centers... at least I've never seen such a case.

    Best regards,

    Kazuhiro

    Arturo Gutierrez
  • Kazuhiro
    Kazuhiro Member Posts: 20 Blue Ribbon
    edited May 2019

    Hello,

    BTW, concerning your second question,

    >Why every celldisk has 192 MB of free space ? Has any special use?

    I think it is because the total size of griddisks is 192MB smaller than the size of celldisk.

    In my environment the freeSpace of celldisk CD_00_xxx01 is 0,

    because the total size of griddisks (DATA_CD_00_xxx01 and RECO_CD_00_xxx01) in the celldisk

    is almost exactly as the same as the size of the celldisk.

    DATA_CD_00_xxx01    6.681640625T

    RECO_CD_00_xxx01    2.22772216796875T

    -> sum 8.90936279296875 T

        =~ 8.9094085693359375T

    So your original question, Why 192MB free space?

    I think you have to ask someone who designed or created the

    diskgroups and grid disks.

    CellCLI> list celldisk detail

             name:                   CD_00_xxx01

             comment:

             creationTime:           2018-04-16T16:39:30+09:00

             deviceName:             /dev/sda

             devicePartition:        /dev/sda

             diskType:               HardDisk

             errorCount:             0

             freeSpace:              0

             id:                     6dcf8810-77d9-48ac-8339-b7eba6774905

             physicalDisk:           R464WK

             size:                   8.9094085693359375T

             status:                 normal

            

            

    CellCLI> list griddisk detail

             name:                   DATA_CD_00_xxx01

             asmDiskGroupName:       DATA

             asmDiskName:            DATA_CD_00_XXX01

             asmFailGroupName:       XXX01

             availableTo:

             cachedBy:               FD_00_xxx01

             cachingPolicy:          default

             cellDisk:               CD_00_xxx01

             comment:                "Cluster dm01-cluster diskgroup DATA"

             creationTime:           2018-04-16T16:48:00+09:00

             diskType:               HardDisk

             errorCount:             0

             id:                     3157c312-f66c-4714-8440-334240b781a8

             size:                   6.681640625T

             status:                 active

    ...

             name:                   RECO_CD_00_xxx01

             asmDiskGroupName:       RECO

             asmDiskName:            RECO_CD_00_XXX01

             asmFailGroupName:       XXX01

             availableTo:

             cachedBy:

             cachingPolicy:          none

             cellDisk:               CD_00_xxx01

             comment:                "Cluster dm01-cluster diskgroup RECO"

             creationTime:           2018-04-16T16:48:54+09:00

             diskType:               HardDisk

             errorCount:             0

             id:                     b5b9de6a-703f-4dad-9282-5e7511e8bc78

             size:                   2.22772216796875T

             status:                 active

    Best regards,

    Kazuhiro

    Arturo Gutierrez
Sign In or Register to comment.