6 Replies Latest reply: Feb 14, 2013 4:20 AM by 936049 RSS

    M5000 splitting existing domain

    936049
      Hi

      We have an M5000 with one domain running Sun Cluster on it. We would like to split the domain into two domains. We have 2 CPU Boards / 4 Mem Boards and 2 IO
      Is it possible to split it in a way that the existing Domain has 5 of 8 CPU and 3 Memboard of 4 plus one IO board?
      How could I split the existing domain without affecting it? I can easily shutdown the M5000 as I can switch the ressources to the other node first.

      Here the showhardconf

      XSCF> showhardconf
      SPARC Enterprise M5000;
      + Serial:BCF092004F; Operator_Panel_Switch:Locked;
      + Power_Supply_System:Single; SCF-ID:XSCF#0;
      + System_Power:On; System_Phase:Cabinet Power On;
      Domain#0 Domain_Status:Running;

      MBU_B Status:Normal; Ver:0201h; Serial:BC0920031A ;
      + FRU-Part-Number:CF00541-0478 07 /541-0478-07 ;
      + Memory_Size:64 GB;
      + Type:1;
      CPUM#0-CHIP#0 Status:Normal; Ver:0401h; Serial:PP0910014R ;
      + FRU-Part-Number:CA06761-D202 E0 /375-3568-05 ;
      + Freq:2.400 GHz; Type:32;
      + Core:4; Strand:2;
      CPUM#0-CHIP#1 Status:Normal; Ver:0401h; Serial:PP0910014R ;
      + FRU-Part-Number:CA06761-D202 E0 /375-3568-05 ;
      + Freq:2.400 GHz; Type:32;
      + Core:4; Strand:2;
      CPUM#1-CHIP#0 Status:Normal; Ver:0401h; Serial:PP090903WE ;
      + FRU-Part-Number:CA06761-D202 E0 /375-3568-05 ;
      + Freq:2.400 GHz; Type:32;
      + Core:4; Strand:2;
      CPUM#1-CHIP#1 Status:Normal; Ver:0401h; Serial:PP090903WE ;
      + FRU-Part-Number:CA06761-D202 E0 /375-3568-05 ;
      + Freq:2.400 GHz; Type:32;
      + Core:4; Strand:2;
      CPUM#2-CHIP#0 Status:Normal; Ver:0401h; Serial:PP091702L0 ;
      + FRU-Part-Number:CA06761-D202 E0 /375-3568-05 ;
      + Freq:2.400 GHz; Type:32;
      + Core:4; Strand:2;
      CPUM#2-CHIP#1 Status:Normal; Ver:0401h; Serial:PP091702L0 ;
      + FRU-Part-Number:CA06761-D202 E0 /375-3568-05 ;
      + Freq:2.400 GHz; Type:32;
      + Core:4; Strand:2;
      CPUM#3-CHIP#0 Status:Normal; Ver:0401h; Serial:PP091604HZ ;
      + FRU-Part-Number:CA06761-D202 E0 /375-3568-05 ;
      + Freq:2.400 GHz; Type:32;
      + Core:4; Strand:2;
      CPUM#3-CHIP#1 Status:Normal; Ver:0401h; Serial:PP091604HZ ;
      + FRU-Part-Number:CA06761-D202 E0 /375-3568-05 ;
      + Freq:2.400 GHz; Type:32;
      + Core:4; Strand:2;
      MEMB#0 Status:Normal; Ver:0101h; Serial:BF09175UHL ;
      + FRU-Part-Number:CF00541-0545 06 /541-0545-06 ;
      MEM#0A Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-48256a66;
      + Type:2A; Size:2 GB;
      MEM#0B Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-48256a7c;
      + Type:2A; Size:2 GB;
      MEM#1A Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-48256a51;
      + Type:2A; Size:2 GB;
      MEM#1B Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-48256a4d;
      + Type:2A; Size:2 GB;
      MEM#2A Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-48256a7d;
      + Type:2A; Size:2 GB;
      MEM#2B Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-48256a52;
      + Type:2A; Size:2 GB;
      MEM#3A Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-48256a4c;
      + Type:2A; Size:2 GB;
      MEM#3B Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-48256a75;
      + Type:2A; Size:2 GB;
      MEMB#1 Status:Normal; Ver:0101h; Serial:BF091863HN ;
      + FRU-Part-Number:CF00541-0545 06 /541-0545-06 ;
      MEM#0A Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-482569c2;
      + Type:2A; Size:2 GB;
      MEM#0B Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-48256a73;
      + Type:2A; Size:2 GB;
      MEM#1A Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-48256a79;
      + Type:2A; Size:2 GB;
      MEM#1B Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-48256a83;
      + Type:2A; Size:2 GB;
      MEM#2A Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-48256ac8;
      + Type:2A; Size:2 GB;
      MEM#2B Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-48256ac9;
      + Type:2A; Size:2 GB;
      MEM#3A Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-48256ac0;
      + Type:2A; Size:2 GB;
      MEM#3B Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-48256aab;
      + Type:2A; Size:2 GB;
      MEMB#4 Status:Normal; Ver:0101h; Serial:BF091863L9 ;
      + FRU-Part-Number:CF00541-0545 06 /541-0545-06 ;
      MEM#0A Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-482569cd;
      + Type:2A; Size:2 GB;
      MEM#0B Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-482569dd;
      + Type:2A; Size:2 GB;
      MEM#1A Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-48256989;
      + Type:2A; Size:2 GB;
      MEM#1B Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-4825698d;
      + Type:2A; Size:2 GB;
      MEM#2A Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-477f7355;
      + Type:2A; Size:2 GB;
      MEM#2B Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-48256980;
      + Type:2A; Size:2 GB;
      MEM#3A Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-4825698b;
      + Type:2A; Size:2 GB;
      MEM#3B Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-48256981;
      + Type:2A; Size:2 GB;
      MEMB#5 Status:Normal; Ver:0101h; Serial:BF09175U6N ;
      + FRU-Part-Number:CF00541-0545 06 /541-0545-06 ;
      MEM#0A Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-477f74e5;
      + Type:2A; Size:2 GB;
      MEM#0B Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-477f7398;
      + Type:2A; Size:2 GB;
      MEM#1A Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-477f7392;
      + Type:2A; Size:2 GB;
      MEM#1B Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-477f73e0;
      + Type:2A; Size:2 GB;
      MEM#2A Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-477f734c;
      + Type:2A; Size:2 GB;
      MEM#2B Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-477f7397;
      + Type:2A; Size:2 GB;
      MEM#3A Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-477f7395;
      + Type:2A; Size:2 GB;
      MEM#3B Status:Normal;
      + Code:ce0000000000000001M3 93T5660QZA-CE6 4151-477f74e2;
      + Type:2A; Size:2 GB;
      DDC_A#0 Status:Normal;
      DDC_A#1 Status:Normal;
      DDC_A#2 Status:Normal;
      DDC_A#3 Status:Normal;
      DDC_B#0 Status:Normal;
      DDC_B#1 Status:Normal;
      IOU#0 Status:Normal; Ver:0101h; Serial:BF0905133A ;
      + FRU-Part-Number:CF00541-2240 03 /541-2240-03 ;
      + Type:1;
      DDC_A#0 Status:Normal;
      DDCR Status:Normal;
      DDC_B#0 Status:Normal;
      PCI#1 Name_Property:network; Card_Type:Other;
      PCI#3 Name_Property:SUNW,qlc; Card_Type:Other;
      IOU#1 Status:Normal; Ver:0101h; Serial:BF091760TC ;
      + FRU-Part-Number:CF00541-2240 04 /541-2240-04 ;
      + Type:1;
      DDC_A#0 Status:Normal;
      DDCR Status:Normal;
      DDC_B#0 Status:Normal;
      PCI#1 Name_Property:network; Card_Type:Other;
      PCI#3 Name_Property:SUNW,qlc; Card_Type:Other;
      XSCFU Status:Normal,Active; Ver:0101h; Serial:BF09030HTG ;
      + FRU-Part-Number:CF00541-0481 04 /541-0481-04 ;
      OPNL Status:Normal; Ver:0101h; Serial:BF09030CDF ;
      + FRU-Part-Number:CF00541-0850 06 /541-0850-06 ;
      PSU#0 Status:Normal; Serial:1357FYG-0848AD00FR;
      + FRU-Part-Number:CF00300-2011 0202 /300-2011-02-02;
      + Power_Status:On; AC:200 V;
      PSU#1 Status:Normal; Serial:1357FYG-0848AD00FU;
      + FRU-Part-Number:CF00300-2011 0202 /300-2011-02-02;
      + Power_Status:On; AC:200 V;
      PSU#2 Status:Normal; Serial:1357FYG-0848AD00H3;
      + FRU-Part-Number:CF00300-2011 0202 /300-2011-02-02;
      + Power_Status:On; AC:200 V;
      PSU#3 Status:Normal; Serial:1357FYG-0848AD00H5;
      + FRU-Part-Number:CF00300-2011 0202 /300-2011-02-02;
      + Power_Status:On; AC:200 V;
      FANBP_C Status:Normal; Ver:0301h; Serial:BF0849PR0H ;
      + FRU-Part-Number:CF00541-0848 05 /541-0848-05 ;
      FAN_A#0 Status:Normal;
      FAN_A#1 Status:Normal;
      FAN_A#2 Status:Normal;
      FAN_A#3 Status:Normal;
      XSCF>



      XSCF> showboards -a
      XSB DID(LSB) Assignment Pwr Conn Conf Test Fault
      ---- -------- ----------- ---- ---- ---- ------- --------
      00-0 00(00) Assigned y y y Passed Normal
      01-0 00(01) Assigned y y y Passed Normal
      XSCF>



      XSCF> showdomainstatus -a
      DID Domain Status
      00 Running
      01 -
      02 -
      03 -
      XSCF>
        • 1. Re: M5000 splitting existing domain
          bobthesungeek76036
          CPUs have to go in pairs. So an odd number if CPUs is impossible.
          • 2. Re: M5000 splitting existing domain
            rukbat
            Isn't all this described in the system documentation?
            http://docs.oracle.com/cd/E19580-01/index.html

            In particular, there seems to be a reasonably good set of guidelines in the M-Series Administration Guide:
            http://docs.oracle.com/cd/E19580-01/E21618/E21618.pdf (PDF)
            • 3. Re: M5000 splitting existing domain
              rukbat
              Same question.
              Crossposted to another web site after here, but before first response here:
              http://www.unix.com/solaris/215971-m5000-splitting-existing-domain.html

              Guess what? The response received there points to ...
              ... the documentation !
              http://docs.oracle.com/cd/E19580-01/E21618/Domains.html#50503739_pgfId-1013202
              • 4. Re: M5000 splitting existing domain
                936049
                Hi

                The documentation does give me valuable information but it does not answer my question or I have not found any hint.
                We have only 4 Memory boards.

                This is how it is now

                XSB00
                00-0 CPU0-0 Memb 0
                00-1 CPU0-1 Memb 1
                00-2 CPU1-0
                00-3 CPU1-1

                XSB01
                01-0 CPU0-0 Memb 4
                01-1 CPU0-1 Memb 5
                01-2 CPU1-0
                01-3 CPU1-1

                We would like to have something like this

                Domain 0

                XSB00
                00-0 CPU0-0 Memb 0
                00-1 CPU0-1 Memb 1
                00-2 CPU1-0 Memb 2
                00-3 CPU1-1
                01-3 CPU1-1

                Domain 1

                XSB01
                01-0 CPU0-0 Memb 4
                01-1 CPU0-1
                01-2 CPU1-0

                Can I then leave XSB00 in Uni-Mode and put XSB01 in Quad-Mode?
                • 5. Re: M5000 splitting existing domain
                  bobthesungeek76036
                  933046 wrote:
                  Hi

                  The documentation does give me valuable information but it does not answer my question or I have not found any hint.
                  We have only 4 Memory boards.

                  This is how it is now

                  XSB00
                  00-0 CPU0-0 Memb 0
                  00-1 CPU0-1 Memb 1
                  00-2 CPU1-0
                  00-3 CPU1-1

                  XSB01
                  01-0 CPU0-0 Memb 4
                  01-1 CPU0-1 Memb 5
                  01-2 CPU1-0
                  01-3 CPU1-1

                  We would like to have something like this

                  Domain 0

                  XSB00
                  00-0 CPU0-0 Memb 0
                  00-1 CPU0-1 Memb 1
                  00-2 CPU1-0 Memb 2
                  00-3 CPU1-1
                  01-3 CPU1-1

                  Domain 1

                  XSB01
                  01-0 CPU0-0 Memb 4
                  01-1 CPU0-1
                  01-2 CPU1-0

                  Can I then leave XSB00 in Uni-Mode and put XSB01 in Quad-Mode?
                  Yes but you will not yield what you wish. When a system board is in quad-XSB mode, each CPU <b>must</b> have a corresponding memory board. So if you turn XSB01 in quad-mode, you will loose two of your CPUs. You will need at a minimum three more memory boards to accomplish your task. I'm not sure about mixing Uni- and Quad- in a domain. Never done that and I don't know if it can be done. If not, you will need four more memory boards.

                  Edited by: bobthesungeek76036 on Feb 13, 2013 9:30 AM
                  • 6. Re: M5000 splitting existing domain
                    936049
                    Thanks a lot for your answer. This has helped me. In this case I can only run the two boards in Uni-Mode and split it by 50/50