This site is currently read-only as we are migrating to Oracle Forums for an improved community experience. You will not be able to initiate activity until January 30th, when you will be able to use this site as normal.

    Forum Stats

  • 3,890,299 Users
  • 2,269,776 Discussions
  • 7,916,824 Comments

Discussions

Categories

Oracle T8 -4 Server Networking best practice.

User_5M0EJ
User_5M0EJ Member Posts: 13 Green Ribbon
in SPARC Servers

Experts,

Designing Oracle T8-4 networking , have 4 onboard NIC and 2 dual port NIC 10G card installed .

CDOM & IDOM configured.

Want to distribute the bandwidth for Data ,heartbeat, Management ports.

Once I create aggregation aggr0 for ( 2 Ports for one NIC ) .


1 . Is it possible to create multiple Virtual switch on aggr0 to segregate bandwidth for each workload

2 . In case only one Virtual switch possible Is it possible to limit network bandwidth on virtual NIC's for each workload.

3 . Is there any other best practice for cluster interconnect ,because the same traffic going via 10G interface based on tagged VLAN.

Regards,

Satya

· Share on TwitterShare on Facebook

Answers

  • Nik
    Nik Blocked Member Posts: 2,879 Bronze Crown

    Hi.

    Please clear:

    1. What Solaris version You use at Control an Guest Domains? Different Solaris version give different features.
    2. What type of Cluster You use? ( Oracle Solaris Cluster or Oracle Real Application Cluster).
    3. What general topology ? (You have many T8-4 server and build cluster between this servers or between guest domains on different server or some else).


    General recommendation - use separate Hi-Speed NIC for every type of workloads.

    But only You can know what performance really required.

    Regards,

    Nik

    · Share on TwitterShare on Facebook
  • User_5M0EJ
    User_5M0EJ Member Posts: 13 Green Ribbon

    Hi Nik,


    1. What Solaris version You use at Control an Guest Domains? Different Solaris version give different features.

    Solaris 11

    1. What type of Cluster You use? ( Oracle Solaris Cluster or Oracle Real Application Cluster).

    Oracle Solaris Cluster

    1. What general topology ? (You have many T8-4 server and build cluster between this servers or between guest domains on different server or some else).

    Its between 2( T8-4 ) in Global zone level - Local zones are part of RG


    Regards,

    Satya

    · Share on TwitterShare on Facebook
  • Nik
    Nik Blocked Member Posts: 2,879 Bronze Crown

    Hi.

    1 You can not share physical link between virtual switches. (IMHO). You can create many virtual switches that used different physical ports.

    2 You can limit bandwidth for link. https://docs.oracle.com/cd/E80106_01/html/E80109/controlnetworkbandwidth.html

    3 Solaris Cluster interconnect require at least 2 separate links. This links can share physical data-links, but use separate VLANs.


    Is not clear for me: How many Solaris Clusters You plan use ?

    T8-4 -> Split to LDOM (Cluster Node) -> Split to local Zones (Many Clusters)

      T8-4 (Cluster Node) -> Split to local Zones (Single Cluster)

    https://docs.oracle.com/cd/E69294_01/html/E69310/bacfbadd.html#scrolltoc


    Regards,

    Nik

    · Share on TwitterShare on Facebook
  • User_5M0EJ
    User_5M0EJ Member Posts: 13 Green Ribbon

    Thanks, much Nik for your time.


    image.png


    1 ] You can not share the physical links between virtual switches. (IMHO). You can create many virtual switches that used different physical ports.

    As per the above draft, it's not possible to create multiple vswitch - Is my understanding was correct.

    In such a case, if I can able to create one vswitch - I can limit bandwidth as per  https://docs.oracle.com/cd/E80106_01/html/E80109/controlnetworkbandwidth.html



    3]

    Is not clear for me: How many Solaris Clusters You plan use ?

    T8-4 -> Split to LDOM (Cluster Node) -> Split to local Zones (Many Clusters)

      T8-4 (Cluster Node) -> Split to local Zones (Single Cluster)


    The setup is as bleow

    Cluster between Global Zones and Local Zones are part of Resource Group for Cluster.

    In this case, how can I plan my heartbeat link as virtual Interconnect (vnet )with VLAN ID tagged?


    image.png


    · Share on TwitterShare on Facebook
  • Nik
    Nik Blocked Member Posts: 2,879 Bronze Crown

    Hi.

    Not reason clustering Control and IO Domains.

    You should clustering only domains that run some Data-services.


    Reason for create IO-domains- for increasing availability of guest domains.

    So IO-domain should provide independent path for disk and network resources.

    As You plan use only one network aggregation - Are You sure that You need create separate IO Domain ?


    For every cluster You should use separate VLAN for every cluster interconnect link.


    Regards,

    Nik

    · Share on TwitterShare on Facebook
  • User_5M0EJ
    User_5M0EJ Member Posts: 13 Green Ribbon

    Thank you Nik

    How I can create redundancy between Control Domain and IO domain in the above setup while configuring the IDOM . I mean distribution of CPU Memory and PCI , FC

    T8-4 ( 1 CDOM = 1 IDOM)


    Regards

    Satya

    · Share on TwitterShare on Facebook
  • Nik
    Nik Blocked Member Posts: 2,879 Bronze Crown
    edited Nov 8, 2021 3:43PM

    Hi.

    Recommended best practice: https://www.oracle.com/technetwork/server-storage/vm/ovmsparc-best-practices-2334546.pdf



    https://www.oracle.com/a/ocom/docs/sparc-t8-m8-server-architecture.pdf page 22, 23

    Control and IO Domain should use different Root-Complex ( Different IO controllers - prefer).


    Share current configuration of T8-4 server.

    (Type and number of network cards, type and number of FC-cards , RAM, number of CPU, number of internal disks).


    Regards,

    Nik

    · Share on TwitterShare on Facebook