now I realise that the NEM doesn't have an OS as such like the B1600 switch, how do I actually configure everything so my blades can use it?
I have now a lot of questions, as it would seem that I was expecting to create VLANs etc on the NEM and then map these to
the blades so they would have access to various networks.
So, please accept I don't know anything about the hxge, the NEM or what I am supposed to tell the network admin who is configuring
the 10Gb port for me at the other end and I will ask some apparently newbie questions.
1) When I plug the fibre into the 10Gb switch port, the network admin has asked me if I want to trunk the connection, is it
just "for a host" and some VLAN information. I am not sure what to tell them! I figured the 10gb port would allow lots of different
VLAN's and so on...?
2) When configuring VLAN's etc, does the NEM/hxge (I don't know which does what at this point), can I configure things
so a blade (LDOM) has presence on different VLAN's - how do I do this? We figured the idea was the NEM just made patching easier
back to the switch and there would flexibility in presenting different VLAN's or networks to each LDOM or blade - advice please!
3) Can someone who is on a similar setup please give me some pointers, so I am on the right track?
I am sorry for the vague questions, but I have just gone round in circles with this setup at the moment and need some advice....
I assume you're talking about the "Virtualized Network Expansion Module" (HXGE). The system is not a Layer 2 switch, as you're describing. Rather, it merely shares PHY connections to individual MACs on each blade (provided you have the right Fabric Expansion Module on the blade). So, any packets you receive on one of the VNEM's uplink ports is seen by every blade, according to the "failover" settings of the VNEM (e.g. if both uplinks are active, blades 0 - 4 see VNEM uplink 0 and blades 5-9 see uplink 1). There's no way to "filter" packets based on VLAN via the VNEM.
To answer your questions:
1) If you want every blade to see traffic from a particular VLAN and only that VLAN, then you do not want a trunk port. You want an "access" port, and assign the default VLAN accordingly. If you intend to do any VLAN based networking on any of the blades, then yes, you want a 802.1q trunk port. At that point, it depends on the OS layer on each blade to deal with VLAN tagging appropriately. Solaris 10 and VMware seem to handle this just fine, but I've had issues with WIndows Server 2008 R2 (but possibly only because I'm new to Hyper-V networking).
2) I assume you're talking about configuring VLAN tagging in Solaris 10. That's easy. You just multiply the VLAN ID number by 1000, then add the device ordinal. For example, with two VNEMs in our chassis, each blade will see /dev/hxge0 and /dev/hxge1. To assign an IP to VLAN 11 on hxge1, you plumb hxge11001. For VLAN 5 on hxge0, you would plumb hxge5000, etc. Note that if you do not use VLAN tagging in the OS (i.e. hxge0 or hxge1), all traffic will be sent on the default VLAN assigned by the switch.
3) I just did :)