I try to enable FCoE on a server with Oracle Linux 6.3 but I have some troubles...
The server is an UCS C240-M3 with an UCS 1225 VIC (Virtual Interface Card). Per default, my Linux has set 6 network interfaces with DHCP (eth0 -> eth5). It corresponds to a 4 Gigabit PCI card + 2 Virtual NIC from the VIC 1225 adapter. The VIC 1225 is supposed to be used for Networking and FCoE, thanks to VLANs.
Linux displays two FCoE devices with lspci:
[root@localhost /]# lspci | grep -i fcoe 0d:00.0 Fibre Channel: Cisco Systems Inc VIC FCoE HBA (rev a2) 0e:00.0 Fibre Channel: Cisco Systems Inc VIC FCoE HBA (rev a2)
And it appears that drivers are well installed:
[root@localhost ~]$ dmesg | grep fnic fnic: Cisco FCoE HBA Driver, ver 18.104.22.168 fnic 0000:0d:00.0: PCI INT A -> GSI 46 (level, low) -> IRQ 46 fnic 0000:0d:00.0: setting latency timer to 64 fnic 0000:0d:00.0: irq 75 for MSI/MSI-X fnic 0000:0d:00.0: irq 76 for MSI/MSI-X fnic 0000:0d:00.0: irq 77 for MSI/MSI-X fnic 0000:0d:00.0: irq 78 for MSI/MSI-X scsi1 : fnic fnic 0000:0e:00.0: PCI INT A -> GSI 40 (level, low) -> IRQ 40 fnic 0000:0e:00.0: setting latency timer to 64 fnic 0000:0e:00.0: irq 79 for MSI/MSI-X fnic 0000:0e:00.0: irq 80 for MSI/MSI-X fnic 0000:0e:00.0: irq 81 for MSI/MSI-X fnic 0000:0e:00.0: irq 82 for MSI/MSI-X scsi2 : fnic
How can I enable FCoE on my Linux ? I found some help on the Internet, and examples show how to use ethN interface etc.... I don't know how we habitually identify the interface (eth*) to use with FCoE. Is the FCoE interface considered as a "network" interface by Linux, and generates a script in /etc/sysconfig/network-script ? Am I supposed to see additional interfaces (eth6 and eth7) and use it for FCoE ?
Have you checked the documentation?
To enable FCoE on the Ethernet interface:
# dcbtool sc ethX app:fcoe e:1
To view configured FCoE devices:
# fcoeadm -i
You cannot enable DCB on a virtual interface, because it does not have the necessary hardware.
I cannot use eth0 .. eth5 because they are dedicated for network usage.
Where is the problem? FCoE will be on it's own VLAN device.
FCoE is a network protocol running on Ethernet, alongside traditional IP traffic. The FCoE protocol is encapsulated using a VLAN interface, corresponding to the physical adapter that is connected to your FCoE Switch. Unlike a regular NIC adapter, your CNA (converge network adapter) can offload low level frame processing and simultaneously support LAN (TCP/IP) and SAN (FCoE, iSCSI) traffic, for which you use the dcbtool.
Once you determined the adapter port, e.g. eth1, you create the appropriate /etc/sysconfig/network-script/ifcfg-eth1 script. The creation and configuration of the script is similar to any other Ethernet device, such as eth0. Then you copy the /etc/fcoe/ifcfg-ethx script to /etc/fcoe/ifcfg-eth1 to create the VLAN for FCoE, which will show up as a separate device after you restart the appropriate services.
Ok but in my case, even if the physical Cisco card is a CNA device (it is plugged with a link to the FCoE switch, and a link to the Ethernet network), the Operating System doesn't see such a device since we have configured on the UCS :
- Two virtual interfaces for Ethernet traffic
- Two virtual interfaces for FCoE traffic
Here we have the same behavior as a virtual machine, when it is connected to virtual devices. For the operating system, the two virtual devices configured on UCS side for FCoE are seen as physical dedicated FCoE devices.
So I don't need to configure an ethX device for both Ethernet and FCoE, because both FCoE and Ethernet traffic are separated.
My knowledge about FCoE was not very high. Now, I know that a network must be properly configured in order to allow FCoE traffic to work, since FCoE is totally based on the network.
So, we have configured Ethernet network on our virtual interfaces and the storage admin has created LUNs on the storage array. No supplementary configuration was necessary since drivers for the FCoE interface are included in Linux kernel.