1FreeBSD Driver for Intel(R) Ethernet 10 Gigabit PCI Express Server Adapters 2============================================================================ 3/*$FreeBSD: stable/10/sys/dev/ixgbe/README 251964 2013-06-18 21:28:19Z jfv $*/ 4 5Jun 18, 2013 6 7 8Contents 9======== 10 11- Overview 12- Supported Adapters 13- Building and Installation 14- Additional Configurations and Tuning 15- Known Limitations 16 17 18Overview 19======== 20 21This file describes the FreeBSD* driver for the 22Intel(R) Ethernet 10 Gigabit Family of Adapters. 23 24For questions related to hardware requirements, refer to the documentation 25supplied with your Intel 10GbE adapter. All hardware requirements listed 26apply to use with FreeBSD. 27 28 29Supported Adapters 30================== 31 32The driver in this release is compatible with 82598 and 82599-based Intel 33Network Connections. 34 35SFP+ Devices with Pluggable Optics 36---------------------------------- 37 3882599-BASED ADAPTERS 39 40NOTE: If your 82599-based Intel(R) Ethernet Network Adapter came with Intel 41optics, or is an Intel(R) Ethernet Server Adapter X520-2, then it only supports 42Intel optics and/or the direct attach cables listed below. 43 44When 82599-based SFP+ devices are connected back to back, they should be set to 45the same Speed setting. Results may vary if you mix speed settings. 46 47Supplier Type Part Numbers 48 49SR Modules 50Intel DUAL RATE 1G/10G SFP+ SR (bailed) FTLX8571D3BCV-IT 51Intel DUAL RATE 1G/10G SFP+ SR (bailed) AFBR-703SDZ-IN2 52Intel DUAL RATE 1G/10G SFP+ SR (bailed) AFBR-703SDDZ-IN1 53LR Modules 54Intel DUAL RATE 1G/10G SFP+ LR (bailed) FTLX1471D3BCV-IT 55Intel DUAL RATE 1G/10G SFP+ LR (bailed) AFCT-701SDZ-IN2 56Intel DUAL RATE 1G/10G SFP+ LR (bailed) AFCT-701SDDZ-IN1 57 58The following is a list of 3rd party SFP+ modules and direct attach cables that 59have received some testing. Not all modules are applicable to all devices. 60 61Supplier Type Part Numbers 62 63Finisar SFP+ SR bailed, 10g single rate FTLX8571D3BCL 64Avago SFP+ SR bailed, 10g single rate AFBR-700SDZ 65Finisar SFP+ LR bailed, 10g single rate FTLX8571D3BCV-IT 66 67Finisar DUAL RATE 1G/10G SFP+ SR (No Bail) FTLX8571D3QCV-IT 68Avago DUAL RATE 1G/10G SFP+ SR (No Bail) AFBR-703SDZ-IN1 69Finisar DUAL RATE 1G/10G SFP+ LR (No Bail) FTLX1471D3QCV-IT 70Avago DUAL RATE 1G/10G SFP+ LR (No Bail) AFCT-701SDZ-IN1 71Finistar 1000BASE-T SFP FCLF8522P2BTL 72Avago 1000BASE-T SFP ABCU-5710RZ 73 74NOTE: As of driver version 2.5.13 it is possible to allow the operation 75of unsupported modules by setting the static variable 'allow_unsupported_sfp' 76to TRUE and rebuilding the driver. If problems occur please assure that they 77can be reproduced with fully supported optics first. 78 7982599-based adapters support all passive and active limiting direct attach 80cables that comply with SFF-8431 v4.1 and SFF-8472 v10.4 specifications. 81 82Laser turns off for SFP+ when ifconfig down 83-------------------------------------------------------- 84"ifconfig down" turns off the laser for 82599-based SFP+ fiber adapters. 85"ifconfig up" turns on the later. 86 8782598-BASED ADAPTERS 88 89NOTES for 82598-Based Adapters: 90- Intel(R) Ethernet Network Adapters that support removable optical modules 91 only support their original module type (i.e., the Intel(R) 10 Gigabit SR 92 Dual Port Express Module only supports SR optical modules). If you plug 93 in a different type of module, the driver will not load. 94- Hot Swapping/hot plugging optical modules is not supported. 95- Only single speed, 10 gigabit modules are supported. 96- LAN on Motherboard (LOMs) may support DA, SR, or LR modules. Other module 97 types are not supported. Please see your system documentation for details. 98 99The following is a list of 3rd party SFP+ modules and direct attach cables that have 100received some testing. Not all modules are applicable to all devices. 101 102Supplier Type Part Numbers 103 104Finisar SFP+ SR bailed, 10g single rate FTLX8571D3BCL 105Avago SFP+ SR bailed, 10g single rate AFBR-700SDZ 106Finisar SFP+ LR bailed, 10g single rate FTLX1471D3BCL 107 10882598-based adapters support all passive direct attach cables that comply 109with SFF-8431 v4.1 and SFF-8472 v10.4 specifications. Active direct attach 110cables are not supported. 111 112Third party optic modules and cables referred to above are listed only for the 113purpose of highlighting third party specifications and potential compatibility, 114and are not recommendations or endorsements or sponsorship of any third party's 115product by Intel. Intel is not endorsing or promoting products made by any 116third party and the third party reference is provided only to share information 117regarding certain optic modules and cables with the above specifications. There 118may be other manufacturers or suppliers, producing or supplying optic modules 119and cables with similar or matching descriptions. Customers must use their own 120discretion and diligence to purchase optic modules and cables from any third 121party of their choice. Customer are solely responsible for assessing the 122suitability of the product and/or devices and for the selection of the vendor 123for purchasing any product. INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL 124DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF 125SUCH THIRD PARTY PRODUCTS OR SELECTION OF VENDOR BY CUSTOMERS. 126 127Configuration and Tuning 128======================== 129 130The driver supports Transmit/Receive Checksum Offload and Jumbo Frames on 131all 10 Gigabit adapters. 132 133 Jumbo Frames 134 ------------ 135 To enable Jumbo Frames, use the ifconfig utility to increase the MTU 136 beyond 1500 bytes. 137 138 NOTES: 139 140 - The Jumbo Frames setting on the switch must be set to at least 141 22 bytes larger than that of the adapter. 142 143 - There are known performance issues with this driver when running 144 UDP traffic with Jumbo Frames. 145 146 The Jumbo Frames MTU range for Intel Adapters is 1500 to 16114. The default 147 MTU range is 1500. To modify the setting, enter the following: 148 149 ifconfig ix<interface_num> <hostname or IP address> mtu 9000 150 151 To confirm an interface's MTU value, use the ifconfig command. To confirm 152 the MTU used between two specific devices, use: 153 154 route get <destination_IP_address> 155 156 VLANs 157 ----- 158 To create a new VLAN pseudo-interface: 159 160 ifconfig <vlan_name> create 161 162 To associate the VLAN pseudo-interface with a physical interface and 163 assign a VLAN ID, IP address, and netmask: 164 165 ifconfig <vlan_name> <ip_address> netmask <subnet_mask> vlan 166 <vlan_id> vlandev <physical_interface> 167 168 Example: 169 170 ifconfig vlan10 10.0.0.1 netmask 255.255.255.0 vlan 10 vlandev ixgbe0 171 172 In this example, all packets will be marked on egress with 802.1Q VLAN 173 tags, specifying a VLAN ID of 10. 174 175 To remove a VLAN pseudo-interface: 176 177 ifconfig <vlan_name> destroy 178 179 180 Checksum Offload 181 ---------------- 182 183 Checksum offloading supports both TCP and UDP packets and is 184 supported for both transmit and receive. 185 186 Checksum offloading can be enabled or disabled using ifconfig. 187 Both transmit and receive offloading will be either enabled or 188 disabled together. You cannot enable/disable one without the other. 189 190 To enable checksum offloading: 191 192 ifconfig <interface_num> rxcsum 193 194 To disable checksum offloading: 195 196 ifconfig <interface_num> -rxcsum 197 198 To confirm the current setting: 199 200 ifconfig <interface_num> 201 202 203 TSO 204 --- 205 206 TSO is enabled by default. 207 208 To disable: 209 210 ifconfig <interface_num> -tso 211 212 To re-enable: 213 214 ifconfig <interface_num> tso 215 216 LRO 217 --- 218 219 Large Receive Offload is available in the driver; it is on by default. 220 It can be disabled by using: 221 ifconfig <interface_num> -lro 222 To enable: 223 ifconfig <interface_num> lro 224 225 226 Important system configuration changes: 227 --------------------------------------- 228 229 When there is a choice run on a 64bit OS rather than 32, it makes a 230 significant difference in improvement. 231 232 The interface can generate a high number of interrupts. To avoid running 233 into the limit set by the kernel, adjust hw.intr_storm_threshold 234 setting using sysctl: 235 236 sysctl hw.intr_storm_threshold=9000 (the default is 1000) 237 238 For this change to take effect on boot, edit /etc/sysctl.conf and add the 239 line: 240 hw.intr_storm_threshold=9000 241 242 If you still see Interrupt Storm detected messages, increase the limit to a 243 higher number, or the detection can be disabled by setting it to 0. 244 245 The default number of descriptors is 2048, increasing or descreasing 246 may improve performance in some workloads, but change carefully. 247 248 249Known Limitations 250================= 251 252For known hardware and troubleshooting issues, refer to the following website. 253 254 http://support.intel.com/support/go/network/adapter/home.htm 255 256Either select the link for your adapter or perform a search for the adapter 257number. The adapter's page lists many issues. For a complete list of hardware 258issues download your adapter's user guide and read the Release Notes. 259 260 UDP stress test with 10GbE driver 261 --------------------------------- 262 Under small packets UDP stress test with 10GbE driver, the FreeBSD system 263 will drop UDP packets due to the fullness of socket buffers. You may want 264 to change the driver's Flow Control variables to the minimum value for 265 controlling packet reception. 266 267 Attempting to configure larger MTUs with a large numbers of processors may 268 generate the error message "ix0:could not setup receive structures" 269 -------------------------------------------------------------------------- 270 When using the ixgbe driver with RSS autoconfigured based on the number of 271 cores (the default setting) and that number is larger than 4, increase the 272 memory resources allocated for the mbuf pool as follows: 273 274 Add to the sysctl.conf file for the system: 275 276 kern.ipc.nmbclusters=262144 277 kern.ipc.nmbjumbop=262144 278 279 Lower than expected performance on dual port 10GbE devices 280 ---------------------------------------------------------- 281 Some PCI-E x8 slots are actually configured as x4 slots. These slots have 282 insufficient bandwidth for full 10Gbe line rate with dual port 10GbE devices. 283 The driver will detect this situation and will write the following message in 284 the system log: "PCI-Express bandwidth available for this card is not 285 sufficient for optimal performance. For optimal performance a x8 PCI-Express 286 slot is required." 287 288 If this error occurs, moving your adapter to a true x8 slot will resolve the 289 issue. 290 291 292 293Support 294======= 295 296For general information and support, go to the Intel support website at: 297 298 www.intel.com/support/ 299 300If an issue is identified with the released source code on the supported 301kernel with a supported adapter, email the specific information related to 302the issue to freebsd@intel.com 303 304 305 306License 307======= 308 309This software program is released under the terms of a license agreement 310between you ('Licensee') and Intel. Do not use or load this software or any 311associated materials (collectively, the 'Software') until you have carefully 312read the full terms and conditions of the LICENSE located in this software 313package. By loading or using the Software, you agree to the terms of this 314Agreement. If you do not agree with the terms of this Agreement, do not 315install or use the Software. 316 317* Other names and brands may be claimed as the property of others. 318 319 320