FILE NAME: basplnx-2.0.5-1a.src.rpm TITLE: Compaq Tested and Approved bcm5700 Advanced Server Program for Linux VERSION: 2.0.5 LANGUAGE: English CATEGORY: Software Solutions DIVISIONS: Systems PRODUCTS AFFECTED: Compaq NC7770 PCI-X Gigabit Server Adapter Compaq NC7780 Gigabit Server NIC OPERATING SYSTEM: Red Hat Linux Professional 7.1 Red Hat Linux Professional 7.2 SuSE SLES 7 Caldera OpenLinux Server 3.1 PREREQUISITES: Pentium-based computer, 128Mb RAM minimum EFFECTIVE DATE: January 28, 2002 SUPERSEDES: basplnx-2.0.5-1.src.rpm DESCRIPTION: This RPM Package Manager contains the Compaq Tested and Approved basplnx Linux teaming driver for use with the bcm5700 network devices. ENHANCEMENTS/FIXES: Documentation enhancements HOW TO USE: Table of Contents ================= Introduction Overview Packaging Install - RPM package Team Configuration and Sample Scripts Configuration Script Variables and Definitions Red Hat 7.1 & 7.2 Considerations SuSE SLES 7 Considerations Caldera 3.1 Considerations Uninstall - RPM package Limitations Introduction ============ This file describes the Linux teaming driver for the Compaq NC77xx bcm5700 series 10/100/1000 Mbps Ethernet Network Controllers. Additional information can be found in the MAN page for basp, and in /usr/share/doc/basplnx-2.0.5/release.txt after driver installation. For SuSE, see /usr/share/doc/packages/basplnx/release.txt. For Caldera, see /usr/doc/basplnx-2.0.5/release.txt. User guides and additional Compaq server network adapter information can be found at . Overview =========== The bcm5700 Advanced Server Program for Linux (also referred to as BASP) is a kernel module designed to team Compaq NC77XX server adapters into one virtual network interface providing load balancing, fault-tolerance, and VLAN tagging. These features are provided by creating teams that can consist of 1 to 8 adapters. Each adapter can be designated as PRIMARY or HOT-STANDBY. All PRIMARY adapters will participate in dynamic fail over and load balancing operations by sending and receiving a portion of the total traffic. HOT-STANDBY adapters take over only in the event that all PRIMARY adapters have lost their links or have failed. VLANs can be configured on a team and allow for multiple VLANs with different VLAN ID's. A virtual interface (also referred to as a virtual device) is created for each VLAN added. This virtual device is then given a name for configuration purposes. The virtual device is created after a user-defined basp team-script runs and can then be configured with an IP address. The bcm5700 Advanced Server Program for Linux supports Smart Load-Balance (SLB) and Generic Trunking. Both modes support load balance and dynamic fail over features by default. SLB mode works with all Ethernet switches and does not require configuring the switch to any special trunking mode. Only IP traffic will be load-balanced in both inbound and outbound directions. Generic trunking requires an Ethernet switch capable of supporting such technology and must be properly configured. Generic trunking is protocol-independent and requires that all adapters be set to PRIMARY. Compaq has tested and approved the bcm5700 Advanced Server Program for Linux with Red Hat 7.1, 7.2, SuSE SLES 7 and Caldera 3.1. The following installation procedures work with these Linux distributions. Minimal test has been done on other Linux distributions, however, the bcm5700 Advanced Server Program for Linux is designed to work on Linux 2.2.X and 2.4.X kernels. Before installing the teaming module bcm5700 Advanced Server Program for Linux, make sure all Compaq NC77XX Server Adapters work independently with the base driver bcm5700. The current network settings will then be replaced upon successfully starting BASP with the configuration supplied in a user defined team-script. Insure that the Compaq Server has the latest system ROM. Packaging ========= The driver is released in a source RPM format. The file name for the package is basplnx-.src.rpm and is dependant on kernel source code. Install - RPM package (Red Hat, SuSE, Caldera) ======================== 1) Install the RPM source package. > rpm -ivh basplnx-{version}.src.{arch}.rpm 2) Change directory to the RPM path and build the binary driver for the kernel. Note that the RPM path is different for the different Linux distributions. > cd /usr/src/{redhat, packages, OpenLinux} > rpm -bb SPECS/basplnx.spec 3) Install the newly built package > rpm -ivh RPMS/i386/basplnx-{version}.{arch}.rpm NOTE - When installing on Caldera 3.1, warnings will be displayed for chkconfig command not being found and various modules not being an ELF file type. These are not driver specific and can be ignored. 4) To load the driver, > insmod basp 5) Follow the team configuration section below to set up the teams. A team configuration script will need to be run for each desired team. Executing a script will create the virtual interface for the team which can then be configured. 6) After the desired team-script has been copied and configured, go to either the Red Hat, SuSE, or Caldera installation section below to finish the installation. The bcm5700 Advanced Server Program for Linux is installed in the /etc/rc.d/init.d directory. This script will be executed at runlevel 2, 3, 4 and 5. When "basp" is run, it will search the /etc/basp directory to list all the files with "team-" prefix, and then it will invoke the "baspteam" script to add or delete the teams. It is required that each "team-*" file in /etc/basp represents 1 team. Team Configuration and Sample Scripts ================================== The bcm5700 Advanced Server Program for Linux package will create three sample files in the /etc/basp/sample directory after package installation. The sample file names are "team-sample", "team-gec" and "team-vlan". The team configuration process involves the following steps: 1) Copy one configuration script for each desired team from the "/etc/basp/samples" directory to the "/etc/basp" directory. Note the configuration script must be prefixed with "team-". 2) Modify the configuration script for the appropriate network environment. Modify the configuration script to (a) change the team type (b) add/delete physical network interfaces (c) add/delete virtual network interfaces Note that when configuring teaming, at least one Primary Adapter is required. See the notes section below and the Configuration Scripts Variables and Definitions section for more information. 3) Once the team-script/s are configured, skip to either the Red Hat, SuSE or Caldera installation section below to finish installation. Notes: -SLB teams require at least one PRIMARY adapter. -Generic Trunking teams require all adapters to be PRIMARY mode adapters. -Forming multiple teams is possible by copying the sample files into the "/etc/basp" directory and modifying the file name to be unique as well as modifying the file information as described in the sample files. -Each team should have a unique file name. Example: "team-zero" and "team-one" -Each team should have a unique team name, "TEAM_NAME". Example: "TeamZero" and "TeamOne" -Each team should have a unique team identification number, "TEAM_ID". Example: "0" and "1" -Each team should have a unique virtual name, "TEAM_VA0_NAME". Example: "sw0" and "sw1" team-sample This script contains a SLB team configuration with 3 NICs: eth0, eth1 and eth2. The team name is "TeamSample". All 3 NICs are primary. One virtual interface is also created for this team and the name of the virtual interface is "sw0". "sw0" is the device that "ifconfig" should be run against to set up the IP address. VLANs are not enabled in this script. team-gec This configuration script creates a GEC team with 3 network interfaces, eth0, eth1 and eth2. The team name is "TeamGEC". All 3 NICs are primary. One virtual interface is added to the team with the name "sw0" and VLANs are not enabled. team-vlan This configuration script creates a VLAN team with 3 network interfaces, eth0, eth1 and eth2. The team name is "vlan". All 3 NICs are primary. Two virtual interfaces are added to the team with the names "sw0" and "sw1". The second interface is on vlan "1". The following is a correction for the team-vlan sample script for multiple vlan tagging: FROM: # 1st virtual interface in the team TEAM_VA0_NAME=sw0 TEAM_VA0_VLAN=0 # 2nd virtual interface in the team TEAM_VA0_NAME=sw1 TEAM_VA0_VLAN=1 TO: # 1st virtual interface in the team TEAM_VA0_NAME=sw0 TEAM_VA0_VLAN=0 # 2nd virtual interface in the team TEAM_VA1_NAME=sw1 TEAM_VA1_VLAN=1 Sample files: "team-sample", "team-gec" and "team-vlan" are intended to be customized for each team. The desired script should be copied to /etc/basp directory and retain the "team-" prefix. Configuration Script Variables and Definitions ================================================== The sample files: "team-sample", "team-gec" and "team-vlan" are configuration scripts that follow the syntax listed below. TEAM_ID: this number uniquely identifies a team. TEAM_TYPE: 0 = SLB, 1 = Generic Trunking/GEC/FEC TEAM_NAME: ascii name of the team TEAM_PAx_NAME: ascii name of the physical interface x, where x can be 0 to 7. TEAM_PAx_ROLE: role of the physical interface x 0 = Primary, 1 = Standby. This field must be 0 for Generic Trunking/GEC/FEC team. TEAM_VAx_NAME: ascii name of the virtual interface x, where x can be 0 to 63 TEAM_VAx_VLAN: 802.1Q VLAN ID of the virtual interface x. For untagged virtual interface, i.e. without VLAN enable, set it to 0. The valid VLAN ID can be 0 to 4094. After editing a script, go to either the Red Hat, SuSE or Caldera Considerations section below to finish the installation. Red Hat 7.1 and 7.2 Considerations ============================ This section assumes the RPM package has been installed and a team- configuration script has been copied to /etc/basp and has been edited appropriately for your networking environment. 1) Start the team for the first time: >/etc/rc.d/init.d/basp start The following message will be returned. Adding BASP team [OK] Starting BASP team [FAILED] The team will fail to start for the first time because the virtual network interface is not yet configured. 2) Configure and activate the virtual network interface by entering the following command: >ifconfig up Check the team status by typing: >ifconfig or >baspcfg show "ifconfig" should return the IP address for the virtual interface and indicate the device is up. "baspcfg show" will return team configuration information and status as long as the basp module is loaded. If the team is not up, check that the /etc/basp/team-script contains correct syntax and correct information for your networking environment. 3) Finish the installation by running a Red Hat networking configuration utility such as "netconf" or "linuxconf" to configure the IP address of the virtual network interface . It is important to manually enter the virtual interface name i.e. sw0, when configuring the IP address. This is because the virtual network interface name is not shown in the physical interface device list. Saving this configuration will allow the team to start automatically after a system reboot. Test your teaming installation by pinging the virtual IP address from another host. Teams may be started or stopped manually by running: >/etc/rc.d/init.d/basp start or >/etc/rc.d/init.d/basp stop The baspcfg command can only be executed in Super User mode. Using other login modes will produce the following error message: "Permission Denied. Must be root to manage BASP". SuSE SLES 7 Considerations ==================== This section assumes the RPM package has been installed and a team- configuration script has been copied to /etc/basp and has been edited appropriately for your networking environment. BASP services do not start automatically on SuSE at boot. Thus each reboot of the server requires a manual startup each time unless the following steps are followed. Step 1 and step 2 are one time configurations, however, step 3 procedures are required at each server boot unless the startup commands outlined in step 3 are copied to a runlevel startup script. 1) To ensure that the bcm5700 driver is initialized at boot up, add bcm5700 and the team's virtual interface to the /etc/modules.conf file. For simplicity, assume the team's virtual interface is sw0. The following should be added to the module.conf file: alias sw0 bcm5700 If more than one team exists then the "modules.conf" file should look similar to the example below. Also note that changing default parameters (such as jumbo frames) requires adding an "options" value. To change a default parameter a "value" should be given for each corresponding adapter. The first value in the "options" string is for eth0, the second for eth1, the third for eth2. In other words, if there are six adapters then there should be six values. alias sw0 bcm5700 alias sw1 bcm5700 alias sw2 bcm5700 options bcm5700 = 2) Use SuSE's YAST to insert the team's correct configuration. -Type "yast" at the command line. -Select the "System Administration" option. -Select the "Network Configuration" option. -Select the "Network Base Configuration" option. From the "Network Base configuration" screen configure the team's virtual interface, IP address, and startup option. -Type "f6" to get into the IP configuration section. Configure the IP address and netmask for the team. Exit back to the previous screen. -Now type "f5" to configure the "Device Name". For simplicity, supply "sw0" as the team's virtual interface. Select "". Once there, type in the team's virtual interface, "sw0" and press return. Save and exit to the previous screen. -Now type "f4" to "Active" the team at boot. An "X" will appear in the "Activation" box next to the team just configured. If more than one team exists then repeat these steps for each team and ensure that each team has an unique IP address as well as an unique virtual interface name. 3) This step is required for every reboot unless the first two commands are placed in a runlevel startup script. After booting, please type the following two commands: > /etc/rc.d/init.d/basp start The following message will be displayed: Adding BASP team [OK] Starting BASP team [FAILED] The team will fail to start for the first time because the virtual network interface is not active. Enter the following command for each team. > ifconfig up Check the team status by typing: >ifconfig or >baspcfg show "ifconfig" should return the IP address for the virtual interface and indicate the device is up. "baspcfg show" will return team configuration information and status as long as the basp module is loaded. If the team is not up, check that the /etc/basp/team-script contains correct syntax and correct information for your networking environment. Test your teaming installation by pinging the virtual IP address from another host. Teams may be started or stopped manually by running: >/etc/rc.d/init.d/basp start or >/etc/rc.d/init.d/basp stop Caldera 3.1 Considerations ==================== This section assumes the RPM package has been installed and a team- configuration script has been copied to /etc/basp and has been edited appropriately for your networking environment. 1) Start the team for the first time: >/etc/rc.d/init.d/basp start The following message will be returned. Adding BASP team [OK] Starting BASP team [FAILED] The team will fail to start for the first time because the virtual network interface is not yet configured. 2) Configure and activate the virtual network interface by entering the following command: >ifconfig up Check the team status by typing: >ifconfig or >baspcfg show "ifconfig" should return the IP address for the virtual interface and indicate the device is up. "baspcfg show" will return team configuration information and status as long as the basp module is loaded. If the team is not up, check that the /etc/basp/team-script contains correct syntax and correct information for your networking environment. Test your teaming installation by pinging the virtual IP address from another host. Teams may be started or stopped manually by running: >/etc/rc.d/init.d/basp start or >/etc/rc.d/init.d/basp stop For Caldera, The bcm5700 Advanced Server Program for Linux services do not start automatically upon reboot. The following commands are required after every boot or can be captured in a runlevel startup script (such as rc.local) to start the team: /etc/rc.d/init.d/basp start ifconfig sw0 up Uninstall - RPM package =========================== To uninstall the basp rpm package: > rpm -e basplnx Limitations ================== 1) Automatic team configuration and startup is only supported on the Red Hat distribution. However, the following commands can be added to a runlevel startup script to start the virtual interface upon reboot. /etc/rc.d/init.d/basp start ifconfig up This will start the team and configure the first virtual interface with the desired IP address. Add additional "ifconfig" statements as required to bring up additional virtual interface devices or teams. 2) Always assign static IP address to the virtual network interfaces. 3) The baspcfg commands i.e "baspcfg show" can only be executed in Super User mode. Using other login modes will produce the following error message: "Permission Denied. Must be root to manage BASP". 4) To help facilitate troubleshooting team startup failures, manually stop the services /etc/rc.d/init.d/basp stop before trying a new configuration. Copyright 2000-2002, Compaq Computer Corporation. All rights reserved. Product names mentioned herein may be trademarks and/or registered trademarks of their respective companies.