[geeks] Help: Network issue has me stumped

[Cory Smelosky]

On Sun, 24 May 2015, Joshua Snyder wrote:

> Cory,
>
> Are you using the TSO features of your nic on the host?  While I haven't 
> heard or seen this exact problem, I would suspect that it's something to do 
> Tcp Off-loading as the VM is likely sending 65k sized packets to the host 
> machine and relying on the TSO features to break those large packets into 
> frames for the wire. Given that you are seeing problems at 65k I would really 
> look hard at TSO.
>

Looks like I am!

tcp-segmentation-offload: on
         tx-tcp-segmentation: on
         tx-tcp-ecn-segmentation: off [fixed]
         tx-tcp6-segmentation: on

Disabled on both eth1 and vmbr0 to no change. :(

root at obsidian:~# ethtool -k vmbr0
Features for vmbr0:
rx-checksumming: off [fixed]
tx-checksumming: on
         tx-checksum-ipv4: off [fixed]
         tx-checksum-ip-generic: on
         tx-checksum-ipv6: off [fixed]
         tx-checksum-fcoe-crc: off [fixed]
         tx-checksum-sctp: off [fixed]
scatter-gather: on
         tx-scatter-gather: on
         tx-scatter-gather-fraglist: on
tcp-segmentation-offload: off
         tx-tcp-segmentation: off
         tx-tcp-ecn-segmentation: off
         tx-tcp6-segmentation: off
udp-fragmentation-offload: off [requested on]
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off [fixed]
rx-vlan-offload: off [fixed]
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: on
rx-vlan-filter: off [fixed]
vlan-challenged: off [fixed]
tx-lockless: on [fixed]
netns-local: on [fixed]
tx-gso-robust: off [requested on]
tx-fcoe-segmentation: off [requested on]
tx-gre-segmentation: on
tx-ipip-segmentation: on
tx-sit-segmentation: on
tx-udp_tnl-segmentation: on
tx-mpls-segmentation: on
fcoe-mtu: off [fixed]
tx-nocache-copy: on
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
root at obsidian:~#
root at obsidian:~# clear
root at obsidian:~#
root at obsidian:~# ethtool -k vmbr0^C
root at obsidian:~# clear^C
root at obsidian:~# ethtool -k vmbr0
Features for vmbr0:
rx-checksumming: off [fixed]
tx-checksumming: on
         tx-checksum-ipv4: off [fixed]
         tx-checksum-ip-generic: on
         tx-checksum-ipv6: off [fixed]
         tx-checksum-fcoe-crc: off [fixed]
         tx-checksum-sctp: off [fixed]
scatter-gather: on
         tx-scatter-gather: on
         tx-scatter-gather-fraglist: on
tcp-segmentation-offload: off
         tx-tcp-segmentation: off
         tx-tcp-ecn-segmentation: off
         tx-tcp6-segmentation: off
udp-fragmentation-offload: off [requested on]
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off [fixed]
rx-vlan-offload: off [fixed]
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: on
rx-vlan-filter: off [fixed]
vlan-challenged: off [fixed]
tx-lockless: on [fixed]
netns-local: on [fixed]
tx-gso-robust: off [requested on]
tx-fcoe-segmentation: off [requested on]
tx-gre-segmentation: on
tx-ipip-segmentation: on
tx-sit-segmentation: on
tx-udp_tnl-segmentation: on
tx-mpls-segmentation: on
fcoe-mtu: off [fixed]
tx-nocache-copy: on
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
root at obsidian:~# ethtool -k eth1
Features for eth1:
rx-checksumming: on
tx-checksumming: on
         tx-checksum-ipv4: off [fixed]
         tx-checksum-ip-generic: on
         tx-checksum-ipv6: off [fixed]
         tx-checksum-fcoe-crc: off [fixed]
         tx-checksum-sctp: off [fixed]
scatter-gather: on
         tx-scatter-gather: on
         tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
         tx-tcp-segmentation: off
         tx-tcp-ecn-segmentation: off [fixed]
         tx-tcp6-segmentation: off
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-mpls-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: on
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]


> You should be able to tell by running an ifconfig and looking for the TSO4 
> option.  Or at least that is what Google is telling me. My Freebsd server 
> doesn't have TSO nic's so I can't verify it. ^^
>
>            Josh
>
>

-- 
Cory Smelosky
http://gewt.net Personal stuff
http://gimme-sympathy.org Projects