Re: [Qemu-devel] virtio-net: network stops responding in Win2k3 server

Hi all, it happened again, but not on the win32 machine I was talking before. In that I changed the driver to e1000 and it's up from more than 18 days, never seen that before!

Today the affected machine is a Win2k3 R2 SP2 x64 server, with all updates (before today's) applied. Virtio-net driver is 52.65.104.6500 19/06/2013, loglevel is set to 6.

Qemu startup parameters are as before (one nic this time):

qemu-system-x86_64 -enable-kvm -machine type=pc,accel=kvm -cpu kvm64 -rtc base=localtime -drive file=/vmstore/vm_disks/cineca-service.img,if=virtio -net nic,model=virtio,macaddr=CE:DA:0C:00:00:55 -net tap,script=br224-ifup,vhost=on,ifname=tap-cineca-service -vnc :55 -m 6144 -boot c -k it -usbdevice tablet -name cineca-service -daemonize

Offload parameters for tap-cineca-serv:

rx-checksumming: off

tx-checksumming: off

scatter-gather: off

tcp-segmentation-offload: off

udp-fragmentation-offload: off

generic-segmentation-offload: off

generic-receive-offload: on

large-receive-offload: off

rx-vlan-offload: off

tx-vlan-offload: off

ntuple-filters: off

receive-hashing: off

The network interface (IP: 10.224.4.55) has just stopped responding. I now run ping -t 10.224.4.1 (to the local gateway) from inside the virtual machine and I get "request timeout".

on the tap interface of the host server:

> tcpdump -n -i tap-cineca-serv

tcpdump: WARNING: tap-cineca-serv: no IPv4 address assigned

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode

listening on tap-cineca-serv, link-type EN10MB (Ethernet), capture size 65535 bytes

10:54:06.986752 ARP, Request who-has 10.224.4.1 tell 10.224.4.55, length 28

10:54:12.312851 ARP, Request who-has 10.224.4.1 tell 10.224.4.55, length 28

10:54:14.736914 ARP, Request who-has 10.224.4.197 tell 10.224.4.55, length 28

10:54:17.781544 ARP, Request who-has 10.224.4.197 tell 10.224.4.55, length 28

10:54:17.812755 ARP, Request who-has 10.224.4.1 tell 10.224.4.55, length 28

10:54:23.312894 ARP, Request who-has 10.224.4.1 tell 10.224.4.55, length 28

10:54:23.797236 ARP, Request who-has 10.224.4.197 tell 10.224.4.55, length 28

10:54:28.812902 ARP, Request who-has 10.224.4.1 tell 10.224.4.55, length 28

10:54:34.312949 ARP, Request who-has 10.224.4.1 tell 10.224.4.55, length 28

10:54:39.812811 ARP, Request who-has 10.224.4.1 tell 10.224.4.55, length 28

10:54:45.312855 ARP, Request who-has 10.224.4.1 tell 10.224.4.55, length 28

10:54:50.813006 ARP, Request who-has 10.224.4.1 tell 10.224.4.55, length 28

Debugview log attached.

You can see, until row 00000214, a lot of "[PrepareSendEntry] Sending packet of 42 with NO offload" entries, while I was pinging the gateway.

Then, from inside Windows' device manager, I disable and re-enable the virtio network interface and network return working as usual.

This machine had network working continously for about 10-12 days.

Hope to see news from you. I will make more checks if you ask me what to do.

Regards,

Mario

2013/11/26 ing. Mario De Chenno <address@hidden>

Since it's a production machine, I had to try a remedy first: I changed one network card (pub, of course) to e1000 and now it's up from 62 hours (maybe a record!)

Here is the output for the other card (virtio): I guess parameters should have been the same for the affected interface too (they are now the same for the tap interface connected to e1000 driver, and reflect the underlying physical interface).

Offload parameters for tap-solariconsi:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off

generic-segmentation-offload: off
generic-receive-offload: on
large-receive-offload: off

I cannot switch back the virtual nic just now. I will try to increase debug on other machines, but most are 64bit Win2k3 OSs, so the virtio-net driver is not the same (it's a 64bit version at least...).

Soon or later I will have some debug data to report to the list. In the meantime if you have some configuration advises, feel free to post them.

Thanks,

Mario

2013/11/24 Yan Vugenfirer <address@hidden>

Hi Mario,

Can you check the offload settings of the tap device that is connected to guest?

Run “ethtool -k tap-solaripub”.

On the guest. Raise the log verbosity by going to device manager -> NetKVM device -> Advanced tab -> Logging.Level and changing it to 4. Use DebugView to record the driver tracing (enable kernel trace): http://technet.microsoft.com/en-us/sysinternals/bb896647.aspx

Best regards,
Yan.

Ing. Mario De Chenno
Centro Elaborazione Dati Amministrativi
Seconda Universita' degli Studi di Napoli
via A. De Gasperi 55, 80133 Napoli
telefono: +39 081 5667024
cell.: +39 347 1800708
fax: +39 081 5667013
e-mail: address@hidden

From:	ing. Mario De Chenno
Subject:	Re: [Qemu-devel] virtio-net: network stops responding in Win2k3 server
Date:	Thu, 12 Dec 2013 11:36:19 +0100