nfs-server silent data corruption

[Mike Tancsa]

At 05:57 PM 4/21/2008, Arno J. Klaassen wrote:
> > Hi,
> > How long does it take for the problem to show up ?
>
>
>Less than an hour in general (running the same client script
>simultanuously on a 100Mbps linux box and 1Gbps bds6-x86)

I am running my nic at gig speeds only...   I recompiled the kernel 
this morning to include cpufreq as well as made sure the cool&quiet 
was enabled in the BIOS.



>for info, I test with args '38 999' (38M, try 999 times) on linux
>(slightly adapted script BTW) and '138 999' on bsd. The best 'score' I
>got was 'still 871 iterations to go'


So far I have done 150 loops with an 80MB file and no issues and 200 
loopswith a 160MB file.  My nfe nic does not support MSI and has its 
own interrupt

# vmstat -i
interrupt                          total       rate
irq1: atkbd0                           5          0
irq4: sio0                          3049          1
irq16: twe0                       327046        164
irq19: bge0                       385147        194
irq21: atapci1                    976355        492
irq23: nfe0                     11876726       5986
cpu0: timer                      3966420       1999
cpu1: timer                      3964392       1998


I have powerd started up with
powerd_enable="YES"
powerd_flags="-a adaptive -b adaptive -n adaptive"

FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
  cpu0 (BSP): APIC ID:  0
  cpu1 (AP): APIC ID:  1
ioapic0: Changing APIC ID to 2
ioapic0 <Version 1.1> irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0: <Nvidia AWRDACPI> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, dfde0000 (3) failed
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
cpu0: <ACPI CPU> on acpi0
powernow0: <Cool`n'Quiet K8> on cpu0
cpu1: <ACPI CPU> on acpi0
powernow1: <Cool`n'Quiet K8> on cpu1
acpi_button0: <Power Button> on acpi0
.
.
nfe0: <NVIDIA nForce4 CK804 MCP9 Networking Adapter> port 
0xb400-0xb407 mem 0xfebf9000-0xfebf9fff irq 23 at device 10.0 on pci0
miibus0: <MII bus> on nfe0
e1000phy0: <Marvell 88E1111 Gigabit PHY> PHY 1 on miibus0
e1000phy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 
1000baseTX-FDX, auto
nfe0: Ethernet address: 00:e0:81:58:91:6a
nfe0: [FILTER]

With the "sleep" in my test script, powerd does seem to be fiddling 
with frequencies as well during the inactivity.

# sysctl dev. | grep -i fre
dev.cpu.0.freq: 1800
dev.cpu.0.freq_levels: 2200/110000 2000/105600 1800/89100 1000/49000
dev.powernow.0.freq_settings: 2200/110000 2000/105600 1800/89100 1000/49000
dev.powernow.1.freq_settings: 2200/110000 2000/105600 1800/89100 1000/49000
dev.cpufreq.0.%driver: cpufreq
dev.cpufreq.0.%parent: cpu0
dev.cpufreq.1.%driver: cpufreq
dev.cpufreq.1.%parent: cpu1

# sysctl dev. | grep -i fre
dev.cpu.0.freq: 2200
dev.cpu.0.freq_levels: 2200/110000 2000/105600 1800/89100 1000/49000
dev.powernow.0.freq_settings: 2200/110000 2000/105600 1800/89100 1000/49000
dev.powernow.1.freq_settings: 2200/110000 2000/105600 1800/89100 1000/49000
dev.cpufreq.0.%driver: cpufreq
dev.cpufreq.0.%parent: cpu0
dev.cpufreq.1.%driver: cpufreq
dev.cpufreq.1.%parent: cpu1

         ---Mike