TCP packets remain unsent

[Willy Offermans]

Dear FreeBSD friends,

I have a problem with my relatively new FreeBSD server. I came across the
problem when sending e-mails of larger size and copying files with scp.
The e-mails were not sent out because of time-out error and the copying was
extremely slow, though successful after a while. I already started a thread
on this topic on freebsd-current. See
http://docs.freebsd.org/mail/current/freebsd-current.html, topic
sendmail Broken Pipe Error. I got some help to narrow down the
error: Sending out e-mails of larger size stops at some point. TCP packets
were not transferred to the smarthost causing a timeout error. There were
still some TCP packets waiting to be sent.

My system is a HP ProLiant Gen8 MicroServer with FreeBSD 10.0-STABLE #0
r261266M. The server has two network cards:

bge0 at pci0:3:0:0:        class=0x020000 card=0x2133103c chip=0x165f14e4 rev=0x00 hdr=0x00
    vendor     = 'Broadcom Corporation'
    device     = 'NetXtreme BCM5720 Gigabit Ethernet PCIe'
    class      = network
    subclass   = ethernet
bge1 at pci0:3:0:1:        class=0x020000 card=0x2133103c chip=0x165f14e4 rev=0x00 hdr=0x00
    vendor     = 'Broadcom Corporation'
    device     = 'NetXtreme BCM5720 Gigabit Ethernet PCIe'
    class      = network
    subclass   = ethernet

I do not know if there any known issues with these cards (drivers).

Before the time out error occurs, the CPU loading of natd and dhcpd is
steadily increasing to extreme values to my opinion:

PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND

 1235 root          1  93    0 28908K  2144K RUN     0  54:05  71.78% natd
 1614 dhcpd         1   4    0 26784K 14936K RUN     0  29:24  38.77% dhcpd

I followed an advice of another FreeBSD friend to modify tcp_input.c

<snip>
...
You may want to locally apply SVN r258821 to sys/netinet/tcp_input.c,
in case it has not been merged back to the FreeBSD version you use:

--- sys/netinet/tcp_input.c     (revision 258820)
+++ sys/netinet/tcp_input.c     (revision 258821)
@@ -2429,13 +2429,15 @@
                hhook_run_tcp_est_in(tp, th, &to);

                if (SEQ_LEQ(th->th_ack, tp->snd_una)) {
-                       if (tlen == 0 && tiwin == tp->snd_wnd) {
+                       if (tlen == 0 && tiwin == tp->snd_wnd &&
+                           !(thflags & TH_FIN)) {
                                TCPSTAT_INC(tcps_rcvdupack);
                                /*
                                 * If we have outstanding data (other than
                                 * a window probe), this is a completely
                                 * duplicate ack (ie, window info didn't
-                                * change), the ack is the biggest we've
+                                * change and FIN isn't set),
+                                * the ack is the biggest we've
                                 * seen and we've seen exactly our rexmt
                                 * threshhold of them, assume a packet
                                 * has been dropped and retransmit it.

...
</snip>

However this did not solve the issue.

So my question to you is if someone else encountered a similar problem with
NetXtreme BCM5720 Gigabit Ethernet PCIe and if someone is able to take this
up and to help me to solve this issue. I would love to run this FreeBSD
server as a swiss clock as other beastie servers, I have setup in the past.

-- 
Met vriendelijke groeten,
With kind regards,
Mit freundlichen Gruessen,
De jrus wah,

Wiel

*************************************
W.K. Offermans
e-mail: Willy at Offermans.Rompen.nl

                                       Powered by ....

                                            (__)
                                         \\\'',)
                                           \/  \ ^
                                           .\._/_)

                                       www.FreeBSD.org