bce(4) - com_no_buffers (Again)

[Tom Judge]

On 09/13/2010 02:11 PM, Andre Oppermann wrote:
> On 13.09.2010 20:48, Pyun YongHyeon wrote:
>> On Mon, Sep 13, 2010 at 10:04:25AM -0500, Tom Judge wrote:
>>> Without BCE_JUMBO_HDRSPLIT then we see no errors.  With it we see
>>> number
>>> of errors, however the rate seems to be reduced compaired to the
>>> previous version of the driver.
Please note that 'rate' here relates to the rate at which
dev.bce.X.com_no_buffers is increasing not to PPS or bandwidth.

However the discussion is still interesting.
>>>
>>
>> It seems there are issues in header splitting and it was disabled
>> by default. Header splitting reduces packet processing overhead in
>> upper layer so it's normal to see better performance with header
>> splitting.
>
> I'm not sure that header splitting really helps much at least for TCP.
> The only place where it could make a difference is at socket buffer
> append time.  There the header get 'thrown away'.  With header splitting
> the first mbuf in the chain containing the header can be returned to the
> free pool.  Without header splitting it's just a offset change in the
> mbuf.
>
> IIRC header splitting was introduced with the Tigeon cards which were
> the first programmable network cards and the first to support putting
> the header in a different mbuf.  Header splitting, in theory, could
> make a difference with zero copy sockets where the data portion in a
> separate mbuf is flipped by VM magic into userspace.  The trouble is
> that no driver fully supports the semantics required for page flipping
> and the zero copy code, if compiled in, is less much less optimized for
> the non-flipping case than the standard code path.  With the many dozen
> gigabyte per second memory copy bandwidth of current CPU's it remains
> questionable whether the page-flipping VM magic is actually faster than
> a plain kernel/userspace copy as in the standard code path.  I generally
> recommend not to use ZERO_COPY_SOCKETS.
>
> I suspect in the case of the bce(4) driver the change in header splitting
> is probably not the cause of the performance difference.
>


-- 
TJU13-ARIN