cvs commit: src/sys/netgraph ng_base.c

[Gleb Smirnoff]

glebius     2005-07-21 12:08:37 UTC

  FreeBSD src repository

  Modified files:
    sys/netgraph         ng_base.c 
  Log:
  Problem description:
  
    At the end of ng_snd_item(), node queue is processed. In certain
  netgraph setups deep recursive calls can occur.
    For example this happens, when two nodes are connected and can send
  items to each other in both directions. If, for some reason, both nodes
  have a lot of items in their queues, then the processing thread will
  recurse between these two nodes, delivering items left and right, going
  deeper in the stack. Other setups can suffer from deep recursion, too.
  The following factors can influence risk of deep netgraph call:
   - periodical write-access events on node
   - combination of slow link and fast one in one graph
   - net.inet.ip.fastforwarding
  
  Changes made:
  
   - In ng_acquire_{read,write}() do not dequeue another item. Instead,
     call ng_setisr() for this node.
   - At the end of ng_snd_item(), do not process queue. Call ng_setisr(),
     if there are any dequeueable items on node queue.
   - In ng_setisr() narrow worklist mutex holding.
   - In ng_setisr() assert queue mutex.
  
  Theoretically, the first two changes should negatively affect performance.
  To check this, some profiling was made:
  
  1) In general real tasks, no noticable performance difference was found.
  
  2) The following test was made: two multithreaded nodes and one
  single-threaded were connected into a ring. A large queues of packets
  were sent around this ring. Time to pass the ring N times was measured.
  This is a very vacuous test: no items/mbufs are allocated, no upcalls or
  downcalls outside of netgraph. It doesn't represent a real load, it is
  a stress test for ng_acquire_{read,write}() and item queueing functions.
  Surprisingly, the performance impact was positive! New code is 13% faster
  on UP and 17% faster on SMP, in this particular test.
  
  The problem was originally found, described, analyzed and original patch
  was written by Roselyn Lee from Vernier Networks. Thanks!
  
  Submitted by:           Roselyn Lee <rosel verniernetworks com>
  
  Revision  Changes    Path
  1.103     +17 -67    src/sys/netgraph/ng_base.c