svn commit: r323884 - stable/11/sys/dev/cxgbe/tom

[John Baldwin]

Author: jhb
Date: Thu Sep 21 23:10:59 2017
New Revision: 323884
URL: https://svnweb.freebsd.org/changeset/base/323884

Log:
  MFC 323630: Avoid reusing the wrong buffer for a DDP AIO request.
  
  To optimize the case of ping-ponging between two buffers, the DDP code
  caches the last two buffers used keeping the pages wired and page pods
  stored in the NIC's RAM.  If a new aio_read() request uses one of the
  same buffers, then the work of holding pages, etc. can be avoided.
  However, the starting virtual address of an aio buffer was not saved,
  only the page count, length, and initial page offset.  Thus, an
  aio_read() request could match a different buffer in the address
  space.  (Earlier during development vm_fault_hold_quick_pages() was
  always called and the vm_page_t values were compared, but that was
  eventually removed without being adequately replaced.)  Fix by storing
  the starting virtual address and comparing that (along with other
  fields) to determine if a buffer can be reused.
  
  Sponsored by:	Chelsio Communications

Modified:
  stable/11/sys/dev/cxgbe/tom/t4_ddp.c
  stable/11/sys/dev/cxgbe/tom/t4_tom.h
Directory Properties:
  stable/11/   (props changed)

Modified: stable/11/sys/dev/cxgbe/tom/t4_ddp.c
==============================================================================
--- stable/11/sys/dev/cxgbe/tom/t4_ddp.c	Thu Sep 21 23:10:56 2017	(r323883)
+++ stable/11/sys/dev/cxgbe/tom/t4_ddp.c	Thu Sep 21 23:10:59 2017	(r323884)
@@ -1277,7 +1277,8 @@ pscmp(struct pageset *ps, struct vmspace *vm, vm_offse
     int pgoff, int len)
 {
 
-	if (ps->npages != npages || ps->offset != pgoff || ps->len != len)
+	if (ps->start != start || ps->npages != npages ||
+	    ps->offset != pgoff || ps->len != len)
 		return (1);
 
 	return (ps->vm != vm || ps->vm_timestamp != vm->vm_map.timestamp);
@@ -1378,6 +1379,7 @@ hold_aio(struct toepcb *toep, struct kaiocb *job, stru
 	ps->len = job->uaiocb.aio_nbytes;
 	atomic_add_int(&vm->vm_refcnt, 1);
 	ps->vm = vm;
+	ps->start = start;
 
 	CTR5(KTR_CXGBE, "%s: tid %d, new pageset %p for job %p, npages %d",
 	    __func__, toep->tid, ps, job, ps->npages);

Modified: stable/11/sys/dev/cxgbe/tom/t4_tom.h
==============================================================================
--- stable/11/sys/dev/cxgbe/tom/t4_tom.h	Thu Sep 21 23:10:56 2017	(r323883)
+++ stable/11/sys/dev/cxgbe/tom/t4_tom.h	Thu Sep 21 23:10:59 2017	(r323884)
@@ -112,6 +112,7 @@ struct pageset {
 	int len;
 	struct ppod_reservation prsv;
 	struct vmspace *vm;
+	vm_offset_t start;
 	u_int vm_timestamp;
 };