From nobody Mon Mar 06 17:26:26 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PVlqR0B9Fz3w96r; Mon, 6 Mar 2023 17:26:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PVlqQ6nmjz457w; Mon, 6 Mar 2023 17:26:26 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1678123587; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=+lUZvkuwiZN2bqhrpWxnErtbj4ukNa5zJypQn5ZFPAM=; b=cj7a8s4wztKH9oSi5JV8L+AQ2ogPssDEZ02/wVt/Fdk7EJtfolxHQPh+NQrGoBmO/mr9Ak DHQtcRGb8Dad/qDvnI+6vjiNQHYObDY/aKdh4aznldfRG2iVnhndzFbXmnk1bsGaVKi2AM Plf2xyYsa7hUIk/iZFZc2aLvBqb6hca4idyCz43oU801CpPn03hVdg4629Xm+iSa0Qs7Fu FDtkoQcgmVLDygLjp4CmzlWf/yzPI6RVSpDQDZWKMx2zaW7EmvKMkCaHwP42PqxmbeEh3G d0skp1Ep+YSyCxC5/5n3WAdr3cY0gbQEg2RuHLGcLClJn6JlsC5NjKNmxxO5AQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1678123587; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=+lUZvkuwiZN2bqhrpWxnErtbj4ukNa5zJypQn5ZFPAM=; b=F2gYRxI8nUdc5RrpToXTydG+ApML1/3zeb8oHEKJ3cShXosGHufvzk1fnRpFjQ2rEJkYNR Dl5PeiIdw1sODaKMNjjFyPZI3XfMujZIQsydAha96IqlxXJd7rCSOyISRxN/lvNkOBtRPz vru8Rga6mcd9WN9VFwB7hcY7VzAmV9oJ6qEThIesdPoeGF2wiU/qSipzg4YiDUsLA/39GF kjevjCtsLGFhmVcOgnz3Q22ewf3B8HegqRjtb01RH/ya6vJDF+T594BDokdMorbJiyZU2y ziAAxLSEZ4xkC+IRZ5Z+H4/V8G/nZ3tE6Br/jlO61RYJNH6hSncHes9TmxW+YQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1678123587; a=rsa-sha256; cv=none; b=OnEXRHo6TjNMFSo869XdTK3XCEdtOi2HGulZmDOHkJzOrsPa005us4HuXx2XUBMu3KF7x7 qTyxlG7j1Ta3cTwdOna5t7FCVpTmsvncJY8qn/RsqlfOyXT3CUp4ePBy61CSXiqZVI+kRk PdH5FKkzB0lh9qHy5HLG7fif6cgrlJGNDQ2MID+gGsU7oUnrlWkVWBhHuhl4KFDcHRIU83 u3RE0tjPR5z/Z3nK52BYqeqCOi7DRjP+6hJYhZkKAd1pKJMrxv++g6XWaXRM8tHeLsRd25 tsMq6vOfshfbjpQU12aFDviBo3L/LAMOdVoSPFpZr9YvRSbEn6OQZz42sYUOaA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4PVlqQ5jcQz19SP; Mon, 6 Mar 2023 17:26:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 326HQQ34005037; Mon, 6 Mar 2023 17:26:26 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 326HQQRb005036; Mon, 6 Mar 2023 17:26:26 GMT (envelope-from git) Date: Mon, 6 Mar 2023 17:26:26 GMT Message-Id: <202303061726.326HQQRb005036@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Vincenzo Maffione Subject: git: 506336f2cd1f - main - netmap: pkt-gen: init all slots of every tx ring List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: vmaffione X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 506336f2cd1fa0a3ba94bc247d7fad1a71d43ac2 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by vmaffione: URL: https://cgit.FreeBSD.org/src/commit/?id=506336f2cd1fa0a3ba94bc247d7fad1a71d43ac2 commit 506336f2cd1fa0a3ba94bc247d7fad1a71d43ac2 Author: Vincenzo Maffione AuthorDate: 2023-03-06 17:22:09 +0000 Commit: Vincenzo Maffione CommitDate: 2023-03-06 17:22:09 +0000 netmap: pkt-gen: init all slots of every tx ring sender_body() uses OPT_COPY to copy the frame into the destination slot for the first 100,000 packets. Then it removes OPT_COPY to improve performance. The function always starts with the first tx ring. If multiple tx rings are in use, it is possible that the initial 100k packets will only use the first ring. After OPT_COPY is removed, there may come a time when the first ring is full and sender_body() will move to the next ring which was never initialized. As a result it will send all zero packets. (This was discovered when the receiving NIC reported rx errors.) Before any transmissions, step through every tx ring and set NS_BUF_CHANGED on every slot. That will force send_packets() to initialize the slot when first used. Since it only copies when necessary, it performs better than always setting OPT_COPY. With this change, there is no reason for the "drop copy" code. Submitted by: Brian Poole MFC after: 7 days --- tools/tools/netmap/pkt-gen.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/tools/tools/netmap/pkt-gen.c b/tools/tools/netmap/pkt-gen.c index b06fef05579d..296208018fd4 100644 --- a/tools/tools/netmap/pkt-gen.c +++ b/tools/tools/netmap/pkt-gen.c @@ -1604,7 +1604,7 @@ sender_body(void *data) uint64_t n = targ->g->npackets / targ->g->nthreads; uint64_t sent = 0; uint64_t event = 0; - int options = targ->g->options | OPT_COPY; + int options = targ->g->options; struct timespec nexttime = { 0, 0}; // XXX silence compiler int rate_limit = targ->g->tx_rate; struct pkt *pkt = &targ->pkt; @@ -1678,6 +1678,19 @@ sender_body(void *data) targ->frags++; } D("frags %u frag_size %u", targ->frags, targ->frag_size); + + /* mark all slots of all rings as changed so initial copy will be done */ + for (i = targ->nmd->first_tx_ring; i <= targ->nmd->last_tx_ring; i++) { + uint32_t j; + struct netmap_slot *slot; + + txring = NETMAP_TXRING(nifp, i); + for (j = 0; j < txring->num_slots; j++) { + slot = &txring->slot[j]; + slot->flags = NS_BUF_CHANGED; + } + } + while (!targ->cancel && (n == 0 || sent < n)) { int rv; @@ -1714,10 +1727,6 @@ sender_body(void *data) /* * scan our queues and send on those with room */ - if (options & OPT_COPY && sent > 100000 && !(targ->g->options & OPT_COPY) ) { - D("drop copy"); - options &= ~OPT_COPY; - } for (i = targ->nmd->first_tx_ring; i <= targ->nmd->last_tx_ring; i++) { int m; uint64_t limit = rate_limit ? tosend : targ->g->burst;