From nobody Fri Dec 10 12:33:26 2021 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 50F8C18D31B4; Fri, 10 Dec 2021 12:33:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J9VgV4LXdz3vHL; Fri, 10 Dec 2021 12:33:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 469591A4FD; Fri, 10 Dec 2021 12:33:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1BACXQsF069840; Fri, 10 Dec 2021 12:33:26 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1BACXQ1X069839; Fri, 10 Dec 2021 12:33:26 GMT (envelope-from git) Date: Fri, 10 Dec 2021 12:33:26 GMT Message-Id: <202112101233.1BACXQ1X069839@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Andriy Gapon Subject: git: f43d2e1199b9 - stable/12 - iflib_stop: drain rx tasks to prevent any data races List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: avg X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: f43d2e1199b9fd265b72281d2779e2554fa7cb6d Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1639139606; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=RWGZorfeS/81yPlLbQbb1xhxwPKz+R9GftTzbUTUMIU=; b=jHPydkRh3wau5JC8TbIPw5SJkgDcAlGNZSZQ06qjKeYRhrOc2UqfjM0TqUSaSK1pvKUjH+ 4Ov2wDoATn6b+1QRrs0+7MKd1SitI0R434TDuiJwGZF6sOhvOvXn8bTyxPBPJWR2oSxoHQ Fv+mt8Vwqq6ocvbDw0CRWpmX6yH9q+Gvuy8+49ZcQlSIfLqj/dQHfkAMXRKi0mPEGssPDi 3mhnvxprq0AWpKrDym0NwmhTnUbqPOzsHyCTH8q6/JGsDBl7ASp5gDC3z9pzAXXczj2KLG EiNfXfGlTg0zzQ5GMhrXSxjleP4AezEJj99rfnEchSc11ZACAZkpz4l6JPwGBQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1639139606; a=rsa-sha256; cv=none; b=s7FnFPtCazi+Oi/uZqoncWNfJvyf9wTO7IBeRlvCDP3Az/ntDROS/q6Fin583umJ/M+sTG YRrcdMdS1fcTMJ1RBxojLyLtdloTim7gsiQx4Gkj246+42nyFZHiFJrd4QWPBYiu6WAbkR M8mpVy102ohJBF7l8YhcA6pGtBdYlxfkXnqvelYDXups2PyjROM8bjDWwa1iDCmAtiiOTm /dGHVgZhpYXkO83pP589QziYgneSsm6RkjKNNfUWyIJWC4oviaxGbfVIx7W0Xz1j+wL8rR keWpfGyNcmR8r9wP8OUCG9IhHObXME075Z5Fv6XB7g0X7ViZ6eDuhpo+xmaeFw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/12 has been updated by avg: URL: https://cgit.FreeBSD.org/src/commit/?id=f43d2e1199b9fd265b72281d2779e2554fa7cb6d commit f43d2e1199b9fd265b72281d2779e2554fa7cb6d Author: Andriy Gapon AuthorDate: 2021-11-19 07:56:30 +0000 Commit: Andriy Gapon CommitDate: 2021-12-10 12:33:12 +0000 iflib_stop: drain rx tasks to prevent any data races iflib_stop modifies iflib data structures that are used by _task_fn_rx, most prominently the free lists. So, iflib_stop has to ensure that the rx task threads are not active. This should help to fix a crash seen when iflib_if_ioctl (e.g., SIOCSIFCAP) is called while there is already traffic flowing. The crash has been seen on VMWare guests with vmxnet3 driver. My guess is that on physical hardware the couple of 1ms delays that iflib_stop has after disabling interrupts are enough for the queued work to be completed before any iflib state is touched. But on busy hypervisors the guests might not get enough CPU time to complete the work, thus there can be a race between the taskqueue threads and the work done to handle an ioctl, specifically in iflib_stop and iflib_init_locked. PR: 259458 (cherry picked from commit 1bfdb812c786ac2607a82633f9c84a5d16f54079) --- sys/net/iflib.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sys/net/iflib.c b/sys/net/iflib.c index 3a86671c7443..304518e2a24d 100644 --- a/sys/net/iflib.c +++ b/sys/net/iflib.c @@ -2692,7 +2692,8 @@ iflib_stop(if_ctx_t ctx) bzero((void *)di->idi_vaddr, di->idi_size); } for (i = 0; i < scctx->isc_nrxqsets; i++, rxq++) { - /* make sure all transmitters have completed before proceeding XXX */ + gtaskqueue_drain(rxq->ifr_task.gt_taskqueue, + &rxq->ifr_task.gt_task); rxq->ifr_cq_cidx = 0; for (j = 0, di = rxq->ifr_ifdi; j < sctx->isc_nrxqs; j++, di++)