From nobody Tue Sep 26 09:03:30 2023
X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Rvv122h4Tz4tsQ3
	for <freebsd-stable@mlmmj.nyi.freebsd.org>; Tue, 26 Sep 2023 09:03:34 +0000 (UTC)
	(envelope-from kp@FreeBSD.org)
Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
	 client-signature RSA-PSS (4096 bits) client-digest SHA256)
	(Client CN "smtp.freebsd.org", Issuer "R3" (verified OK))
	by mx1.freebsd.org (Postfix) with ESMTPS id 4Rvv1228dPz4dmY;
	Tue, 26 Sep 2023 09:03:34 +0000 (UTC)
	(envelope-from kp@FreeBSD.org)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim;
	t=1695719014;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=NccwK/aDqXGPDjSqVx/0oNiyLI22mp8Fwa/w56W0pvY=;
	b=QKH9H20st7XpfjlpLzBZC3V6gCMPPTkwsJK6cRHRwFgBlj5d72y8xUsPvFd3FML7rNhLfM
	WZcmNJAJdBgbgWs1B5+BzF/FLQgS6lm6h91jylIhFaOtWuIqOKmqiWxZAWvekw5wdlxIGL
	clbe5fnpm2FI0gKZ5Km+boTAdTv3XMwpdc+DvREjDN/Ke47x1LwDVKLcF5fl/rbF40Ktpp
	y/zQ7v0V4g1nmT45sMInur+DM8WhelDL7VHSp60eijxii+2gFmIi4JLu3OFc+6f22UPaxB
	SAPKjtyry4V/jW2uLE4V+Oy5mJeIkCB4G4ZzHY1+t37AFINwUA/CRC4vAu94cw==
ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1695719014; a=rsa-sha256; cv=none;
	b=ir8VwrJfemEdAuFgRIXw3fy4XgUgFL7mD+kAwF1yXrt9m0HDE8ttmf5pzzI7iXZXJQSlhs
	KyfhL61l1O/+Ob+wZqr211OTPbREupJcdrqwFMeNZs4Obf3b4H2P8Q2bAOxS/bV3gox3Al
	wt8WYBnGJ7rCRhPf8nN/U66/OLOM9RAGlNFegBnSbgI6e+Uq671muHkA0MWo7fsCo4EbWx
	cVFdm8Kpy6XmahlY3QnUllDdQ3Yee9mywqGFaVNJZjuoeQ9Pmvw1Rtk/R8uk7lc1VmMufv
	dr+yCRh8RB+caGRMGbCYnV9uxi+tjZuUSKutB+JEWggz7l/BCYRn/kwdcbsoNA==
ARC-Authentication-Results: i=1;
	mx1.freebsd.org;
	none
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org;
	s=dkim; t=1695719014;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references;
	bh=NccwK/aDqXGPDjSqVx/0oNiyLI22mp8Fwa/w56W0pvY=;
	b=SRqeRfFYgGNNe2pMyJ0sEUHZS8m6LjVR8DKjzbYpP49xQcgAGpKiEnJvbulzdJielSmhg2
	QSz06jYhZ89ZTcxcYXQafl1pdYdwiO9KeAJNWzeETxoOjX98ut9wx63LovUdc+0eKCGn9I
	rJHBPEb5glBod2p5v1pKxSfsZaU5FtU31iW30XLhP5fXZB84dO+FwqYYphS3A32iUhPZru
	A7zkAh/PR8nVk6L9ctBf3butEiTYGKjla7KTUFU8oIq6aqEUqUoqeAQ071/xQcjb4DQv/g
	elPtiYIyChH0Zn0TwnaLw55xp8sBf/6f4q2yALL7bVx5ZsndKSw+h/bIdUyuAA==
Received: from venus.codepro.be (venus.codepro.be [5.9.86.228])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "mx1.codepro.be", Issuer "R3" (verified OK))
	(Authenticated sender: kp)
	by smtp.freebsd.org (Postfix) with ESMTPSA id 4Rvv120bpfz8C7;
	Tue, 26 Sep 2023 09:03:33 +0000 (UTC)
	(envelope-from kp@FreeBSD.org)
Received: by venus.codepro.be (Postfix, authenticated sender kp)
 id CFB243F01F;
	Tue, 26 Sep 2023 11:03:31 +0200 (CEST)
From: Kristof Provost <kp@FreeBSD.org>
To: mike tancsa <mike@sentex.net>
Cc: FreeBSD-STABLE Mailing List <freebsd-stable@freebsd.org>
Subject: Re: ifconfig -v ix0 output delay
Date: Tue, 26 Sep 2023 11:03:30 +0200
X-Mailer: MailMate (1.14r5937)
Message-ID: <4AEC7301-88C7-4F42-AFDA-89461AF5B647@FreeBSD.org>
In-Reply-To: <dcf71d1b-7bae-4db7-bd97-9ecb7f4d8a64@sentex.net>
References: <dcf71d1b-7bae-4db7-bd97-9ecb7f4d8a64@sentex.net>
List-Id: Production branch of FreeBSD source code <freebsd-stable.freebsd.org>
List-Archive: https://lists.freebsd.org/archives/freebsd-stable
List-Help: <mailto:stable+help@freebsd.org>
List-Post: <mailto:stable@freebsd.org>
List-Subscribe: <mailto:stable+subscribe@freebsd.org>
List-Unsubscribe: <mailto:stable+unsubscribe@freebsd.org>
Sender: owner-freebsd-stable@freebsd.org
X-BeenThere: freebsd-stable@freebsd.org
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 25 Sep 2023, at 23:12, mike tancsa wrote:
> Hi All,
>
> =C2=A0=C2=A0=C2=A0 A small annoyance, but I was wondering why "ifconfig=
 -v ix0" seems to take a "long time" compared to other 10G nics.=C2=A0 e.=
g.
>
>
> 0(nfs3b2)# time ifconfig -v ix0
> ix0: flags=3D8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu =
1500
> options=3D4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_=
HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWT=
SO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ether 0c:c4:7a:6f:20:a0
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 inet6 fe80::ec4:7aff:fe6f:20=
a0%ix0 prefixlen 64 scopeid 0x1
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 inet6 2607:f3e0:0:6:ec4:7aff=
:fe6f:20a0 prefixlen 64 autoconf
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 inet 10.255.255.132 netmask =
0xffffff00 broadcast 10.255.255.255
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 media: Ethernet autoselect (=
1000baseT <full-duplex>)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 status: active
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 nd6 options=3D23<PERFORMNUD,=
ACCEPT_RTADV,AUTO_LINKLOCAL>
> 0.000u 1.251s 0:01.25 100.0%=C2=A0=C2=A0=C2=A0 167+198k 0+0io 0pf+0w
> 0(nfs3b2)#
>
> vs
>
> % time ifconfig -v cxl0
> cxl0: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu=
 1500
> options=3D6ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_=
HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6,HWRXTST=
MP,NOMAP>
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ether 00:07:43:60:c4:b0
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 inet 10.251.12.1 netmask 0xf=
fffff00 broadcast 10.251.12.255
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 media: Ethernet 10Gbase-Twin=
ax <full-duplex>
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 status: active
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 nd6 options=3D29<PERFORMNUD,=
IFDISABLED,AUTO_LINKLOCAL>
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 plugged: SFP/SFP+/SFP28 Unkn=
own (Copper pigtail)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 vendor: OEM PN: SFP-H10GB-CU=
5M SN: S220304060201 DATE: 2022-03-24
> 0.000u 0.002s 0:00.02 0.0%=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0+0k 0+0io 0pf=
+0w
>
>
> Going through truss, the delay seems to be after "ioctl(3,SIOCGIFSTATUS=
,0x65d9929d0f0)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 ERR#22 'Invalid argument'"
>
>
> socket(PF_INET,SOCK_DGRAM|SOCK_CLOEXEC,0)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 =3D 6 (0x6)
> ioctl(6,SIOCGIFINDEX,0x65d9929cbf0)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D 0 (0x0)
> close(6)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D 0 (0x0)
> ioctl(5,SIOCGDEFIFACE_IN6,0x65d9929cca0)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 =3D 0 (0x0)
> close(5)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D 0 (0x0)
> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 nd6 options=3D23<PERFORMNUD,=
ACCEPT_RTADV,AUTO_LINKLOCAL>
> write(1,"\tnd6 options=3D23<PERFORMNUD,ACCE"...,56) =3D 56 (0x38)
> ioctl(3,SIOCGIFSTATUS,0x65d9929d0f0)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ERR#22 'Invalid argument'
>
>
>
> ioctl(4,SIOCGI2C,0x65d9929ca50)=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D 0 (0x0)

It=E2=80=99s this call. That ends up in ixgbe_read_i2c_byte_generic_int()=
, where it=E2=80=99ll retry 11 times, taking roughly 100ms per attempt.

I either do not understand how SFP type and presence detection is done, o=
r it=E2=80=99s broken, so I=E2=80=99ve not yet been able to persuade it t=
o be less awful. Ideally the Intel people should take a look at this.

Best regards,
Kristof