From nobody Tue Jul 20 07:15:02 2021 X-Original-To: ipfw@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 1F6021273B01 for ; Tue, 20 Jul 2021 07:15:05 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GTVN90NYZz3v7C for ; Tue, 20 Jul 2021 07:15:05 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.codepro.be", Issuer "R3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id BD01D9EA7 for ; Tue, 20 Jul 2021 07:15:04 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: by venus.codepro.be (Postfix, authenticated sender kp) id 2317E359D2; Tue, 20 Jul 2021 09:15:03 +0200 (CEST) From: "Kristof Provost" To: ipfw@FreeBSD.org Subject: dummynet configuration for automated tests Date: Tue, 20 Jul 2021 09:15:02 +0200 X-Mailer: MailMate (1.13.2r5673) Message-ID: <4403D1A2-5162-4639-B6BB-5369EAA3E645@FreeBSD.org> List-Id: IPFW Technical Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-ipfw List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ipfw@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=_MailMate_CDAAED01-3A58-4EC4-BE2B-F145590BF07E_=" Content-Transfer-Encoding: 8bit X-ThisMailContainsUnwantedMimeParts: Y --=_MailMate_CDAAED01-3A58-4EC4-BE2B-F145590BF07E_= Content-Type: text/plain; charset="UTF-8"; format=flowed; markup=markdown Content-Transfer-Encoding: 8bit Hi, I’ve been trying (and failing) to write a few basic test cases for dummynet (with ipfw for now). The full test script can be found here: https://people.freebsd.org/~kp/dummynet.sh but the relevant bit is this: queue_v6_body() { fw=$1 firewall_init $fw dummynet_init $fw epair=$(vnet_mkepair) epair_link=$(vnet_mkepair) vnet_mkjail alcatraz ${epair}b ${epair_link}a vnet_mkjail srv ${epair_link}b set -x ifconfig ${epair}a inet6 2001:db8:42::1/64 no_dad up route add -6 2001:db8:43::/64 2001:db8:42::2 jexec alcatraz ifconfig ${epair}b inet6 2001:db8:42::2 no_dad up jexec alcatraz ifconfig ${epair_link}a inet6 2001:db8:43::2 no_dad up jexec alcatraz sysctl net.inet6.ip6.forwarding=1 jexec srv ifconfig ${epair_link}b inet6 2001:db8:43::1 no_dad up jexec srv route add -6 default 2001:db8:43::2 jexec srv /usr/sbin/inetd -p inetd-alcatraz.pid \ $(atf_get_srcdir)/../pf/echo_inetd.conf # Sanity check atf_check -s exit:0 -o ignore ping6 -i .1 -c 3 -s 1200 2001:db8:42::2 atf_check -s exit:0 -o ignore ping6 -i .1 -c 3 -s 1200 2001:db8:43::2 atf_check -s exit:0 -o ignore ping6 -i .1 -c 3 -s 1200 2001:db8:43::1 reply=$(echo "foo" | nc -w 5 -N 2001:db8:43::1 7) if [ "$reply" != "foo" ]; then atf_fail "Echo sanity check failed" fi jexec alcatraz dnctl pipe 1 config bw 300Byte/s queue 5 mask proto 0xff jexec alcatraz dnctl sched 1 config pipe 1 type wf2q+ mask proto 0xff jexec alcatraz dnctl queue 1 config sched 1 weight 99 queue 5 mask proto 0xff jexec alcatraz dnctl queue 2 config sched 1 weight 1 queue 5 mask proto 0xff firewall_config alcatraz ${fw} \ "ipfw" \ "ipfw add queue 2 ipv6-icmp from any to any icmp6types 128,129" \ "ipfw add queue 1 tcp from any to any" # Single ping succeeds atf_check -s exit:0 -o ignore ping6 -c 3 2001:db8:43::1 # Unsaturated TCP succeeds reply=$(echo "foo" | nc -w 5 -N 2001:db8:43::1 7) if [ "$reply" != "foo" ]; then atf_fail "Unsaturated echo failed" fi # Saturate the link ping6 -i .01 -s 1200 2001:db8:43::1 & # Give that a chance to fill the queue & pipe sleep 1 jexec alcatraz ipfw show # We should now be hitting the limits and get this packet dropped. atf_check -s exit:2 -o ignore ping6 -c 1 -W 1 -s 1200 2001:db8:43::1 # TCP should still just pass for i in `seq 0 4` do reply=$(echo "foo $i" | nc -w 10 -N 2001:db8:43::1 7) if [ "$reply" != "foo $i" ]; then atf_fail "Failed to prioritise traffic on interation $i" fi sleep 1 done jexec alcatraz ipfw flush # This will fail if we don't differentiate the traffic firewall_config alcatraz ${fw} \ "ipfw" \ "ipfw add queue 1 ipv6-icmp from any to any icmp6types 128,129" \ "ipfw add queue 2 tcp from any to any" # Carry over state? killall ping6 ping6 -i .01 -s 1200 2001:db8:43::1 & sleep 1 reply=$(echo "baz" | nc -w 10 -N 2001:db8:43::1 7) if [ "$reply" == "baz" ]; then jexec alcatraz ipfw show atf_fail "TCP still made it through, even when not prioritised" fi } The idea is to set up a very slow link (using a pipe), and then to send both ICMP echo and TCP traffic through it. There’s vastly more ICMP traffic than TCP, and the expectation is that without prioritisation the ICMP traffic will drown out TCP and cause the connection to fail. We then try to use dummynet to give TCP priority over ICMP, so that the TCP connections do succeed. However, I simply cannot get it to behave in any sort of predictable or consistent way. Sometimes the TCP connection succeeds, despite attempts to prioritise ICMP, or vice versa. Clearly I’m misconfiguring something, but at this point I do not understand what. Does anyone see my mistake, or have any relevant configuration examples to share? Thanks, Kristof --=_MailMate_CDAAED01-3A58-4EC4-BE2B-F145590BF07E_=--