From nobody Mon Dec 19 15:39:07 2022 X-Original-To: freebsd-questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NbPM330qcz1G3B8 for ; Mon, 19 Dec 2022 15:51:11 +0000 (UTC) (envelope-from markus.graf@markusgraf.net) Received: from mx.virtual-earth.de (mx.virtual-earth.de [IPv6:2a01:4f8:120:7381:1:1:0:227]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mx.virtual-earth.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NbPM25KtPz3Jln for ; Mon, 19 Dec 2022 15:51:10 +0000 (UTC) (envelope-from markus.graf@markusgraf.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=mx.virtual-earth.de header.s=220816 header.b=aEgDMpOv; spf=pass (mx1.freebsd.org: domain of markus.graf@markusgraf.net designates 2a01:4f8:120:7381:1:1:0:227 as permitted sender) smtp.mailfrom=markus.graf@markusgraf.net; dmarc=pass (policy=none) header.from=markusgraf.net DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mx.virtual-earth.de; s=220816; t=1671465068; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=EwIArlcgGAjeLJHmMn1xi1YPwsmnKBjFFCo0c6ZwQQM=; b=aEgDMpOvQg9KK55Yrcd5esQAshcncDrNE10aRWYk9yqhayAXta+h2a8qdCKOu2AOhv2c0I DWo2WzZj3PIHoK84z2KnYudCAnmXWmNEHHWgSN34yRpnEygjXrPBt7tNn3Ii7RpUiB1U2O BNk/a9R20hvx9dim+T8W8KMk24X7SM0= Received: from beasty (p200300E12f2518000D835901499F7C6D.dip0.t-ipconnect.de [2003:e1:2f25:1800:d83:5901:499f:7c6d]) by mx.virtual-earth.de (OpenSMTPD) with ESMTPSA id 41141c33 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Mon, 19 Dec 2022 16:51:08 +0100 (CET) User-agent: mu4e 1.6.10; emacs 28.2 From: Markus Graf To: freebsd-questions@freebsd.org Subject: ipfw + bridge + epair + tags for vnet jails after upgrade to 13.1 Date: Mon, 19 Dec 2022 16:39:07 +0100 Message-ID: <867cyn2xzn.fsf@markusgraf.net> List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; format=flowed X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[markusgraf.net,none]; R_SPF_ALLOW(-0.20)[+ip6:2a01:4f8:120:7381::/64:c]; R_DKIM_ALLOW(-0.20)[mx.virtual-earth.de:s=220816]; MIME_GOOD(-0.10)[text/plain]; DKIM_TRACE(0.00)[mx.virtual-earth.de:+]; MLMMJ_DEST(0.00)[freebsd-questions@freebsd.org]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_NONE(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4NbPM25KtPz3Jln X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N Hi, I am trying to wrap my head around ifbridge and ipfw. I upgraded a host from 13.0 to 13.1 Up until now, I used ipfw, bridge and epairs on 13.0 and felt like I knew what I am doing. Now I can not get the system to behave like it should, so either my mental model is wrong or something changed. Perhaps someone could help me to understand what I am doing wrong. Environment ----------- I can't have a physical interface as member of the jailbridge, because this leaks virtual mac addresses of epair interfaces to the outside world where my hoster looks unkindly on mac-addresses not belonging to the nic of my server. So I have vnet jails behind a common ifbridge. All jails have their default routes point to the bridge-interface of the host. The host works as a router. Tags stopped working across vnet and bridge ------------------------------------------- On a long running host that is still currently running 13.0 I have this line in a vnet jail with an epair interface acme_j: allow tag 128 tcp from me to any 80,443 via acme_j setup uid root keep-state On the host I see the tags: # ipfw -a list 570 00570 112 11276 count tagged 128 On the updated 13.1 machine the host does not see the tags, or I can't get the host to count them. General understanding and testing --------------------------------- Man ifbridge pointed me to pfil. To learn and understand how these tools interact I put the following rules on the 13.1 host. count log via em0 count log via bridge0 count log via epair0a with epair0a being a member of the bridge. If I fetch a file in the vnet jail containing epair0b the counters of em0 and bridge0 increment, but the counter of epair0a does not increment. Tcpdump -i epair0a does show the traffic though. A similar count rule I left in two days ago has counted to 43 packets 3096 since then. How can that be? My sysctls are -------------- net.link.bridge.ipfw: 0 net.link.bridge.allow_llz_overlap: 0 net.link.bridge.inherit_mac: 0 net.link.bridge.log_stp: 0 net.link.bridge.pfil_local_phys: 0 net.link.bridge.pfil_member: 1 net.link.bridge.ipfw_arp: 0 net.link.bridge.pfil_bridge: 1 net.link.bridge.pfil_onlyip: 1 net.inet.ip.fw.dyn_keep_states: 1 net.inet.ip.fw.dyn_keepalive: 1 net.inet.ip.fw.dyn_short_lifetime: 5 net.inet.ip.fw.dyn_udp_lifetime: 10 net.inet.ip.fw.dyn_rst_lifetime: 1 net.inet.ip.fw.dyn_fin_lifetime: 1 net.inet.ip.fw.dyn_syn_lifetime: 20 net.inet.ip.fw.dyn_ack_lifetime: 300 net.inet.ip.fw.dyn_parent_max: 4096 net.inet.ip.fw.dyn_max: 16384 net.inet.ip.fw.dyn_buckets: 8192 net.inet.ip.fw.curr_max_length: 1 net.inet.ip.fw.curr_dyn_buckets: 256 net.inet.ip.fw.dyn_parent_count: 0 net.inet.ip.fw.dyn_count: 1 net.inet.ip.fw.enable: 1 net.inet.ip.fw.static_count: 51 net.inet.ip.fw.default_to_accept: 0 net.inet.ip.fw.tables_sets: 0 net.inet.ip.fw.tables_max: 128 net.inet.ip.fw.default_rule: 65535 net.inet.ip.fw.verbose_limit: 0 net.inet.ip.fw.verbose: 1 net.inet.ip.fw.autoinc_step: 100 net.inet.ip.fw.one_pass: 1 How do I control what traffic on which interfaces ipfw _should_ see and how do I ask it what it _actually_ sees? net.link.bridge.pfil_onlyip --------------------------- Does this setting mean that arp gets filtered? When does this happen. The docstring says when pfil is enabled. Does this refer to a sysctl being set, and if so which one, or ipfw being enabled? I assume that ipfw uses pfil. Do the pfil settings of net.link.bridge control how it connects to ipfw or am I barking up the wrong tree? Sorry for the wild questions. I am just very confused and propably just missing something very basic. Thank you in advance. Markus -- Markus Graf Tel.: +49 172 840 26 08 Email: markus.graf@markusgraf.net Web: markusgraf.net, headhuntertraining.net