workaround for VMware WS NAT bug triggered by OpenSSH 7.8p1 changes

Sat Dec 22 21:03:24 UTC 2018

In message <0503b382-d886-39a4-d265-b43d8adc15c9 at yuripv.net>, Yuri 
Pankov write
s:
> This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
> --e7sW91Qf9WxzTaujtGEdAimN5k2EtpJ6Q
> Content-Type: multipart/mixed; boundary="3a9zlXDI7Z2P48EdQkgwMdVOMQOEfR5Wm";
>  protected-headers="v1"
> From: Yuri Pankov <yuripv at yuripv.net>
> To: Cy Schubert <Cy.Schubert at cschubert.com>
> Cc: Mark Peek <mp at freebsd.org>, Enji Cooper <yaneurabeya at gmail.com>,
>  Warner Losh <imp at bsdimp.com>, =?UTF-8?Q?Dag-Erling_Sm=c3=b8rgrav?=
>  <des at freebsd.org>, freebsd-current <current at freebsd.org>
> Message-ID: <0503b382-d886-39a4-d265-b43d8adc15c9 at yuripv.net>
> Subject: Re: workaround for VMware WS NAT bug triggered by OpenSSH 7.8p1
>  changes
> References: <201812222027.wBMKRGWJ050853 at slippy.cwsent.com>
> In-Reply-To: <201812222027.wBMKRGWJ050853 at slippy.cwsent.com>
>
> --3a9zlXDI7Z2P48EdQkgwMdVOMQOEfR5Wm
> Content-Type: text/plain; charset=utf-8
> Content-Language: en-US
> Content-Transfer-Encoding: quoted-printable
>
> Cy Schubert wrote:
> > In message <e84b7b4a-89ab-2ad9-ac3a-e08b8491e5cc at yuripv.net>, Yuri=20
> > Pankov write
> > s:
> >> This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
> >> --NAG3HGfiwhsHyGq3aNdsIv1NzTEMODbUH
> >> Content-Type: multipart/mixed; boundary=3D"c7yUHUJpZYpJqOrOWLAb4sE3Rmh=
> 2alrdi";
> >>  protected-headers=3D"v1"
> >> From: Yuri Pankov <yuripv at yuripv.net>
> >> To: Cy Schubert <Cy.Schubert at cschubert.com>
> >> Cc: Mark Peek <mp at freebsd.org>, Enji Cooper <yaneurabeya at gmail.com>,
> >>  Warner Losh <imp at bsdimp.com>, =3D?UTF-8?Q?Dag-Erling_Sm=3Dc3=3Db8rgra=
> v?=3D
> >>  <des at freebsd.org>, freebsd-current <current at freebsd.org>
> >> Message-ID: <e84b7b4a-89ab-2ad9-ac3a-e08b8491e5cc at yuripv.net>
> >> Subject: Re: workaround for VMware WS NAT bug triggered by OpenSSH 7.8=
> p1
> >>  changes
> >> References: <201812222009.wBMK9H5T050103 at slippy.cwsent.com>
> >> In-Reply-To: <201812222009.wBMK9H5T050103 at slippy.cwsent.com>
> >>
> >> --c7yUHUJpZYpJqOrOWLAb4sE3Rmh2alrdi
> >> Content-Type: text/plain; charset=3Dutf-8
> >> Content-Language: en-US
> >> Content-Transfer-Encoding: quoted-printable
> >>
> >> Cy Schubert wrote:
> >>> In message <913730b6-c6f0-60b8-a589-e89e872b7f42 at yuripv.net>, Yuri=3D=
> 20
> >>> Pankov write
> >>> s:
> >>>> Yuri Pankov <yuripv at yuripv.net> wrote:
> >>>>> In-Reply-To: <CAGGgMJf45vkNY6o6-in+kiAFHxsFZpKBc4Oa6qiCFnzKnRjk1g at m=
> ai=3D
> >>
> >>> l.gmail.
> >>>>> com>
> >>>>> Mark Peek wrote:
> >>>>>> On Fri, Dec 21, 2018 at 9:30 PM Enji Cooper <yaneurabeya at gmail.com=
> >
> >>>  wro=3D3D
> >>>>> te:
> >>>>>> =3D3D20
> >>>>>>>
> >>>>>>>> On Dec 21, 2018, at 17:48, Yuri Pankov <yuripv at yuripv.net> wrote=
> :
> >>>>>>>>
> >>>>>>>> Mark Peek wrote:
> >>>>>>>>> Thanks for the cc:. I forwarded the original report on to an=3D=
> 20
> >>> interna=3D3D
> >>>>> l
> >>>>>>>>> VMware desktop product contact.
> >>>>>>>>
> >>>>>>>> Thank you.
> >>>>>>>>
> >>>>>>>>> What version of Workstation or Fusion is this occurring on? I=3D=
> 20
> >>> saw
> >>>>>>>>> Workstation 14 mentioned but curious if it occurs on=3D20
> >>> Workstation 15
> >>>>>>>>> (latest).
> >>>>>>>>
> >>>>>>>> Running the latest available for download: 15.0.2 build-10952284=
> =2E
> >>>>>>>
> >>>>>>> This is affecting me on VMware Fusion 11.0.1-11.0.2. I know it=3D=
> 20
> >>> wasn=3D3DE2=3D3D
> >>>>> =3D3D80=3D3D99t
> >>>>>>> affecting me on 10.x. I didn=3D3DE2=3D3D80=3D3D99t install 11.0.0=
> , so I=3D20
> >>> don=3D3DE2=3D3D80=3D3D99=3D3D
> >>>>> t know if it
> >>>>>>> affects that version...
> >>>>>>>
> >>>>>>> Thanks so much!
> >>>>>>>
> >>>>>>> -Enji
> >>>>>> =3D3D20
> >>>>>> =3D3D20
> >>>>>> BTW, there appears to be a workaround here using -o=3D20
> >>> 'IPQoS=3D3D3Dthroughput=3D3D
> >>>>> '
> >>>>>> (untested by me). I've seen the issue forwarded internally but no=3D=
> 20
> >>> furth=3D3D
> >>>>> er
> >>>>>> discussions yet.
> >>>>>> =3D3D20
> >>>>>> https://communities.vmware.com/thread/590825
> >>>>
> >>>> Yes, that's exactly what the patch attached to original message does=
>  i=3D
> >> f
> >>>> we are running as a VMware guest.  The workaround is known and it wo=
> rk=3D
> >> s,
> >>>> but it's not immediately clear and I just wanted it to be the defaul=
> t
> >>>> for the time being.
> >>> =3D20
> >>> The patch assumes VMWARE_GUEST_WORKAROUND unconditionally. Is this=3D=
> 20
> >>> intended?
> >>
> >> It's the added code that is ifdef'ed VMWARE_GUEST_WORKAROUND, so it ca=
> n
> >> be ripped out easily when no longer needed, and yes, it's enabled
> >> unconditionally for now.  And the check itself is if 'kern.vm_guest'
> >> reports 'vmware'.
> >=20
> > It doesn't look that conditional to me.
>
> Indeed, and that's what I said exactly :-)  The added code is enabled
> unconditionally, and the added code also has a check for vmware guest.
> The ifdefs are there only to show that this is local addition, nothing el=
> se.
>
> I'm not saying it needs to be done this way, this is just something I
> did quickly after installing yet another VM and forgetting to modify my
> ~/.ssh/config to include the workaround.

First and foremost a ticket with VMware should be opened. They really 
need to fix yet another regression.

Regarding the Red Hat bugzilla bug, looks like they're doing the right 
thing by reaching out to VMware. This should be our position as well. 
Add it to ssh_config or sshd_config if one must but have VMware fix 
their bugs. Putting workarounds in our O/S to work around a bug in some 
other vendor's virtualization is something I don't support. If we must 
add the #ifdefs to our ssh, then add an UPDATING entry to say that to 
enable it put VMWARE_GUEST_WORKAROUND or however we choose to enable it 
in src.conf.

This reminds me of an issue we had on Red Hat Linux last year (or was 
it the year before -- time just flies) which between VMware and HPE was 
causing serious performance impact. And before that a vmxnet3 issue 
that caused the Linux kernel to panic. In both cases RH did not change 
their distribution but offered workarounds until the other vendors had 
resolved the issues. RH is taking the same stance here. I think we 
should not patch our ssh but instead offer a workaround just as RH has.

We, FreeBSD, should try to open a ticket or reach out to VMware to add 
a +1 to the issue that RH has already opened. This is the right thing 
to do. In this case we should consider ourselves an O/S vendor too, 
which BTW we are.

BTW the 2018-11-08 entry in the RH bug talks about adding the 
workaround to sshd_config.

-- 
Cheers,
Cy Schubert <Cy.Schubert at cschubert.com>
FreeBSD UNIX:  <cy at FreeBSD.org>   Web:  http://www.FreeBSD.org

	The need of the many outweighs the greed of the few.