From nobody Sun May 28 07:07:08 2023 X-Original-To: freebsd-questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QTV8q1Wq2z4Wjtd for ; Sun, 28 May 2023 07:07:23 +0000 (UTC) (envelope-from pprocacci@gmail.com) Received: from mail-oa1-x29.google.com (mail-oa1-x29.google.com [IPv6:2001:4860:4864:20::29]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QTV8p0nrMz3K09 for ; Sun, 28 May 2023 07:07:22 +0000 (UTC) (envelope-from pprocacci@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20221208 header.b=PgqoTjaa; spf=pass (mx1.freebsd.org: domain of pprocacci@gmail.com designates 2001:4860:4864:20::29 as permitted sender) smtp.mailfrom=pprocacci@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-oa1-x29.google.com with SMTP id 586e51a60fabf-19a36435e81so2556553fac.1 for ; Sun, 28 May 2023 00:07:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685257640; x=1687849640; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=81agzeE+wQ+ZvKC0g1t+aSU2VoCe7Cu+yYww3jAbCls=; b=PgqoTjaa27bYB2KFV8RTNLHcrvVHwc+h0f2+hFetg7n+PvxWmn1BDeWgnsdGzVLA3c IBYa+7Ufad+GB4GdhwaHpiQ0l5Kg+r0hLrCdR8TIvDw5pG3lryGcD9cpqTd88Q+d5nrB 9r9uKJSwfEsB777xXc2Ho8OhCvR4hX3LLvfrBE2yQldGncr4jzyoWmAxnHcE+ImIdaXo CNgQtVClcRk/Jjp5T9ADyThYZPrg4BZ4KI0gNNsRbIKVIyGm/iKcoccSvW3lDLo0h6Ye AaYes6UHvQqCFncfkAfaWyGshhpTaAbrs2m3y/fOlHPzON4y70a5Su3rcpFHZn77/LdK JfDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685257640; x=1687849640; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=81agzeE+wQ+ZvKC0g1t+aSU2VoCe7Cu+yYww3jAbCls=; b=USzVa86ySNpegUtwcxWkPh6tHC4mALm1YazVJ5pKLocUDOTdwejzg48OkI0lOedLex QOnM67+Llct+Dz2o3pcO6UgXNvn4Nbzbr2QcBMSswijw66BmDMDw0miCWxAJfjovO5yk VJvWJKF32aEuVoRTKdNk9W1cYTGGoHxyzuyppH50HilASUCszJ74ChxJuhgy3bP4EtSS tt7n3aJMeuuwAXQyFrNDq8sk/qlSaQpPK5S88ci2kYbPVXRDk19sU1a3yekztJcB+PSw izna1bSiTCWH/lzYOubqrD9App4om81gZJLHZmstYG7IT+E0zvCTitSQ5KHrJP5T9+BQ pbRQ== X-Gm-Message-State: AC+VfDzrh46XdU2nLUWQygL4s1DSQJpNQruYu/CSFO2V2lHiD1h6RnwQ jndal67mTUUqWLx7ll+aTUKFHSty+tiE380gy+19106M7A== X-Google-Smtp-Source: ACHHUZ7fVcu7viXWeaf2YhWC/L1sayE3A9V9dkXghRgd3zbweWBiBjdGHJTwigR5V28DN7JnUkSqbPozRJwQNkv2MFs= X-Received: by 2002:a05:6870:8a0a:b0:19e:2db8:c021 with SMTP id p10-20020a0568708a0a00b0019e2db8c021mr2160345oaq.16.1685257640450; Sun, 28 May 2023 00:07:20 -0700 (PDT) List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Paul Procacci Date: Sun, 28 May 2023 03:07:08 -0400 Message-ID: Subject: Re: What happened to my /tmp/mysql.sock ?!!??! To: Aryeh Friedman Cc: Odhiambo Washington , FreeBSD Mailing List Content-Type: multipart/alternative; boundary="00000000000003a02605fcbb9fca" X-Spamd-Result: default: False [-3.84 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.986]; NEURAL_HAM_MEDIUM(-0.85)[-0.852]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20221208]; R_SPF_ALLOW(-0.20)[+ip6:2001:4860:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCPT_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; TAGGED_RCPT(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; MLMMJ_DEST(0.00)[freebsd-questions@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[2001:4860:4864:20::29:from]; ARC_NA(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; DKIM_TRACE(0.00)[gmail.com:+]; SUBJECT_ENDS_EXCLAIM(0.00)[]; TO_DN_ALL(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2001:4860:4864::/48, country:US]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4QTV8p0nrMz3K09 X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N --00000000000003a02605fcbb9fca Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sun, May 28, 2023 at 1:22=E2=80=AFAM Aryeh Friedman wrote: > On Sun, May 28, 2023 at 12:55=E2=80=AFAM Paul Procacci > wrote: > > > > > > > > On Sat, May 27, 2023 at 10:33=E2=80=AFPM Aryeh Friedman < > aryeh.friedman@gmail.com> wrote: > >> > >> On Sat, May 27, 2023 at 10:17=E2=80=AFPM Paul Procacci > wrote: > >> > > >> > > >> > > >> > On Sat, May 27, 2023 at 8:14=E2=80=AFPM Aryeh Friedman < > aryeh.friedman@gmail.com> wrote: > >> >> > >> >> On Sat, May 27, 2023 at 1:21=E2=80=AFPM Paul Procacci > wrote: > >> >> > > >> >> > > >> >> > > >> >> > On Sat, May 27, 2023 at 1:13=E2=80=AFPM Aryeh Friedman < > aryeh.friedman@gmail.com> wrote: > >> >> >> > >> >> >> On Sat, May 27, 2023 at 1:09=E2=80=AFPM Odhiambo Washington < > odhiambo@gmail.com> wrote: > >> >> >> > 2023-05-27T12:40:12.835631Z 0 [ERROR] Fatal error: Can't open > and lock privilege tables: Table 'mysql.user' doesn't exist > >> >> >> > 2023-05-27T12:40:12.835637Z 0 [ERROR] Fatal error: Failed to > initialize ACL/grant/time zones structures or failed to remove temporary > table files. > >> >> >> > 2023-05-27T12:40:12.835752Z 0 [ERROR] Aborting > >> >> >> > > >> >> >> > > >> >> >> > Please completely uninstall Mysql and reinstall afresh and it > will start. > >> >> >> > >> >> >> This *IS* a completely new install !?!?!? Literally installed 5 > mins > >> >> >> before I posted on a complete fresh VM > >> >> >> > > >> >> >> > Even better, install MariaDB :-) > >> >> >> > >> >> >> Since I need to debug a mysql DB on a production server not an > option > >> >> >> -- > >> >> >> Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org > >> >> >> > >> >> > > >> >> > I didn't catch the version of mysql you were attempting to run bu= t > one of two commands will solve your problem. > >> >> > The first is potentially mysql_install_db. > >> >> > > >> >> > It's possible this command doesn't exist due to it being part of > mysqld in later versions of mysql. If that's the case then the following > should initialize the databases. > >> >> > > >> >> > mysqld --initialize > >> >> Already tried both this seems to be related to a series of bugs all > up > >> >> and down the mysql food chain in the last few weeks: > >> >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D270807 > >> >> > >> >> > >> >> > > >> >> > Once one of the two are completed, then you can try the things > that Mr. Washington was suggesting and repost an new error messages. > >> >> > > >> >> > ~Paul > >> >> > > >> >> > -- > >> >> > __________________ > >> >> > > >> >> > :(){ :|:& };: > >> >> > >> >> > >> >> > >> >> -- > >> >> Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org > >> > > >> > > >> > Some things excluded from the OPS original post (or I had to search > for referenced in the provided logs) > >> > The mysql version is 5.7.41. > >> > He's using latest. Not quarterly. > >> > > >> > Series of commands that I've done to create an mysql 5.7.41 > environment: > >> > zfs send tank/jails/templates/13.1-RELEASE-stripped@initial | zfs > recv tank/jails/hosts/mysql-test > >> > zfs set mountpoint=3D/jails/hosts/mysql-test tank/jails/hosts/mysql-= test > >> > pkg -r /jails/hosts/mysql-test install -y mysql57-server-5.7.41 > >> > /etc/rc.d/jail onestart mysql-test > >> > > >> > *To follow along the jail id that got created has a numeric > identifier of 17. > >> > From there: > >> > > >> > jexec -u root 17 /usr/local/etc/rc.d/mysql-server onestart > >> > # pgrep -j 17 -l mysqld > >> > 58398 mysqld > >> > > >> > I cannot replicate the OPs problem. > >> > > >> > Note the installation from the port: > >> > > >> > "=3D=3D=3D> NOTICE: > >> > > >> > This port is deprecated; you may wish to reconsider installing it:" > >> > >> It is the newest version of mysql that will not barf on the DB I am > >> attempting debug which a 5.1 with some really insane errors in it like > >> foreign-keys that are constraints but point to non-existent records. > >> And this is on a VM so need for fancy jailing. I am also on 13.2. > >> > >> > > >> > ~Paul > >> > -- > >> > __________________ > >> > > >> > :(){ :|:& };: > >> > >> > >> > >> -- > >> Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org > > > > > > > > You need to start giving us more to go on. Of your previous 5 posts, > you've just now decided to tell us it's running in a VM, yet even that > information is incomplete. > > > > Which vm is it running under? Which architecture is that VM running? > i386? adm64? powerpc? etc? > > bhyve (with petitecloud 0.2.6/1a1 running as a front end.... I can > handle any issues there if need to but I can't think of any) on a 12.4 > AMD64 host (ryzen 5 w/ 32 GB and 2 1 TB SSD's) > > > What kind of resources are allocated to the VM? Ram/cpu/disk > > 4 cores, 8 GB of RAM, 300 GB disk combined (mounted at /, > /var/db/mysql and /backups with nfs mounted /home) > > > You also mentioned 13.2, but didn't mention the branch or revision > you're running. Release? Stable? Current? > > release > > > > Given your response I may or may not be able to reproduce your > environment instead of trying to guess what it is. > > > > ~Paul > > -- > > __________________ > > > > :(){ :|:& };: > > > > -- > Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org > Thanks for providing that information. This is extremely helpful and I wish you led with these details. # /usr/local/etc/rc.d/mysql-server onestart 2023-05-28 02:52:49 [WARNING] mysql_install_db is deprecated. Please consider switching to mysqld --initialize 2023-05-28 02:52:49 [ERROR] The data directory '/var/db/mysql' already exist and is not empty. /usr/local/etc/rc.d/mysql-server: WARNING: failed precmd routine for mysql I got the above errors by editing the rc script and removing everything that was spitting useful information to /dev/null And sure enough: # ls -l /var/db/mysql total 640420 drwxrwxr-x 2 root operator 512 May 28 02:31 .snap -r-------- 1 root wheel 655589376 May 28 02:31 .sujournal This is obviously happening because /var/db/mysql is its own mount point and has journing enabled on a UFS FS. I suspect yours is the same. So, reboot into single user mode and turn off journaling on that FS. tunefs -j disable /var/db/myql From there go back into multiuser and continue... cd /var/db/mysql chflags noschg .sujournal rm -rf .sujournal .snap chown mysql:mysql . /usr/local/etc/rc.d/mysql-server onestart Success! pgrep -l mysqld # pgrep -l mysqld 2024 mysqld So, the above was successful to me. I'm unsure if you're running into the same problem. Regardless, modifying the rc scripts and removing the stdout and stderr redirection to /dev/null goes a long way. ~Paul --=20 __________________ :(){ :|:& };: --00000000000003a02605fcbb9fca Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Sun, May 28, 2023 at 1:22=E2= =80=AFAM Aryeh Friedman <aryeh.friedman@gmail.com> wrote:
On Sun, May 28, 2023 at 12:55=E2=80= =AFAM Paul Procacci <pprocacci@gmail.com> wrote:
>
>
>
> On Sat, May 27, 2023 at 10:33=E2=80=AFPM Aryeh Friedman <aryeh.friedman@gmail.co= m> wrote:
>>
>> On Sat, May 27, 2023 at 10:17=E2=80=AFPM Paul Procacci <pprocacci@gmail.com&g= t; wrote:
>> >
>> >
>> >
>> > On Sat, May 27, 2023 at 8:14=E2=80=AFPM Aryeh Friedman <aryeh.friedman@= gmail.com> wrote:
>> >>
>> >> On Sat, May 27, 2023 at 1:21=E2=80=AFPM Paul Procacci <= ;pprocacci@gmail.c= om> wrote:
>> >> >
>> >> >
>> >> >
>> >> > On Sat, May 27, 2023 at 1:13=E2=80=AFPM Aryeh Friedm= an <aryeh.= friedman@gmail.com> wrote:
>> >> >>
>> >> >> On Sat, May 27, 2023 at 1:09=E2=80=AFPM Odhiambo= Washington <odh= iambo@gmail.com> wrote:
>> >> >> > 2023-05-27T12:40:12.835631Z 0 [ERROR] Fatal= error: Can't open and lock privilege tables: Table 'mysql.user'= ; doesn't exist
>> >> >> > 2023-05-27T12:40:12.835637Z 0 [ERROR] Fatal= error: Failed to initialize ACL/grant/time zones structures or failed to r= emove temporary table files.
>> >> >> > 2023-05-27T12:40:12.835752Z 0 [ERROR] Abort= ing
>> >> >> > </quote>
>> >> >> >
>> >> >> > Please completely uninstall Mysql and reins= tall afresh and it will start.
>> >> >>
>> >> >> This *IS* a completely new install !?!?!?=C2=A0 = Literally installed 5 mins
>> >> >> before I posted on a complete fresh VM
>> >> >> >
>> >> >> > Even better, install MariaDB :-)
>> >> >>
>> >> >> Since I need to debug a mysql DB on a production= server not an option
>> >> >> --
>> >> >> Aryeh M. Friedman, Lead Developer, http://www.P= etiteCloud.org
>> >> >>
>> >> >
>> >> > I didn't catch the version of mysql you were att= empting to run but one of two commands will solve your problem.
>> >> > The first is potentially mysql_install_db.
>> >> >
>> >> > It's possible this command doesn't exist due= to it being part of mysqld in later versions of mysql.=C2=A0 If that's= the case then the following should initialize the databases.
>> >> >
>> >> > mysqld --initialize
>> >> Already tried both this seems to be related to a series o= f bugs all up
>> >> and down the mysql food chain in the last few weeks:
>> >> https://bugs.freebsd.org= /bugzilla/show_bug.cgi?id=3D270807
>> >>
>> >>
>> >> >
>> >> > Once one of the two are completed, then you can try = the things that Mr. Washington was suggesting and repost an new error messa= ges.
>> >> >
>> >> > ~Paul
>> >> >
>> >> > --
>> >> > __________________
>> >> >
>> >> > :(){ :|:& };:
>> >>
>> >>
>> >>
>> >> --
>> >> Aryeh M. Friedman, Lead Developer, http://www.PetiteClou= d.org
>> >
>> >
>> > Some things excluded from the OPS original post (or I had to = search for referenced in the provided logs)
>> > The mysql version is 5.7.41.
>> > He's using latest.=C2=A0 Not quarterly.
>> >
>> > Series of commands that I've done to create an mysql 5.7.= 41 environment:
>> > zfs send tank/jails/templates/13.1-RELEASE-stripped@initial |= zfs recv tank/jails/hosts/mysql-test
>> > zfs set mountpoint=3D/jails/hosts/mysql-test tank/jails/hosts= /mysql-test
>> > pkg -r /jails/hosts/mysql-test install -y mysql57-server-5.7.= 41
>> > /etc/rc.d/jail onestart mysql-test
>> >
>> > *To follow along the jail id that got created has a numeric i= dentifier of 17.
>> > From there:
>> >
>> > jexec -u root 17 /usr/local/etc/rc.d/mysql-server onestart >> > # pgrep -j 17 -l mysqld
>> > 58398 mysqld
>> >
>> > I cannot replicate the OPs problem.
>> >
>> > Note the installation from the port:
>> >
>> > "=3D=3D=3D>=C2=A0 =C2=A0NOTICE:
>> >
>> > This port is deprecated; you may wish to reconsider installin= g it:"
>>
>> It is the newest version of mysql that will not barf on the DB I a= m
>> attempting debug which a 5.1 with some really insane errors in it = like
>> foreign-keys that are constraints but point to non-existent record= s.
>> And this is on a VM so need for fancy jailing.=C2=A0 =C2=A0I am al= so on 13.2.
>>
>> >
>> > ~Paul
>> > --
>> > __________________
>> >
>> > :(){ :|:& };:
>>
>>
>>
>> --
>> Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org=
>
>
>
> You need to start giving us more to go on.=C2=A0 Of your previous 5 po= sts, you've just now decided to tell us it's running in a VM, yet e= ven that information is incomplete.
>
> Which vm is it running under?=C2=A0 Which architecture is that VM runn= ing?=C2=A0 i386?=C2=A0 adm64? powerpc?=C2=A0 etc?

bhyve (with petitecloud 0.2.6/1a1 running as a front end.... I can
handle any issues there if need to but I can't think of any) on a 12.4<= br> AMD64 host (ryzen 5 w/ 32 GB and 2 1 TB SSD's)

> What kind of resources are allocated to the VM?=C2=A0 Ram/cpu/disk

4 cores, 8 GB of RAM, 300 GB disk combined (mounted at /,
/var/db/mysql and /backups with nfs mounted /home)

> You also mentioned 13.2, but didn't mention the branch or revision= you're running.=C2=A0 Release?=C2=A0 Stable?=C2=A0 Current?

release
>
> Given your response I may or may not be able to reproduce your environ= ment instead of trying to guess what it is.
>
> ~Paul
> --
> __________________
>
> :(){ :|:& };:



--
Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org


Thanks for provid= ing that information.=C2=A0 This is extremely helpful and I wish you led wi= th these details.

# /usr/local/etc/rc.d/mysql-server onestart
202= 3-05-28 02:52:49 [WARNING] mysql_install_db is deprecated. Please consider = switching to mysqld --initialize
2023-05-28 02:52:49 [ERROR] =C2=A0 The = data directory '/var/db/mysql' already exist and is not empty.
/= usr/local/etc/rc.d/mysql-server: WARNING: failed precmd routine for mysql

I got the above errors by editing the rc script and r= emoving everything that was spitting useful information to /dev/null
<= div>
And sure enough:

# ls -l /var/db/mys= ql
total 640420
drwxrwxr-x =C2=A02 root =C2=A0operator =C2=A0 =C2=A0 = =C2=A0 =C2=A0512 May 28 02:31 .snap
-r-------- =C2=A01 root =C2=A0wheel = =C2=A0 =C2=A0 655589376 May 28 02:31 .sujournal

This is obvious= ly happening because /var/db/mysql is its own mount point and has journing = enabled on a UFS FS.=C2=A0 I suspect yours is the same.
So, reboo= t into single user mode and turn off journaling on that FS.
=C2= =A0tunefs -j disable /var/db/myql

From there g= o back into multiuser and continue...

cd /var/db/m= ysql
chflags noschg .sujournal
rm -rf .sujournal .s= nap
chown mysql:mysql .
/usr/local/etc/rc.d/mysql-serve= r onestart

Success!
pgrep -l mysqld
# pgrep -l mysqld
2024 mysqld

So, the above was su= ccessful to me.=C2=A0 I'm unsure if you're running into the same pr= oblem.=C2=A0 Regardless, modifying the rc scripts and removing the stdout a= nd stderr redirection to /dev/null goes a long way.

~Paul

--
__________________

:(){ :|:& };:
<= /div>
--00000000000003a02605fcbb9fca--