Re: What happened to my /tmp/mysql.sock ?!!??!

From: Paul Procacci <pprocacci_at_gmail.com>
Date: Sun, 28 May 2023 07:07:08 UTC
On Sun, May 28, 2023 at 1:22 AM Aryeh Friedman <aryeh.friedman@gmail.com>
wrote:

> On Sun, May 28, 2023 at 12:55 AM Paul Procacci <pprocacci@gmail.com>
> wrote:
> >
> >
> >
> > On Sat, May 27, 2023 at 10:33 PM Aryeh Friedman <
> aryeh.friedman@gmail.com> wrote:
> >>
> >> On Sat, May 27, 2023 at 10:17 PM Paul Procacci <pprocacci@gmail.com>
> wrote:
> >> >
> >> >
> >> >
> >> > On Sat, May 27, 2023 at 8:14 PM Aryeh Friedman <
> aryeh.friedman@gmail.com> wrote:
> >> >>
> >> >> On Sat, May 27, 2023 at 1:21 PM Paul Procacci <pprocacci@gmail.com>
> wrote:
> >> >> >
> >> >> >
> >> >> >
> >> >> > On Sat, May 27, 2023 at 1:13 PM Aryeh Friedman <
> aryeh.friedman@gmail.com> wrote:
> >> >> >>
> >> >> >> On Sat, May 27, 2023 at 1:09 PM Odhiambo Washington <
> odhiambo@gmail.com> wrote:
> >> >> >> > 2023-05-27T12:40:12.835631Z 0 [ERROR] Fatal error: Can't open
> and lock privilege tables: Table 'mysql.user' doesn't exist
> >> >> >> > 2023-05-27T12:40:12.835637Z 0 [ERROR] Fatal error: Failed to
> initialize ACL/grant/time zones structures or failed to remove temporary
> table files.
> >> >> >> > 2023-05-27T12:40:12.835752Z 0 [ERROR] Aborting
> >> >> >> > </quote>
> >> >> >> >
> >> >> >> > Please completely uninstall Mysql and reinstall afresh and it
> will start.
> >> >> >>
> >> >> >> This *IS* a completely new install !?!?!?  Literally installed 5
> mins
> >> >> >> before I posted on a complete fresh VM
> >> >> >> >
> >> >> >> > Even better, install MariaDB :-)
> >> >> >>
> >> >> >> Since I need to debug a mysql DB on a production server not an
> option
> >> >> >> --
> >> >> >> Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org
> >> >> >>
> >> >> >
> >> >> > I didn't catch the version of mysql you were attempting to run but
> one of two commands will solve your problem.
> >> >> > The first is potentially mysql_install_db.
> >> >> >
> >> >> > It's possible this command doesn't exist due to it being part of
> mysqld in later versions of mysql.  If that's the case then the following
> should initialize the databases.
> >> >> >
> >> >> > mysqld --initialize
> >> >> Already tried both this seems to be related to a series of bugs all
> up
> >> >> and down the mysql food chain in the last few weeks:
> >> >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=270807
> >> >>
> >> >>
> >> >> >
> >> >> > Once one of the two are completed, then you can try the things
> that Mr. Washington was suggesting and repost an new error messages.
> >> >> >
> >> >> > ~Paul
> >> >> >
> >> >> > --
> >> >> > __________________
> >> >> >
> >> >> > :(){ :|:& };:
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org
> >> >
> >> >
> >> > Some things excluded from the OPS original post (or I had to search
> for referenced in the provided logs)
> >> > The mysql version is 5.7.41.
> >> > He's using latest.  Not quarterly.
> >> >
> >> > Series of commands that I've done to create an mysql 5.7.41
> environment:
> >> > zfs send tank/jails/templates/13.1-RELEASE-stripped@initial | zfs
> recv tank/jails/hosts/mysql-test
> >> > zfs set mountpoint=/jails/hosts/mysql-test tank/jails/hosts/mysql-test
> >> > pkg -r /jails/hosts/mysql-test install -y mysql57-server-5.7.41
> >> > /etc/rc.d/jail onestart mysql-test
> >> >
> >> > *To follow along the jail id that got created has a numeric
> identifier of 17.
> >> > From there:
> >> >
> >> > jexec -u root 17 /usr/local/etc/rc.d/mysql-server onestart
> >> > # pgrep -j 17 -l mysqld
> >> > 58398 mysqld
> >> >
> >> > I cannot replicate the OPs problem.
> >> >
> >> > Note the installation from the port:
> >> >
> >> > "===>   NOTICE:
> >> >
> >> > This port is deprecated; you may wish to reconsider installing it:"
> >>
> >> It is the newest version of mysql that will not barf on the DB I am
> >> attempting debug which a 5.1 with some really insane errors in it like
> >> foreign-keys that are constraints but point to non-existent records.
> >> And this is on a VM so need for fancy jailing.   I am also on 13.2.
> >>
> >> >
> >> > ~Paul
> >> > --
> >> > __________________
> >> >
> >> > :(){ :|:& };:
> >>
> >>
> >>
> >> --
> >> Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org
> >
> >
> >
> > You need to start giving us more to go on.  Of your previous 5 posts,
> you've just now decided to tell us it's running in a VM, yet even that
> information is incomplete.
> >
> > Which vm is it running under?  Which architecture is that VM running?
> i386?  adm64? powerpc?  etc?
>
> bhyve (with petitecloud 0.2.6/1a1 running as a front end.... I can
> handle any issues there if need to but I can't think of any) on a 12.4
> AMD64 host (ryzen 5 w/ 32 GB and 2 1 TB SSD's)
>
> > What kind of resources are allocated to the VM?  Ram/cpu/disk
>
> 4 cores, 8 GB of RAM, 300 GB disk combined (mounted at /,
> /var/db/mysql and /backups with nfs mounted /home)
>
> > You also mentioned 13.2, but didn't mention the branch or revision
> you're running.  Release?  Stable?  Current?
>
> release
> >
> > Given your response I may or may not be able to reproduce your
> environment instead of trying to guess what it is.
> >
> > ~Paul
> > --
> > __________________
> >
> > :(){ :|:& };:
>
>
>
> --
> Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org
>


Thanks for providing that information.  This is extremely helpful and I
wish you led with these details.

# /usr/local/etc/rc.d/mysql-server onestart
2023-05-28 02:52:49 [WARNING] mysql_install_db is deprecated. Please
consider switching to mysqld --initialize
2023-05-28 02:52:49 [ERROR]   The data directory '/var/db/mysql' already
exist and is not empty.
/usr/local/etc/rc.d/mysql-server: WARNING: failed precmd routine for mysql

I got the above errors by editing the rc script and removing everything
that was spitting useful information to /dev/null

And sure enough:

# ls -l /var/db/mysql
total 640420
drwxrwxr-x  2 root  operator        512 May 28 02:31 .snap
-r--------  1 root  wheel     655589376 May 28 02:31 .sujournal

This is obviously happening because /var/db/mysql is its own mount point
and has journing enabled on a UFS FS.  I suspect yours is the same.
So, reboot into single user mode and turn off journaling on that FS.
 tunefs -j disable /var/db/myql

From there go back into multiuser and continue...

cd /var/db/mysql
chflags noschg .sujournal
rm -rf .sujournal .snap
chown mysql:mysql .
/usr/local/etc/rc.d/mysql-server onestart

Success!
pgrep -l mysqld
# pgrep -l mysqld
2024 mysqld

So, the above was successful to me.  I'm unsure if you're running into the
same problem.  Regardless, modifying the rc scripts and removing the stdout
and stderr redirection to /dev/null goes a long way.

~Paul

-- 
__________________

:(){ :|:& };: