Making pkg_XXX tools smarter about file types...

[Bruce A. Mah]

If memory serves me right, Tim Kientzle wrote:
> The attached patch modifies the pkg_install
> tools to inspect the file contents--rather than the
> filename extension--to determine the
> compression method in use.  It then feeds the data
> into the correct invocation of 'tar'.
> I've also modified exec.c/lib.h to
> factor out and expose some common code that
> formats shell command lines.
> 
> This approach makes it possible, for instance, to
> fix a single file extension (e.g. '.freebsd'
> or '.package') that would not have to change
> even if the internal format of a package were
> to change (as has already occurred once, with
> the transition from gzip to bzip2 compression).
> 
> Note that this could also be fairly easily extended
> to support a variety of alternative archive
> types.  (E.g., the pkg_XXX tools could be
> modified to support 'zip' or 'tar' archives
> transparently to the user.)

(A month and a half passes...I meant to get back to you earlier but
didn't have any time to play with this before.  Actually I still
don't, but...)

The concept is good, and it's something we've needed for awhile.  I
suspect you followed the various adventures of pkg_add and sysinstall
when we tried supporting both bzip2 and gzip packages for various
releases and developer previews, before we settled on the current
"bzip2 for 5.X and gzip for 4.X" as something that actually worked.

A little feedback on the patch itself (functionality only): Basically,
it works great for the case of a package coming in on stdin.  If the
package comes from a file, then pkg_add wants to make two passes over
the package, first to get the +CONTENTS file and second to actually
unpack everything.  When the first tar process finishes reading the
+CONTENTS file, it closes its pipe (due to the --fast-read argument).
However, pkg_add still seems to be writing to the pipe...this seems to
be bad.

An example with pkg_add built with CFLAGS=-DDEBUG:

tomcat:add# cat ~/tmp/bash.pkg | ./pkg_add -
Piping package '-' to cmd 'tar -xpjf - '
updating /etc/shells
Executing /usr/sbin/mtree -U -f +MTREE_DIRS -d -e -p /usr/local >/dev/null
Executing mkdir /var/db/pkg/bash-2.05b.004
Executing chmod a+rx /var/db/pkg/bash-2.05b.004
Executing mv ./+DESC /var/db/pkg/bash-2.05b.004
Executing mv ./+COMMENT /var/db/pkg/bash-2.05b.004
Executing mv ./+MTREE_DIRS /var/db/pkg/bash-2.05b.004
Executing rm -rf /var/tmp/instmp.BGdXjm
tomcat:add# ./pkg_add ~/tmp/bash.pkg
Piping package '/usr/users/bmah/tmp/bash.pkg' to cmd 'tar -xpjf - --fast-read - +CONTENTS'
Broken pipe

It works if I remove the --fast-read flag from the tar, but that's not
the right answer.

Bruce.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20030328/d8d1ebea/attachment.bin