New FreeBSD package system (a.k.a. Daemon Package System (dps))

[Ivan Voras]

Jona Joachim wrote:

> I don't think it would be a good idea to use SQLite for this purpose.
> First of all using the file system is the Unix way of doing things. It's
>  efficient and easy to use, it transparent and user friendly. You can
> simply run vi to inspect a text file but you can't do this which an
> sqlite database. You have to learn sqlite to do it.

That particular barrier to entry / learning curve is very shallow.

In the end, it's a tradeoff between speed in the general case and ease
of use (from the developer side), vs convenience in the extreme cases.

> Furthermore I don't think the pkg_* tools are slow. They are quite fast
> IMO. If you let pkg_info print the entire list of installed ports it's
> only slow because of your line-buffered console. Just redirect the
> output to a file and you'll see that it's blazing fast. If I compare it

Err, nope. :) I see Eric has provided the numbers.

> for example to Debians apt-get/apt-cache commands it's much faster.

AFAIK Debian has the same dichotomy FreeBSD has: a tree of text files
used by dpkg, and a binary database (cache) used by apt.

> portupgrade is very slow, that's true. First of all it's written in Ruby
> which is not one of the fastest languages but there is another thing
> that slows it down considerably, which is rebuilding its database.

In my (limited) experience, this sort of task should not depend much on
the speed of the language. The most CPU-intensive task portupgrade does
is resolving dependencies, and on a running system this is a DAG forest
of about 500 nodes. I know portupgrade has some highly unoptimal code in
it (if I understand the code correctly, there's a brute force check for
cyclic dependancies in it!), but still, in itself, I think the choice of
Ruby isn't performance-critical.

> Furthermore I think it would be a very bad idea to include sqlite in
> base. There is already a lot of third party stuff in base. The
> philosophy of the BSDs is to provide and maintain an entire OS. This is
> quite the opposite of how a GNU/Linux system is designed. Both ways have
> their pros and cons. An advantage of the BSD way of doing things is that
> the developers know the code very well and have control over the quality
> of the code. If you include 3rd party software into the FreeBSD base
> system you make the FreeBSD project depend on the people that wrote that
> code. Of course you could fork it but the FreeBSD developers are not
> necessarily familiar with the code. Security patches would have to be
> merged all the time and a lot of communication between the two projects
> is needed.

I think this line of reasoning was made invalid by the continuing
inclusion of sendmail, bind, expat (xml parser!), etc.

Not that I don't realize this increases the burden on maintainability,
but including a "frozen" branch of a library, which is supported, but
won't be changed for ages isn't going to increase it much.

Offloading much of the "smarts" to a database would also permit easier
reimplementation of portupgrade-like tool in C, since the heavy parsing
/ regex facilities scripting languages offer won't be used as much.

But yes, it's a heavy departure from "the unix way".

> I think the best way to go would be to use only folder hierarchies and
> text files and write a libary in C that provides portupgrade
> functionality. The code under src/usr.sbin/pkg_install/lib/ would be a
> good base for this. Then you could use a frontend program that makes use
> of this library. This frontend could be a CLI program or a GUI based
> program.

The issue in this thread (at least for me) is performance and
reliability, and creating a C wrapper around the current situation won't
solve neither.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 250 bytes
Desc: OpenPGP digital signature
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20070512/3de80f52/signature.pgp