what next for the pkg_install rewrite

[Julien Laffaye]

On Fri, Aug 20, 2010 at 12:00 PM, Ivan Voras <ivoras at freebsd.org> wrote:

>
> And... both ideas are completely wrong. SQLite can be imported as a
> single C file + header, which you must agree is practically the
> optimum, and its license is "public domain" which is, if anything,
> "freer" than BSDL and eminently compatible with it.
>
> And if we don't want to build a sqlite.so which can conflict with the
version from ports, we can statically link libpkg to it. Incognito.


> BDB in base offer exactly one (1) single "feature", if you can call it
> that: storing and retrieving key-value binary blobs. It has no
> practical concurrency support, it's locking is laughable and upgrading
> data stored as binary blobs is even more nightmarish than maintaining
> the current plist format (and it will lead to similar uglyness soon -
> rather than upgrading the data piece called "x", I'm sure developers
> will introduce new keys called "x_extdata", "x_moredata" etc etc).
>

If we are going key-value storage, I think that TinyCDB is worth a look.


>
> SQLite on the other hand solves all these in the following way:
>
> 1) stores proper records, not blobs
> 2) has proper locking & concurrency support, ACID
> 3) the database schema can be modified on the fly for upgrading -
> fields can be added to existing table while preserving data.
> 4) is endian-agnostic, 32/64-bit agnostic and portable (I mean the
> database file) to an extremely large number of platforms. It is
> already used as a system database in OS X, iPhone and Android.
>
> Note that I'm promoting SQLite to replace the /var/db/pkg/* tree of
> directories and files with ad-hoc structure - all data in there should
> be in one SQLite database, which is stored in a single file.
>

Indeed, all this points can help. The major drawback of SQLite, in my
opinion, is the inclusion of long SQL statement in c code. So if we are
going this way, we must have strict rules to avoid "polluting" the source.


> [...]
>
> As for backward compatibility: basically it can be done in two ways:
> 1) build a one-time converter that will read the present plist
> database and output a new database or 2) build a compatibility layer
> which would support both the old and the new format at the same time,
> so people upgrading will continue to use their old data. I consider
> the latter wasteful in terms of developer resources and because
> bit-rot will eventually make support for the old format decrepit.
>

Agreed, a one time converter seems to be the solution. Especially if the
transition occur between two major FreeBSD version (say, 9 - > 10).


>
> > 2. XML is a bad idea. Great in theory, wonderful in my browser, but a
> > bloated plaintext file with a lot of complexity. I would prefer a
> > database solution that could be dumped to a simple text file format if
> > needed.
>
> XML, at least in my proposal, would not be used for the system package
> database, but would replace (either in part or all with a single file)
> today's "+" files in the packages, together with other purposes where
> some metadata transfer is required.
>
> That said, I would also be happy with other self-described formats
> like JSON. XML is the front runner because it's more standard
> (industry standard, whether someone likes this fact or not!) and there
> is already a parser in base.
>

The pro of XML here is that we have a parser in base, The con is its
verbosity. But anyway the manifest is not meant to be human readable.
Nevertheless, I prefer JSON, because it's less complex than XML and we can
have a parser/emiter in a single 500SLOC .c file.
And it looks sexier than XML, eh : http://pastebin.com/8hzPSSJC