removing CVS in Handbook Updating and Upgrading chapter

Sun Jan 27 07:12:30 UTC 2013

Hi Warren, All,

I don't mean to exasperate you by pushing this thread, but this is a critical entry point for new FreeBSD users,

On Jan 26, 2013, at 9:05 PM, Warren Block wrote:
> On Sat, 26 Jan 2013, Isaac (.ike) Levy wrote:
>> On Jan 25, 2013, at 2:12 PM, Warren Block wrote:
>>>>> CVS is going away soon, and we should not be advising people to start using them now.
>>>>> This diff entirely removes cvsup, csup, and CVS references from the Updating and Upgrading chapter.  SVN URLs are also changed to the preferred form and links to the SVN mirrors are added.
>>>>> Rendered: http://www.wonkity.com/~wblock/temp/cuttingedge-nocvs.html
>>>>>  Diff: http://www.wonkity.com/~wblock/temp/cuttingedge-nocvs.diff
>> 
>> Regarding src, this appears to be jumping the gun quite a bit, with possibly bad consequences:
>> 
>> + ports, cvsup access end-of-lifed Feb 28
>>  (cool! drop cvs/cvsup references for it)
>> - source, cvsup deprecated - no end-of-life date set
>>  (until canonical replacement is in place)
>> 
>> Instead of replacing *all* CVS urls with SVN, I would like to advise you to merely make note of cvsup being deprecated for src?
> 
> Deprecation warnings are already in the current version, since November 17:
> http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/current-stable.html

I understand, and this deprecation warning is good.

cvsup is deprecated, but does not yet have an end-of-life date.
Additionally, this process currently has no analogy, and for base src, there is not an analogous canonical alternative path.

cvsup is currently the only canonical way on a REL or RELENG branch to fetch the sources, using only the base system.

However, Nov. 17 isn't very long ago, warning of a process change which people have used and relied on for well over a decade- the kind of users which represent the majority of the base of the FreeBSD project, (people running lots of servers).

>> Not sure if I need to explain this, but:
>> For a large number of system integrators, building userland/kernel from source is critical.
>> Most of these builds happen before ports/pkg get installed, (if they even do).  The current state of SVN, binary packages, ports mechanism changes, and otherwise- all make for some nasty chicken/egg problems for many systems integrators.
> 
> This part of the Handbook refers to fetching source for -CURRENT or -STABLE.  We should not be suggesting CVS to new users who want to run development versions of FreeBSD.  It's a misdirection, like a Perldoc-esque "here's an example, but you should never, ever do this".

I completely appreciate and understand this sort of nastiness.  However, the alternative is far worse for new users- a real-world example from last week,

> Existing CVS users already know how to use it, being existing and all.

Correct, but that's not really the point- cvsup is still the canonical way to build the system from sources, from a base system install.

> So the removal of CVS information from this chapter should not harm anyone already using it and should help by not steering newcomers to the wrong tool.

I think the disconnect here is this:
buildworld and buildkernel are not specifically developer tools, FreeBSD users build, and maintain, their systems from source- particularly the vast number of FreeBSD machines humming silently driving the internet at many layers.  Custom kernels are common for a myriad of reasons, sometimes to add features, sometimes to strip them down.  For performance, for security, for clarity.
Beyond the kernel, real-world security patches are often applied directly to production source builds, often in a hurry - 0-days can drop on anyone's head.

Because cvsup for base/src does not have a clear replacement right now, and is therefore does not have an end-of-life date set, cvsup is still the canonical user too.  csup(1) is even still in base.

--
For developers, I can agree that cvsup is totally the wrong tool- but that information should perhaps be in the developers handbook, in the "tools" section:
http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/tools.html

(If yall' think that's a good idea, I'll happily write up a page to start with, and send the patch within 36 hrs from now!)

However, this is what new users face if you make this change right now:

##############################################################################
OLD (current, deprecated) way:

1) # csup /path/to/ports-supfile
 (e.g. # csup -h cvsup14.us.freebsd.org /usr/share/examples/standard-supfile )
- note: canonical (but perhaps slowest) default server already in this config file

2) # move on to buildworld/buildkernel dance...

##############################################################################
NEW (work in progress, developers have cut over) way:

1) Install subversion (choose your own adventure)
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/svn.html

2) #noop# pkg_add -r subversion
3) #noop# pkg install devel/subversion
   (security incident cleanup, no binaries available)

                                        * <--+
?) # cd /usr/ports/devel/subversion          |
   # make install clean                      |
(whops, gotta fetch ports first right now)   |
# portsnap fetch ----------------------------+

Now, we just acquired this on our system too:
  + SQLite3 (not the SQLite in base
  + APR
  + Expat 2.x (not the Expat/libbsdxml now in base)
  + Neon or Serf
  + GNU gettext and libintl
  + libiconv

# APR could be a *big* problem if you're next steps are to load up a particular version of the Apache web server, (like a great number of FreeBSD users do)
Steps 5-?

?) dust off ctm(1), it's still in base- but will it work…
(dunno haven't thought about it in years)

?) svnsup is frequently discussed, (google confusion), but apparently not yet functional.
(this is a perfect idea, but not live.)

?) diving into portupgrade, a worse/heavier situation even:
  - depends on ruby (fatter dep tree than svn)
  - is a port (to manage ports)
  - has a small, loyal following (financial, even)

# svn checkout https://svn0.us-east.FreeBSD.org/base/head /usr/src
Error validating server certificate for 'https://svn0.us-east.freebsd.org:443':
 - The certificate is not issued by a trusted authority. Use the
   fingerprint to validate the certificate manually!
 - The certificate hostname does not match.
Certificate information:
 - Hostname: svnmir.nyi.FreeBSD.org
 - Valid: from Aug 12 23:01:31 2012 GMT until Aug 12 23:01:31 2013 GMT
 - Issuer: clusteradm, FreeBSD.org, (null), CA, US (clusteradm at FreeBSD.org)
 - Fingerprint: 06:D1:23:DE:5E:7A:F7:2B:7A:7E:74:95:5F:54:8D:5C:B0:D6:2E:8F
(R)eject, accept (t)emporarily or accept (p)permanently? 

Decide how to handle this SSL cert, (compare the Fingerprint manually until the dust settles with this new env):
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/svn-mirrors.html

N) # move on to buildworld/buildkernel dance…
##############################################################################

Now, those SVN problems appear to be getting worked out in several places:

+ subversion-static just hit ports (cool!)
Knocks an SVN version down to bare essentials- but this is a work in progress (ABI incompatibilities for static build?  Dependencies not *quite* flushed out clean yet?)

+ "svnsup" could emerge sooner than later
As a working in-base utility, a simple thing to fetch deltas using the svn/svn+http/svn+https protocols, using a tiny program in the base OS. 

+ actual svn infrastructure is stabilizing, growing, and coming into focus- (for developers even!)

- freebsd-update(8), I've been told, has some (currently broken) bits for fetching src for a given REL, but this may get expanded soon(?)
  - if true, it has some show-stopping bug [I'm unclear on what right now]?
  - man page states: "fetch and install binary updates to FreeBSD"
   (these are not the droids you are looking for, move along)
  - man page does not state "fetches base REL/RELENG sources trivially"
  - freebsd-update.conf(5) allows for commenting out all components except src

--
So are you guys absolutely certain that removing cvsup instructions, *just* for building from canonical sources, is appropriate?

Remember, this is for *users*, not developers.  But FreeBSD users typically want to use carp(4), or lagg(4), maybe dtrace(1M), perhaps zfs(8), or jail(8).
Perhaps they want to compile their public/private new C programs using clang(1).
Perhaps new FreeBSD users are evaluating performance for their Mail, Database, or Web infrastructure.  Perhaps new FreeBSD users are building new firewall/network appliances.  Perhaps they are doing embedded systems development, with FreeBSD as a platform.

Perhaps they are using no high-level languages aside from POSIX shell, (and for goodness sake FreeBSD sh(1) just got command history!!!!)

What more could a true UNIX lover want, other than FreeBSD?

Best,
.ike