FreeBSD problems and preliminary ways to solve
Adrian Chadd
adrian at freebsd.org
Tue Aug 23 06:49:54 UTC 2011
And that's been my experience too.
Linux users and advocates are very .. loud. FreeBSD users and
advocates tend not to be so loud.
Maybe that needs to change.
Adrian
On 23 August 2011 14:06, selven <pcthegreat at gmail.com> wrote:
> Good post as well as pretty much scary. You mentionned about the purity of
> FreeBSD at some point, that's one of the reasons why i prefer FreeBSD
> (probably many of my friends who do work with it also), lately, with all
> those ubuntu fans, trying to get the company to keep using FreeBSD has
> become pretty much a hassle. Since in a technical board, with 10 people, 7
> out of 10 are ubuntu users, with votes one always loses, and the funny thing
> :s 7 out of 10 don't even know the difference between FreeBSD and linux,
> they just voice out ubuntu because 'you will end up in trouble finding
> people to maintain your stuffs if these 3 leaves' (weirdly we even fixes
> there buntu bugs :s ). This user base is really a huge problem.
>
> On Thu, Aug 18, 2011 at 3:10 AM, Vadim Goncharov <vadim_nuclight at mail.ru>wrote:
>
>> Hi,
>>
>> I have bad news.
>>
>> A month ago techlead of Rambler's Mail department declared in his blog
>> that they begin migration from FreeBSD to Debian/Ubuntu. Comments to the
>> blog entry (there was much flame) revealed that search department of
>> Russian search engine #1 Yandex.com also plans to migrate their
>> search cluster - about 30,000 servers in several DCs (~60% of total
>> their servers - to Linux. The problem here is that both big companies
>> were using FreeBSD from the beginning (~1997), so migration will be
>> rather expensive.
>>
>> The official reasons (really semi-official, as these are individual
>> blogs), in short, were:
>> * inadequate package manager and huge monolithic base system
>> * lack of OpenVZ-like virtualization (need CPU limiting)
>> * a FreeBSD marketshare forecast for 5 years
>>
>> Despite of several our committers working for them, the operating
>> expenses for FreeBSD are considered too high by these companies. They've
>> not confirmed, but the key problem is probably shortage of FreeBSD
>> specialists on the market to hire (e.g. Yandex has about 20 FreeBSD
>> admins and about 84 Linux admins, for all other their services and
>> 40% of servers). So this is problem of FreeBSD's too low
>> userbase/marketshare, caused, in turn, by other FreeBSD's problems.
>>
>> Given that they is just planning today, but not yet started, we have to
>> solve our problems or FreeBSD will effectively die[1]. Currently
>> https://ssl.netcraft.com/ssl-sample-report//CMatch/oscnt_all shows that
>> still 3% of servers are running FreeBSD. We are currently probably at
>> the tip of 'hype curve'[2] and then our userbase will begin to shrink,
>> and the task is to solve problems, so after hard time it will rise again
>> (hopefully more than it was before).
>>
>> [1] 'Die' means shrinking userbase size to those of NetBSD/OpenBSD.
>> [2] http://www.metodolog.ru/01493/2.gif
>>
>> Earlier this year in this list marcel@ have said about objective view:
>> http://lists.freebsd.org/pipermail/freebsd-arch/2011-January/010999.html
>> from the outside, not FreeBSD people. I have posted a call to Russian
>> community in my blog (http://nuclight.livejournal.com/128712.html) and
>> gathered a list of problems along with possible solutions to some of
>> them. The rest of this message is summary of them.
>>
>> In general, people sympathizing FreeBSD agree that there are no fatal
>> technical problems and that FreeBSD could compete with mainstream Linux.
>> That said, the 80/20 principle is in effect and 80% losed users for
>> FreeBSD are caused by 20% problems. The trouble here is that what
>> "outside" people view as considerable problems, we here inside FreeBSD
>> view as something insignificant ("do it yourself"). Thus 80% could be
>> achieved by 20% of work. I have sorted them in order of decreased
>> importance (priority):
>>
>> 1. Social problems of community (and marketing, docs, ...)
>> 2. Lack of drivers and virtualization.
>> 3. Ports and packages.
>> 4. Base system, closely related with packages.
>> 5. Too short major releases' cycle.
>> 6. Bug tracker, unicode and other less important trivia.
>> (bonus) FreeBSD strengths to be concentrated on.
>>
>> Now all of them more detailed.
>>
>> > 1. Social (psychologic) problems of community (marketing, docs, ...).
>>
>> This is the most important one, because all technical problems are just
>> won't get solved because are even not viewed as problems. The FreeBSD
>> Project does not listen to users' needs. The typical response when poor
>> user want something is: "we don't need this, we won't change for you",
>> with "where are your patches?" at best. Then many users go out when see
>> such attitude toward them.
>>
>> The key points are:
>>
>> 1) *The competent user is not zealot*.
>> 2) The system is *for users, not for developers*.
>>
>> With first: when user sees the system costs too much time for him, and
>> that those problems won't get fixed and even considered such - he will
>> switch to another OS. This may mean that he is able to follow our advice
>> hoe to do some or another thing (e.g. to recompile all ports), but this
>> is unacceptable to him because this is too much maintenance (another
>> system by objectivity requires less work). Many businesses fall into
>> this category. They will not with FreeBSD by any price just because they
>> love OS. Unlike ourselves.
>>
>> With second: that's simple and not simple. It is simple in that by
>> nature each person don't make a work just for himself but for all
>> others, who are now "users" to him, too - just by induction that's not
>> for developers only. It is not simple due to question "Why do we need
>> those who don't contribute anything back to us?" which random committer
>> has right to ask. The answer, in short, is: there tasks which could be
>> only done by large groups of people, e.g. big corporations. For
>> corporations to invest and contribute back - the system must be popular
>> enough. To be popular enough means there must be a big amount of users
>> who is not contributing anything. It is useful in itself, though, that
>> some of those users, say 1%, will become contributors, that is, absolute
>> number of our developers will also benefit from FreeBSD being much more
>> popular.
>>
>> There are opinions in our community like:
>>
>> | > Personally I'd like to think that that we write an OS for users.
>> |
>> | We write an OS for the people who can and will use an OS written by
>> | us.
>>
>> | FreeBSD is whatever its developers make it be
>>
>> These are wrong and harmful nowadays. The world has changed. We have to
>> admit it or we will die. We have to admit mistakes and *change* some of
>> the ways we're doing things: any answer like "I don't agree, we don't
>> need these users/features/etc., we *won't* do this for someone" - is
>> just another step to grave.
>>
>> Of course that doesn't mean to do everything - we often have not enough
>> time/resources to do. But that's another question and should not be
>> mixed with attitude - "it's good but we have no resources, will you do?"
>> vs "we don't need this, go away" (that's not only about features but
>> about keeping something too).
>>
>> So ("system is for users") we need to (re)define who our user is. I view
>> the current situation as following:
>>
>> 100% .---------------. The stratification of overall
>> | Ubuntu | our userbase. Areas are:
>> | target | 6
>> | audience | 1 - official developers
>> expand -> |---------------|
>> to here | not our, but | 2 - actively participating and
>> | competent | 5 sending patches, contributors
>> | users | and other "zealots" (whose
>> |===============| "soul is with FreeBSD")
>> |announce@ only | 4
>> involved |---------------| 3 - not so active but discussing
>> | subscribed to | in mail lists, patches go
>> | our mailing | 3 sometimes
>> | lists |
>> committed |- - - - - - - -| 4 - busy with work to read lists
>> | | 2
>> |_______________| 5 - can use, but costs are now high
>> | *@freebsd.org | 1
>> 0% `---------------' 6 - we'll never want them
>>
>> The terms "involved" and "committed" were taken from
>> http://en.wikipedia.org/wiki/The_Chicken_and_the_Pig for those who are
>> "at the heart" of the community, at "periphery" and all others.
>> Currently, only those in (1) and partially (2) are listened to,
>> sometimes with "call for ..." in (3). Meanwhile, the most important of
>> our users, who are using FreeBSD in production, who are busy with real
>> work in real world, are in (4) and sometimes (5). Those are largely
>> ignored by the Project - even messages to announce@ last years are less
>> about important events and calls for volunteers/money.
>>
>> The other social problem is lack of companies which offer commercial
>> support of FreeBSD like RedHat does.
>>
>> As a possible solution to find out what user needs are, and to
>> compensate the fact that not all users in (4) and (5) care enough about
>> our principles and spirit, I propose a system for "weighted democracy".
>> This is a voting system (I could implement it), say, there is a mail
>> voting at freebsd.org to which users send filled in vote forms, with
>> selected answers from a survey published in announce at .
>>
>> The system has a database where users are recorded given their FreeBSD
>> activity in mail lists etc., and votes are summed as follows:
>>
>> + 1.00 vote for anyone not in database (not involved to FreeBSD)
>> + a decimal logarithm of number of posts to mail lists (2 votes for 100
>> posts, 3 votes for 1000 posts)
>> + a binary logarithm of num PRs in GNATS db (6 votes for 64 send-pr's)
>> + a proportion (say, N/2) of entries for this person in "svn log" (e.g.
>> in "Submitted by:")
>> + an assigned (by core@) number of votes in special exceptional DB (for
>> corporation and the like)
>>
>> The system then presents results for each answer: 1) how many users
>> voted, 2) how many votes summed.
>>
>> This is by no mean to measure "exact contribution", but to defend from
>> anonymous users and trolls who not care about amount of work developers
>> will need to. The results are viewed as a feedback to core team, not as
>> a final decision - that's all for what marketing is solely exist. There
>> no adequate feedback from users to developers currently at all -
>> individual posts in mail lists are too small for statistics (what about
>> ministat(1) for users, huh?..).
>>
>> What is also in this part? Documentation. No, not in that sense. The
>> observations show that while there are described solutions to problems
>> in system, it's difficult for users to find it. It's somehow organized
>> or misorganized that solution is not intuitive for them (e.g. someone
>> migrating from Windows complained that it was difficult to find 'simply
>> how to format a flash drive' in Handbook). I don't know how to handle
>> this properly. May be catalogues with Best Practices howto's pointing to
>> exact sections of Handbook, FAQ, articles etc.? May be better search
>> mechanisms?..
>>
>> And the last here about too much conservatism and rigidness in our camp.
>> As one of the opponents said (roughly translated):
>>
>> | FreeBSD, historically, was good system, let's say, "valid system
>> | for solid people". Pureness, strictness, etc. The base system is
>> | consequence of this approach. The trouble is, that in 2011 nobody
>> | needs pureness and strictness. Customers need transparency and fast
>> | reaction to their requests. If there is no transparency and speed,
>> | you get CentOS 6 (just released, at last). Or FreeBSD (Gentoo,
>> | Solaris, you name it).
>>
>> I don't agree that those certainly contradicts each other, but we
>> definitely need to change, er, something. And don't keep something just
>> because it is tradition, if that tradition has no more technical reasons
>> for our users. Let's shift conservatism to another field where it is
>> really needed (e.g. legacy support and other examples below).
>>
>> Finally, two more quotes from the arch at freebsd.org archives of last
>> 2 years:
>>
>> | From: Cyrille Lefevre <cyrille.lefevre-lists at laposte.net>
>> | [...] however, you answer remember me why I quit FreeBSD, cease
>> | fire, courtesy is the key
>>
>> and
>>
>> | From: Julian H. Stacey
>> | Though I & many others have sent lots of send-pr, contributing even
>> | a spelling correction to FreeBSD is much harder than to e.g.
>> | http://wikipedia.org
>>
>> The threshold of entering to FreeBSD is too high and should be lowered.
>> I can confirm the last quote on my own example, 2 months ago. I've
>> submitted a patch to ipfw, adding call/return rule actions. It allows
>> to organize rules in somewhat like pf anchors, and, more importantly,
>> iptables chains, enabling FreeBSD ipfw to compete with Linux at least
>> partially in this sphere. The patch wasn't taken in maillists, I had
>> to push it in IRC, and push hard: the whole needed by many big users
>> thing was deferred (and still not MFCed to 8.x) just because some
>> #define's and printf's were Not So Proper & Right Way! F*cking shame.
>>
>> > 2. Lack of drivers and poor guest virtualization.
>>
>> It is known that FreeBSD supports less hardware than competitors. It is,
>> however, a matter of popularity - the more system will have users, the
>> more drivers will be created by 3rd parties, so we cannot directly
>> affect this. It's kind of exclusive circle.
>>
>> But there is one area - virtualization - without supporting it the
>> FreeBSD niche will collapse very fast. It is necessary to say that it is
>> about guest (para-)virtualization only, not host mode. Host mode market
>> is already lost for FreeBSD for quite some time, may be something in the
>> future, but today it will waste of resources. What is needed here is
>> just analogue to OpenVZ (and jails/rctl already done more than half of
>> the work here).
>>
>> And in guest mode FreeBSD *urgently needs* working drivers/utilities for
>> all common suits, Citrix XenServer, VMWare vSphere (ESXi), Xen*,
>> Microsoft Hyper-V, etc. This must be the primary direction for
>> developers' forces and FreeBSD Foundation's money. We have may be about
>> 1 year here. Why? Because of cloud computing and VPS/VDS. It's no
>> matter what OS runs hoster if clients will use FreeBSD and we still have
>> users. I've been already asked by a customer for which I wrote
>> a non-portable kqueue-based daemon to rewrite for Linux because they
>> want to go for Amazon EC2 (it's cheaper as only used resources are
>> accounted) and FreeBSD there has still many in ISSUES scaring them...
>>
>> > 3. Ports and packages.
>>
>> What was the main problems with large-scale installations of FreeBSD in
>> that businesses? In short, that binary packages are not equal in rights
>> to ports, and that complicates things (i.e. requires too much work) when
>> one have many (> 10) servers. This was listed to me as:
>>
>> 1) No pkg and pkg-devel versions. The -devel version is headers, static
>> libs, programmer examples, etc. not needed in production (we could
>> say this part is what is actually depended on in B-deps).
>>
>> 2) Package name is dependent on options, so packages with another opts
>> don't work well when dependencies are rebuilt.
>>
>> 3) Conflicts: no way to have apache13 and apache22 the same time.
>>
>> 4) No dependence on base system. You may cut out something, recompile
>> world, deploy it on cluster and just then see that some packages are
>> now don't work.
>>
>> 5) Dependencies are badly designed. No version ranges in dependencies,
>> no alternative packages, no priorities in package search.
>>
>> 6) Update problems. The version is just coded into name of package, and
>> dependencies are on the entire name, so there are situations when
>> install/upgrade of just one package may require rebuild 3/4 of all
>> pkgs. You cannot easiy modify installed package without editing pkgdb
>> manually. It is impossible to upgrade/replace package by out of the
>> box tools.
>>
>> 7) Base system has no "out of the box" tools for package upgrade. Our
>> business opponents say this the least problem as one can always
>> install portupgrade, but conclude that overall base system concept
>> does not play well with full-featured packages (see also next part
>> about base system).
>>
>> 8) There is no -STABLE supported branches in ports.
>>
>> All of this could be avoided (they know about tinderbox etc.) but just
>> requires too much work, for their basic tasks like automated upgrade of
>> entire system & packages or reinstall of needed packages.
>>
>> That's problems. Next, possible (gathered) solutions.
>>
>> It is obvious that current packages are not first-class citizens, in
>> comparison with ports. They want ba able to run most machines without
>> a compiling at all (BTW, our desktop users need the same), but setting
>> build farms when there are many machine roles is hard.
>>
>> So packages need to be "equal in rights" in ports. The ports can have
>> things like this:
>>
>> .if ${OSVERSION} < 700104 || ${OSVERSION} >= 900000
>>
>> or this:
>>
>> LIB_DEPENDS+= profiler.1:${PORTSDIR}/devel/google-perftools
>>
>> but packages are not so flexible, all you have is:
>>
>> @pkgdep perl-5.8.9_2
>>
>> skv@ proposed the following changes:
>>
>> * OPTIONS need radio-buttons (e.g. only one of MySQL, PostgreSQL,
>> SQLite) and dialog(1) supports it.
>>
>> * Options must be included and installed to /var/db/ports/*/options
>> (this will allow to rebuild installed binary pkg as port)
>>
>> * Info about options must be included to /var/db/pkg/*/+CONTENTS like:
>>
>> @option WITH_SSL
>> @option WITHOUT_DEBUG
>>
>> * Dependencies must be able to specify needed OPTIONS, both required to
>> set and required to be unset, somewaht like:
>>
>> RUN_DEPENDS+= foobar:${PORTSDIR}/devel/foobar:+SSL:-SQLITE
>>
>> This will allow to detect conflicts with installed packages with
>> incompatible options.
>>
>> * For the package file names, introduce presets, e.g.:
>>
>> OPTIONS_PRESETS= default "+SSL:-DEBUG" \
>> lite "-SSL:-DEBUG"
>>
>> And preset name could be put to pkg name (may be "" for default).
>>
>> * (internal) move away from CVS, rebalance to category-subcategory.
>>
>> These ideas in later discussion evolved to another additions. Let say we
>> are able to use multiple repositories, where "repository" is a variant
>> of /usr/ports tree and packages built from it. Then, each port allows to
>> build several packages from it, with different options. Now, if we have
>> a port called "softina" and user does
>>
>> pkg_add -r softina
>>
>> then dependency search must be made given needed options. So @pkgdep
>> consists just of "softina" and versions like ">=1.1" and options. Also,
>> if packages are equal in rights to ports, they need integrity/security
>> check. So, package file name is now like:
>>
>> softina-1.2.3_1:repo.id:1312929890.tbz # chars allowed by windows
>>
>> Here is a repository id, just a hostname like "freebsd.org" for official
>> ports, and an unique build id in that repo, though it were suggested
>> that option preset name instead of that name will be better (because
>> human-readable). And `pkg_add -r' fetches a single file
>>
>> softina.pkgmeta
>>
>> consisting of sections for each actual package file. Each section has
>> a copy of what already is in .tbz file: name, comment, options,
>> dependencies (and their versions and options). And one thing not present
>> in .tbz - it's digital signature. Fetching pkg_add looks up local key
>> for given repository id to check. Fetching .pkgmeta beforehand also
>> allows to calculate if all needed dependencies are present in repo to
>> not fail in the middle of 100 packages as current pkg_add may do. The
>> signature is in another file and optional, so user could install the
>> plain .tbz file manually (it still contains all needed information,
>> .pkgmeta is only a copy except a signature).
>>
>> The one other thing which is also optional to @pkgdep is again
>> repository id. This is to allow following situation:
>>
>> * company has 10 it's own internal projects
>> * it also has 20 modified ports from original tree
>> * internal pkgs depend on modified ports, not original
>> * it's local port tree/pkg repo may hand only those, not full tree
>>
>> Then internal pkgs may be depended on modified ports to be not
>> intermixed by mistake with original versions from official tree.
>> Repository IDs are used for that, this is optional mechanism, though.
>> End machines have two repositories in their config files, local and
>> offical, and all works correct, and local repo doesn't need to be a full
>> clone of ports tree.
>>
>> The problem of -devel ports could be solved by using a global knob like
>> WITH_DEVEL (analogue of WITHOUT_X11), and options affect parts of
>> pkg-plist. The solved problem: glib has perl in R-deps, just for one
>> script, but this script is needed only for _build_ of dependent ports.
>> Now, if you install irssi, you will always have glib and perl, but you
>> don't need perl to run irssi. And irssi could have it's build dependence
>> of glib:+DEVEL and run of just plain glib.
>>
>> Of course we don't want to split ports to perl, perl-base, perl-modules,
>> perl-doc, etc. like in Debian, and options could be solution here - one
>> port, several packages. Build farm may now build not one, but 2-3 common
>> option presets.
>>
>> Here we go to another related problem, though - ports infrastructure.
>> I've read 16 chapters of http://www.netbsd.org/docs/pkgsrc/ and found
>> several interesting moments (let's concentrate on most close sibling of
>> our ports, though e.g. slots in Gentoo and Arch Linux system are also
>> worth looking too). They already have many of what we need, and since
>> pkg_install/ports by Jordan Hubbard is common ancestor, it may be easier
>> to port from them to us needed features.
>>
>> For example, there is problem generating plist for our porters. And what
>> they have, a program to create port from distfile:
>>
>> | Run the program url2pkg, which will ask you for a URL. Enter
>> | the URL of the distribution file (in most cases a .tar.gz file) and
>> | watch how the basic ingredients of your package are created
>> | automatically.
>>
>> Just fix oddities later manually, but saves from all boilerplate work!
>>
>> Next, skv@ said about radio-buttons, and thay also have it, in two
>> favors: first with one from group always set, second allowin all clear:
>> "Exactly one of the following gecko options is required"
>> "At most one of the following database options may be selected"
>>
>> | PKG_OPTIONS_REQUIRED_GROUPS is like PKG_OPTIONS_OPTIONAL_GROUPS, but
>> | building the packages will fail if no option from the group is
>> | selected. PKG_OPTIONS_NONEMPTY_SETS is a list of names of sets of
>> | options. At least one option from each set must be selected.
>>
>> The OPTIONS, however, are handled in different way:
>>
>> $ grep "PKG.*OPTION" mk.conf
>> PKG_DEFAULT_OPTIONS= -arts -dvdread -esound
>> PKG_OPTIONS.kdebase= debug -sasl
>> PKG_OPTIONS.apache= suexec
>>
>> Dependencies also allow versions:
>>
>> BUILD_DEPENDS+= lua>=5.0:../../lang/lua
>> DEPENDS+= screen-[0-9]*:../../misc/screen
>> DEPENDS+= screen>=4.0:../../misc/screen
>>
>> Checksum files for packages exist on their own, and only those may be
>> signed not the packages itself (an alternative to .pkgmeta approach
>> described above).
>>
>> Another useful thing to borrow is patches/* separately from files/*
>>
>> The most visible thing in pkgsrc superior to ports, is buildlink3
>> framework:
>>
>> | Buildlink is a framework in pkgsrc that controls what headers and
>> | libraries are seen by a package's configure and build processes.
>> | [...] Please note that the normal system header and library paths,
>> | e.g. /usr/include, /usr/lib, etc., are always searched -- buildlink3
>> | is designed to insulate the package build from non-system-supplied
>> | software.
>> | [...]
>> | Some packages in pkgsrc install headers and libraries that coincide
>> | with headers and libraries present in the base system. Aside from
>> | a buildlink3.mk file, these packages should also include a
>> | builtin.mk file that includes the necessary checks to decide whether
>> | using the built-in software or the pkgsrc software is appropriate.
>> | [...] When building packages, it's possible to choose whether to set
>> | a global preference for using either the built-in (native) version
>> | or the pkgsrc version of software to satisfy a dependency. This is
>> | controlled by setting PREFER_PKGSRC and PREFER_NATIVE.
>> | PREFER_PKGSRC= yes
>> | PREFER_NATIVE= getopt skey tcp_wrappers
>>
>> There are more automation:
>>
>> | Up to now, the file PLIST, which contains a list of the files that
>> | are installed by the package, is nearly empty. Run
>> | bmake print-PLIST >PLIST
>> | to generate a probably correct list.
>>
>> Yes, they rely on their derivative of BSD make, bmake, which it itself
>> worth merging deifferencies to our make (for example, there is a clean
>> and small alternative to autotools, mk-configure, using bmake and
>> written in style of .include <bsd.prog.mk>). There may be other examples
>> of userland tools worth porting from NetBSD to us instead of directly
>> upstream, e.g. awk... but that's another story, let's continue pkg.
>>
>> Their developments of pkg_* tools contain facilities to implement a good
>> package manager out-of-the-box, e.g. pkg_add there has flags:
>>
>> | -A Mark package as installed automatically, as dependency of
>> | another package. You can use
>> | pkg_admin set automatic=YES
>> | to mark packages this way after installation, and
>> | pkg_admin unset automatic
>> | to remove the mark. If you pkg_add a package without
>> | specifying -A after it had already been automatically
>> | installed, the mark is removed.
>> | -U Replace an already installed version from a package.
>> | Implies -u.
>> | -u If the package that's being installed is already installed,
>> | an update is performed.
>>
>> These are crucial to effective binary package management.
>>
>> > 4. Base system, closely related with packages.
>>
>> The next in turn was concept of monolithic base system. The troubles
>> with base system were:
>>
>> 1. To change the components of the base you may only recompile it with
>> corresponding src.conf(5) options, but there is no track in base
>> system which are actually installed now. You cannot binary upgrade
>> a custom world, even if it is just unmodified -STABLE instead of
>> -RELEASE.
>>
>> 2. Consequently, there is no way to check integrity (MD5 etc.) of any
>> non-RELEASE variant (freebsd-update IDS is very limited).
>>
>> 3. No ties between base system and packages: who knows what previous
>> admin has installed, you may have compiler or may not have, etc.
>> Packages may silently broke if some part of base system SUDDENLY
>> disappears, as no dependency information is recorded.
>>
>> 4. Base system is monolithic, so there is no easy way to replace one
>> component with another - ports replacing base parts are hemorrhoids.
>>
>> So, they conclude, the base system concept should be eliminated and be
>> split to bare kernel and gazillion of packages.
>>
>> But we all know what benefits base system gives to us: it is actually
>> a *system* where all components are in concordance to each other, not
>> just a bunch of packages. Also, if it will be all split, this will
>> require *much more* efforts from our committers to keep everything in
>> sync. And our resources (efforts) are limited.
>>
>> Actually, when you face two contradictory ways, all-or-nothing, both
>> with pros and cons - you should select neither of them, but try to find
>> something which will *synthesize* both of them. Such finding is an
>> interesting engineering (often inventor) task. And dougb@ already said
>> once that between base as-is and splitting all must be something middle.
>> It's Cathedral vs Bazaar, after all, and our Cathedral should be updated
>> and still use it's cathedral benefits. After all, NetBSD has proposal of
>> syspkg in 2004, and still has older format - just because it is not BSD
>> way, something new has to be invented.
>>
>> The most obvious solution is to split base not to ~500 packages, but to,
>> say, ~50, and change boundaries. Why should freebsd.org developers place
>> boundaries in packages between themselves? So most natural solution is
>> to split packages out of base by the vendor criteria. Luckily, this work
>> is *already done* to some extent. So this will be minimal efforts
>> overhead, if any.
>>
>> Looking at /usr/src/sontrib and http://wiki.freebsd.org/ContribSoftware
>> we can identify many of what could became a package. There can be
>> different approaches to criteria "what is in base system":
>>
>> 1. Only what is done on freebsd.org: all contrib must go to pkgs.
>>
>> 2. Whose effective vendor is now *@*bsd.org: contrib from other BSDs may
>> still live, and those with ceased upstream or renamed
>> (non-conflicting with ports) soft like libbsdxml, too.
>>
>> 3. Axe out only the most odious contrib parts: BIND (as Peter said in
>> archives, host/dig could be resurrected from Attic), sendmail (could
>> be replaced by dma) and several others, presumably GCC/binutils & CVS
>> (I've also heard about painful Kereberos interferencing with Samba).
>>
>> The latter is most probable :-) given known conservatism and parts
>> inter-dependencies (openssl is required by freebsd-update, and it's not
>> *@*bsd.org - already an exception). But it still needs clear definition
>> of what base system is destined for.
>>
>> A philosophical observation. The more "fat" base system is, the more
>> things are supposed about end user (the "WHEREs" in "SELECT"), so the
>> more narrow target audience is. The more thin base system is, the more
>> popular OS can be. You elitists may sleep well: FreeBSD will never be
>> as popular of Linux whose "rod" is just kernel, not base sys.
>>
>> So each component of the base must be justified for the task, and for
>> task we may be should define to which user FreeBSD is targeted. Let it
>> be, say, task "the fresh setuped machine must be able to connect to
>> network to download and install binary packages". Then you don't need
>> GCC but you need editor to edit resolv.conf, less to view man pages,
>> tcpdump to debug network problems, send-pr to report a bug about
>> non-installed packages and thus a mail agent able to handle outgoing
>> report mail and daily periodic scripts to root while you have sex with
>> this (dma is suitable, sendmail is overkill). And so on.
>>
>> Axing out GCC to packages has another benefit: the newer GCC could be
>> used, and base could be polished to be more compiler-agnostic (hello,
>> clang). But this requires binary packages to have equal rights with
>> ports, of course. A base without a compiler is not a dramatic - you
>> already need some ports to do "make release", and doc team already began
>> to split docs to packages, that's a right direction. For POLA reasons
>> all axed out packages (and sendmail too, respect traditions) should be
>> just packaged on CD1.
>>
>> There may be another approach, not package one, but it is still in a fog
>> (and don't solve ports overriding base well). It's like the kernel and
>> modules: monolithic base system is just as monolithic kernel before
>> invention of modules - you the same way don't know what's in. Let's
>> dream a little - imagine we have our own DVCS, combining benefits of
>> centralized and distributed one, and not requiring splitting to multiple
>> repos like git, but allowing to use any given subset of files (not dirs
>> and subtrees as svn, *any* "inode"-based subset) from central repo
>> (this one "more equal" from all equal distributed repos). Such system
>> could be placed instead freebsd-update, binary updates of STABLE etc.
>> go to special branch, and user updates only those files which he have
>> on his system, VCS automatically tracks it.
>>
>> > 5. Too short major releases' cycle.
>>
>> I've once read a thread:
>>
>> From: mdf at FreeBSD.org
>> Subject: Schedule for releases
>> Date: Tue, 21 Dec 2010 09:47:08 -0800
>>
>> where e.g. julian@ have said:
>>
>> | Generally a company wouldn't want to go through the pain of an OS
>> | upgrade more than about once in three years and often it's longer..
>> | It IS a pain for them.
>>
>> And many business people reported the same. And it is known many people
>> are doing backports, testing etc., they should be motivated to give work
>> back to FreeBSD. While it is more social problem, it is also a
>> DVCS/bugtracker problem, but more rare major releases would still help
>> on it's own.
>>
>> It is known why the current scheme was adopted: the 4.11/5.3 case,
>> a horror. But between X.4 and X.11 there are even *several* intermediate
>> choices. What happens today: many conservative users (or builders of
>> products) consider -STABLE is really stable at the X.2 release. But just
>> after than an (X+1).0 is forked and all developers' attention goes to
>> new branch, X.3 and X.4 are not seriously developed. And due to
>> timelines described above, many users will then upgrade right away to
>> (X+2).Y, not (X+1). There are many 6.x and 7.x users in the wild, many
>> of 7.x users will upgrade directly to 9.x skipping 8.x. Is this good?
>> No.
>>
>> Aside of many branches receive not enought production testing, our
>> committers must do MFCs to TWO branches, stable/7 and stable/8. While
>> usualy having no real possibility to test properly. Nonsense.
>>
>> Is supporting two stable branches not using more efforts that it was in
>> 4.x times? Not so? Still could be made better.
>>
>> Proposed solution: prolonging major releases fork time a little. Just to
>> time so only one stable branch will exist. I hope increasing branch
>> lifetime to one minor release will help: last will be not .4, but .5,
>> and new .0 fork should occur at least after .3, not .2. We are not
>> Ubuntu. I understand that pace of changes is high, 3 years for each .0
>> may be unacceptable, but waht about at least 2.5 years? Not like 1.8
>> years currently. Rememeber, .5 and even .6 is still far from .11.
>>
>> > 6. Bug tracker, unicode and other less important trivia.
>>
>> GNATS is too old and unfriendly e.g. to user attachments. It has only
>> one adavantage: user is able to send report from CLI by mail without any
>> registration. This is essential. May be the good alternative will be RT
>> used at cpan.org (it also accepts by mail). Hopefully this will help
>> a little to solving problems. Not sure, though - and unsolved bugs are
>> also make users to go away from FreeBSD.
>>
>> Users also want properly working UTF-8 out-of-the-box. Not only console,
>> but full collation support, etc.
>>
>> There was suggestion about automatic kldloading of drivers, but this
>> work already began (for USB) in June - more matte of our documentation
>> and press relations, huh.
>>
>> Installer. Must be more featureful and more user friendly :-) This is
>> often may give negative first impression if installer was unable to do
>> something even if system after this could work well.
>>
>> Other problems, like broken cvsup, are exist, but are not critical to
>> Project's survival.
>>
>> > FreeBSD strengths to be concentrated on.
>>
>> The paragraph about virtualization above, as long as points below, are
>> roughly translated words of one small business representative in Russia,
>> who often acts as integrator.
>>
>> 1. BSD License. Very good for embedded vendors, we should be able to
>> compile and work both base and ports without licenses requiring to
>> open the sources. It would be good to not loose functionality without
>> them.
>>
>> 2. Kernel features for storage (ZFS, HAST, GEOM). We need to go to
>> NAS/SAN solutions niche while we have ability - it is still, because
>> Solaris etc. in unknown state with Oracle, BTRFS still beta. We have
>> about 1 year max. It would be nice to roll out "box" solution for a
>> failover iSCSI storage (2 PCs with ZFS raids replicated by HAST and
>> accessible by iSCSI with automatic role change).
>>
>> 3. Kernel features for complex network solutions (netgraph, carp, ipfw).
>> The niche for routers & traffic analysis is still ours. It would be
>> nice to take e.g. pfSense and agree with some vendor (Netgear,
>> D-Link, etc) to put on sale hardware with FreeBSD inside.
>>
>> So these are main ways - embedded (NAS & routers). Other market is still
>> not our, e.g. not an application server until there will be a killer-app
>> for SCTP. May be also for a DB: Solaris was first recommended for
>> PostgreSQL, with some efforts we may become the first. Of course, the
>> main way - if some commercial company will offer FreeBSD.
>>
>> We still have some time, but almost no time. We need to take decisions
>> right now.
>>
>>
>> That's all for today. Thanks to everyone who has patience to carefully
>> read this all entirely.
>>
>> --
>> WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight at mail.ru
>> [Moderator of RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org
>> ][LJ:/nuclight]
>>
>> _______________________________________________
>> freebsd-arch at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-arch
>> To unsubscribe, send any mail to "freebsd-arch-unsubscribe at freebsd.org"
>>
>
>
>
> --
> *Pirabarlen Cheenaramen *| $3|v3n* *
> L'escalier, Mauritius
>
>
> /*memory is like prison*/
> (user==selven)?free(user):user=malloc(sizeof(brain));
> P Save electricity & disk space. Cat this mail to >/dev/null 2>&1 after use.
> _______________________________________________
> freebsd-arch at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-arch
> To unsubscribe, send any mail to "freebsd-arch-unsubscribe at freebsd.org"
>
More information about the freebsd-arch
mailing list