Will Linux Ever Make it to the Desktop?
NB: When this article says “Linux,” it refers the whole gig, not just the kernel. Also, this is about Linux on the Desktop. Severs are a completely different thing. OK?
Almost every year someone declares it to be “The Year of the Linux Desktop.” Yet, these pundits are wrong–every year. Definitely, Linux has made a lot of progress since the days of Red Hat 6.0, but it still has major architectural problems that have existed since the beginning (and actually, in the pre-Linux days as well).
Will Linux ever make it to the desktop? For a long time, I was optimistic about this. Linux has a terrific community of smart people and companies built around it. With this community, we’ve built a pretty amazing free operating system that does work. Maybe not for everyone, but it works for you and me. I like Linux because it makes my computer think like I think. I like Linux because I can easily tell my computer to resize all my photos into thumbnails, to create a list of all the postal abbreviations in the US, and to replace a line in a bunch of PHP files.
However, as it is, Linux is never going to replace Windows as Joe Average’s computer. It doesn’t matter how cool AIGLX/XGL look, how pretty Ubuntu’s Gnome is, or how it doesn’t get infected with Windows malware. As I see it, Linux has 6 fundamental problems that hold it back from being up to par with Windows or Mac OS X:
- The Linux kernel has no way for hardware manufacturers to distribute drivers with their hardware.
- Namespace Conflicts: Installing everything under one prefix just doesn’t work.
- No Platform: Linux has no list of standard libraries/versions of libraries that all distros are guaranteed have. (ISVs need this).
- A lot of important projects and libraries don’t seem to understand the importance of stable APIs/ABIs (and the pain breaking them causes).
- ELF needs some work.
- Centralized repositories for installation just don’t work.
Hardware Drivers
Hardware manufacturers face two (closely) related problems:
- Drivers (modules) must be distributed with the kernel.
- The kernel has an unstable interface.
Drivers must be distributed with the kernel
While not technically true (out-of-tree modules can and do work), for hardware to work “out of the box,” the driver needs to be part of the Linux tree. As the kernel is currently, there is no way for hardware manufacturers to ship a CD with a wizard and drivers that the user can simply pop in their drive–all drivers need to be compiled against the kernel they will be loaded by.
Even the optimal hardware manufacturer who releases their drivers under the GPL and sends them upstream for inclusion in the official kernel tree still face a major problem; they must wait for their users to upgrade to the new version of the kernel. Until that happens, their hardware doesn’t work out of the box. However, the manufacturer and the user don’t want to wait, they want their stuff to work “right now.” Which brings us to number 2…
The kernel’s unstable interface
The kernel has a fairly unstable interface. The most obvious example was the 2.4 -> 2.5 change which had a large number of changes. I don’t have a huge problem with this kind of “major version” breakage. Architectures changes happen (and usually for the better). You see similar things in the Windows and Mac OS X world, with different drivers for Windows 2000, Server 2003, XP, and Vista, etc. While not these kinds of changes are not optimal, often cause upgrade pain, and add to maintenance costs, sometimes they’re unavoidable.
However, the kind of breakage I’m talking about doesn’t happen between these large releases. The “bad” breakage happens in the minor releases (2.6.8 -> 2.6.9, 2.6.17 -> 2.6.18). This causes 2 problems:
- HW Manufacturers need to constantly update their drivers (if they’re not maintained by the Linux community).
- Drivers usually can’t be backported easily.
I’m going to use a software example first, because it’s something that bit me recently:
The VMware Workstation needs to install some kernel modules on the host system (and on the guest system if you want improved performance). The VMware folks provide a nice script that tries to automatically build and install the modules. It worked fine on my Slackware 11.0 with the fairly recent 2.6.19 kernel. However, I ran into trouble trying to get the guest system modules working on Fedora Core 6 (kernel 2.6.18) and openSUSE 10.1 (2.6.17, IIRC). On both these systems, the build failed rather spectacularly with lots of compile errors flying by.
It turned out that there were a few changes in some kernel functions and structs that the kernel provided. After a few hours of Google and nano and I had the drivers working, but it was a tremendous hassle. On Windows, the guest drivers (binaries) installed with just a click of the mouse, thanks to the fairly stable Windows API.
The VMware issue is an example of how the kernel’s interface isn’t stable and how hardware manufacturers need to deal with the hassle of maintaining up to date drivers (and the users need to find and apply the patches that the manufacturers provide).
Conclusion: The kernel needs to provide a stable interface for hardware manufacturers to use.
Namespace Conflicts
Traditionally, UNIX systems have followed the philosophy of dumping all programs under one roof, namely in /usr. This inevitably leads to conflicts where two applications have the same name. The FHS tells ISVs to install into /opt/appname. This works in theory, but not in practice, because distros don’t support anything outside of /usr (and sometimes /usr/local), but certainly don’t support adding new paths.
Conclusion: Fixing this (correctly) requires a new directory layout. GoboLinux provides one option. I personally favour something more unique than just the application name–a rootname (like is used in Autopackage) is probably the safest way to namespace something: @domainname.com/project/version.
No Platform
A platform is a set of libraries and applications that are guaranteed to be available for ISVs to use. These libraries must be very, very stable with few API additions and no ABI breaks (unless dealt with properly).
The LSB’s goal is for a standard platform. However, their platform was way too small to be of any use whatsoever. Statically linking kdelibs, aspell, gecko, python, whatever, will not work. It will simply create many bloated, memory hogging applications. A useful platform should provide the majority of libraries that desktop applications need (Qt/Gtk, Gecko, *spell, libcurl, glib…the list goes on and on).
In theory, distros are platforms. However, they fail on two important points:
- They’re not stable
- Different distros have provide different platforms
Stability
One of the most important corner-stones of a good platform is that old applications continue to function. Windows is a good example: An application written for and compiled on Windows 95 will still most likely run on Windows XP. If you’re lucky, you can even get Windows 3.11 apps working.
Linux distros, on the other hand, often break backwards compatibility. For example, KDE 2 was written using Qt 2. Almost as soon as KDE 3 was released (which used Qt 3), distros stopped distributing Qt 2, and moved up to Qt 3. Suddenly, any applications that used Qt 2 no longer worked on modern distros. This kind of removal of libraries can cause immense pain in the ISV world:
Imagine you’re an ISV who uses Qt 2 in your (Linux) application. You distribute a binary linked against Qt 2. At this time, all distros ship Qt 2 (or it’s available from their repositories), so you don’t bother to include an extra 15MB of files with your application, and rely on the distro to provide Qt for you. Suddenly, Qt 3 is released, KDE 3 is released, meaning that distros can drop Qt 2 because none of the apps they ship depend on it. However, you, as the ISV, are suddenly inundated by a deluge of support calls wanting to know why your app stopped working when they upgraded to Red Hat 8.
Distros as platforms
Distros have gone in the direction of each distro providing their own unique platform [1]–that is, one with no guarantee of compatibility with any other distro. This (in my opinion) is the worst kind of fragmentation possible. As ISV will sell an application as working on “Fedora Core,” while the Ubuntu people are left totally out in the cold with no support, and possibly no working binary at all. This doesn’t work unless 1 distro is going to “win” and have all the desktop market share, which I don’t see happening.
Conclusion: Create a list of the common libraries that desktop applications use, force stability of these libraries, and make sure distros ship these libraries (including the old versions).
The Importance of Stability
This goes along with the platform, but focuses on the projects instead of the distros. Unfortunately, a lot of high-profile projects don’t seem to understand the importance of ABI stability.
Examples include gcc and Python. Unintentional breaks also occur a lot too, see libtiff, freetype, and GTK+.
Conclusion: Stability is an essential requirement for any decent platform. Educate developers about the importance of stability. Don’t allow unstable libraries into the platform.
ELF
ELF, the format that binaries and libraries are in, needs some work. Basically, instead of loading DSOs in a tree, like you’d expect, it just binds against whatever library is loaded into the image. So, you can easily get your app bound to the wrong ABI of a library if a dependency needs a different version of a dep than you do. See this for more information.
Conclusion: Fix ELF so that it loads shared libraries as expected.
Centralized Repositories
All distros currently use a centralized repository for installing software. While it looks kind of cool at first, and it kind of works at a small level, it just doesn’t work in the end:
- Users don’t want to wait until J. Random Hacker gets Firefox 2.2 packaged up and put into the repository, they want it right now.
- Application developers can’t release beta versions that users can test easily.
- The people doing the packaging (the distros) aren’t usually the most qualified people to package the application (that would be the app devs). Sometimes they mistakenly ship broken packages (bad patches, not including essential files, etc).
- Distros can and will arbitrarily drop packages from their repositories, leaving users no way to easily acquire the software.
- The protection that repositories provide against mal-ware is questionable at best.
- Centralized repositories provide a huge single point of failure. Repository servers have been cracked (Debian), and almost certainly will be in the future. All it takes is for a server to be compromised for a few hours for it to infect thousands of machines.
- It doesn’t work at all for proprietary vendors. “Truly Free” distributions won’t allow non-free stuff in their repositories. Non-Free as in beer ISVs (e.g., stuff you have to pay for) will probably be fairly reluctant to give their software over to these repositories.
- It doesn’t scale. As Mike pointed out, it fails the Microsoft test. Imagine if Microsoft suddenly said that all software needed to be downloaded through http://download.microsoft.com for security reasons to protect their users from mal-ware. a) It wouldn’t work–there’s too much software for Microsoft to keep up to date and approve, and b) It would be way too monopolistic.
Conclusion: Distros have the power and control right now and nobody likes to lose control. Hopefully one distro will see the light.
Final Thoughts
As you can see, there’s a lot of work to do. Can it be done? I don’t know–maybe it can, but definitely not without a major shift in the current “UNIX Philosophy” that Linux has (binaries? not my problem, dump it all in /usr, keep it all centralized, etc).
Perhaps a new operating system would be a better place to start. There’s a huge number of very important “Linux” projects that are not Linux dependent at all–KDE/Qt, Gnome/Gtk+, Samba, Firefox, and CUPS to name a few. All of these projects (and many, many others) are vital to any free operating system. Linux has an amazing community of companies and individuals who have all put tremendous effort into the free desktop–it would be a shame to see it go to waste.
April 16th, 2007 at 5:59 am
I’d like to see the OLPC Sugar system be tweaked for adult users and mainstream laptop technology. I think it could be a good start for a better platform, whilst reusing a lot of the Linux codebase.
April 16th, 2007 at 6:07 am
No, not OK… trying to maintain a different ELF binary system and a different library system for “desktop” as opposed to “server” is a completely brain-dead idea. The system for binary format and library linkage should be generally applicable on desktops, laptops, small business, big business and supercomputers and everything in between. And there’s absolutely no reason why that cannot be achieved.
Linux seems to be making steady growth in both the server space and the desktop space and the embedded space too. What are these mystery pundits so wrong about exactly? Linux is a bit like global warming… you can keep saying it isn’t happening (and many people still do), but that doesn’t matter because it happens anyway.
As for stability… the world changes, deal with it. Really.
The current objectives of most of the Open Source world are development and growth. These objectives are being achieved. Your objectives might be different, by all means put your effort into building a distribution that is more stable than the others if you think that will be useful. It’s pretty annoying to hear people tell me that they know what I want better than I do, then explaining to me what I “must” do.
ELF links exactly as I would expect and how I’ve always expected. The last thing I want is multiple versions of the same code running in the same memory image — that’s just a nightmare to program around because every program needs to store global state in some form or other and now you end up with multiple copies of the same global state or worse you end up with incompatible code manipulating a single piece of global state memory. Trying to debug such a situation is hopeless. It’s the last place anyone would want to go… but if you want to go there then go and write your own dynamic linker, Linux lets you do that.
Not much of the Open Source world is deeply concerned with the distribution issues of proprietary vendors. However, proprietary vendors continue to exist and some of them (e.g. Oracle) have decided to become Linux vendors as well. Here’s one: http://www.cadsoft.de/ and they support MS-Windows and Mac too so obviously they have figured out how to compile multiple versions of their code in a reasonably robust manner.
Finally, there’s no such thing as “Joe Average” user, just like there’s no such thing as a one-size-fits-all running shoe. Every user is different, they all have different needs. If you push them all into being “average” then they will all be unhappy. That’s why system diversity is a good thing.
Just because you find it convenient to try to categorise the world into nice, simple distinct units does not imply that anyone else should bother to respect your stereotypes.
Having standards is also a good thing, but not the sort of standards that are too inflexible and get in the way of progress. Even the posix standard is continuing to evolve — expect ppoll() and pselect() to join posix before long. The world moves, it takes all the running you can do just to stand in the same place. If that requires a recompile now and then, well big deal.
I’ve never noticed that Firefox had a problem getting out fresh versions to a wide variety of platforms on a regular basis. The official firefox release binaries and/or the various distro archive binaries are enough for about 95% of users — the ones who want to be more bleading edge are usually also the ones who can figure out how to compile from source, and that’s a good thing because the less knowledgable users automatically end up with more mature packages that have been through the testing of the more adept users. It’s a self-regulating system.
April 16th, 2007 at 10:09 am
Tel: The world can change without making lives hard for third party software vendors (open source or non-free). The software you’ve mentioned as successful in providing works-for-most-distros binaries for Linux are huge vendors. Firefox, Oracle both most likely have teams with expertise knowledge in binary compatibility. Smaller vendors probably don’t. Your average CS student spare-time open source hacker most likely don’t. I certainly didn’t before I started to be involved in Autopackage.
There is nothing wrong with adding new functions to an API. GTK+ does this all the time and still (mostly) maintains both API and ABI backward compatibility as long as you don’t use these new symbols. Your ppoll and pselect examples would fall into this category as well.
Both Apple and Microsoft has been able to maintain backwards compatibility, so it’s not an impossible task. Granted, they both spend huge amounts of resources on it - resources not always available in open source projects - but that doesn’t mean that we (F/OSS world) should ignore the issue completely. Also, sometimes it’s required to do a fresh start (MacOS 9 vs OS X, GTK 1.x vs GTK 2.x etc.) and it’s perfectly legal to do so. Just make sure to provide a migration route (parallel installable libs, emulating older system etc.).
Also, don’t be so hasty criticizing someone for dividing the world into distinct units when you do the exact same thing. (Regarding which type of users who wants to run the latest firefox). There are many reasons for wanting to use the latest version of something. I’ve been hit several times by this issue, and I’m using Ubuntu which is doing fairly frequent releases. Last time I was bit by this was when I wanted a newer version of Banshee. Sure, I can wait until the next ubuntu is released, but is it *really* needed that I upgrade my entire OS just because I want a newer version of a music player? You can’t just shake this issue away by saying that only tech people needs a newer version than the one in the repository.
One thing I do agree with you is that this should be solved for all systems - not just the desktop. But what I think that Taj meant is that desktop systems are typically more targeted by these kind of issues, since a typical server environment has dedicated admins that handles the compatibility issues (only install from repo, compile from source where needed, test test test)
April 16th, 2007 at 10:24 am
Tel Wrote:
I fail to understand your point here. You argue that the current ELF linker is doing exactly what you want and then complain that the results (global state corruption due to incompatible code) of its behavior is a nightmare. I agree. It *is* a nightmare. A better linker would keep the two loaded libs as completely separate entities, each with their own “global” state. It would also make sure that if MyApp is linked against v1 of libfoo and libbar [which is linked in by MyApp] is linked against v2 of libfoo, they would only access symbols from their respective version of libfoo.
No symbol collisions, no incompatible code accessing a super-shared global state. No hard bugs to debug in gdb. Everybody’s happy
April 16th, 2007 at 11:14 am
I highly agree.
However, I think we are talking to the wrong people. Nobody in the Linux developer world cares about this.
There are a lot of users that complain about this but gets highly criticized because they are questioning the distro / repository model.
The biggest benefit of the distro model is the easy way of finding software and updating the software that is installed. If Autopackage or Zeroinstall could provide that, it would be great - but right now, they simply don’t can’t (look at the amount of software in Debian compared to Autopackage / Zeroinstall).
The problem is that only the Linux distros are mature enough to compete against Windows (I think, please correct me if I am wrong) and that’s why people are using them.
We need a competing alternative: could it be BSD or Haiku?
April 16th, 2007 at 11:22 am
Isak, do you have the backports repos enabled? Backports brings newer versions of packages to older distros, at least in the Ubuntu world.
For #2 under repos:
The kind of people who are implied when people talk about “making it to the desktop” (something which it has already done in my family due to Ubuntu being a lot easier to use than Windows) are not the kind to be beta-testing. The kind of people who beta-test are the kind who probably know a bit about coding themselves–enough to type ./configure && make && sudo checkinstall. The “checkinstall” makes it into an easy to “apt-get remove” deb. The kind of people who are afraid to compile (the kind assumed in this kind of discussion) are probably not equipped to deal with the problems that result when using beta releases, so this really doesn’t matter.
#7 in repos:
Three letters: CNR. Linspire/Freespire’s Click N Run is now merging into the Ubuntu line. This lets commercial vendors charge their fees instead of trusting their stuff to regular repos.
April 16th, 2007 at 11:44 am
Mackenzie,
You are not addressing the kernel stuff.
Also, regarding backports: can I get F-Spot 0.3.5 for dapper?
I will answer myself: no, I can’t.
Also, regarding beta-testing: beta-testers don’t necessarily want to compile. They have better things to do.
April 16th, 2007 at 9:40 pm
Will any non-Windows OS ever make it to the desktop? Besides Linux, MacOS X is the only serious competitor that Windows has, but even OS X’s market share is not much more than 10%, despite the fact that it has existed for years.
April 23rd, 2007 at 3:35 am
As far as path prefixes go, it can be fixed by the application by putting unique id’s into filenames and paths, eg. /usr/local/bin/myapp-version-{some-guid} and /usr/local/share/myapp-version-{some-guid}.
There is stable interfaces in Linux, in the form of portable APIs like Java, or API-independant protocols like X11 and DBus - these can be used from multiple programming languages, on many versions of many distros, right now. As an example, libhal’s API is unstable, but if you connect to hald straight from DBus, you can skip using libhal completely.
If you want a stable C API, either ditch ELF and make another object file format, or force his highness Ulrich Drepper to accept the -Bdirect patch and make gcc generate Bdirect ELF sections by default (ie. without the need for some special switch that nobody will know or use); I’m sure if Red Hat’s management got involved in the decision, the patch would be accepted very quickly.
Centralized repositories are evil and I hate them - but what else do you offer people? It is bitterly ironic that autopackage, which is a far superior solution, has less than 1% of the packages in Ubuntu’s repos. The way to win here, is to build autopackages of software which is almost never packaged or really difficult to install: MythTV (a mission to install, entire distro - MythKnoppix - just to get it working), asterisk (entire distros: Trixbox, AsteriskNow are built around it), mplayer (many people including me run the Windows version under wine, it’s easier than compiling the Linux version), and so on. Get this kind of software autopackaged and watch autopackage really take off.
April 29th, 2007 at 12:49 pm
Hi..
Nice to see your blog. The content is really great. Well I am Viral Patel from India. I have developed an Operating System. I called it TAJ, An Object Oriented Operating System. Taj Operating System is a multitasking , multithreading, multiuser operating system.
I just wanted to share my expirence with you. My site is http://www.viralpatel.net and http://geocities.com/taj_os
June 7th, 2007 at 6:42 pm
[…] Will Linux Ever Make it to the Desktop? I’m OK, You’re Not OK Linux DDK API problem much worse than expected Ian Murdock - It […]
June 15th, 2008 at 9:01 pm
Pass Drug Test Blog…
……
September 8th, 2008 at 10:39 pm
what kind of memory does my computer use…
Six Apart started a working group in February 2006 to improve the Trackback protocol with the goal to eventually have it approved as…
September 13th, 2008 at 4:00 pm
desktop computer deal…
Some weblog software programs, such as Wordpress, Movable Type and Community Server, support automatic pingbacks where all the links…