November 2000 Column

[ Site Index] [ Linux Index] [ Feedback ]


What is the largest computer company? If you ask around, most people will look puzzled and say "Microsoft, of course". This is the understandable -- but wrong -- legacy of a decade of propaganda (and in the software world it's true). But in general computing, the biggest company is, as it always has been, IBM.

And in 2000, IBM came down with a startlingly evangelical dose of that old-time Linux religion.

To understand why, you really need to cast your mind back to the 1960's. That's when IBM clealry rose to dominate the commercial computer market. Computers in those days were big beasts -- rooms full of huge humming boxes containing board after board full of discrete transistors and core memory. Machines that loaded programs from a hopper full of punched cards, executed them, then spat out more punched cards with an answer encoded on them. The program that loaded and executed whatever was on those cards, the Loader, was the primitive single-celled ancestor of today's operating system; and it was different for every model of computer. Indeed, there was no such thing as a standardized CPU architecture; every model of computer was different.

IBM, even then, was a large corporation. They developed computers for the civil market, and under contract to the military; in particular, as the limitations of one range of computers became apparent (such as the early 1604 series machines) they'd start development work on a new and incompatible range, the 360 series. (Back in those days the idea of software portability was a novelty; most software was still written in hand-carved assembly language, and no software industry existed as such.) However, during the 1960's the first real operating systems were developed in order to facilitate time sharing and to make it easier to write software in high level languages.

With this development came the spread of software from one computing centre to another; operating systems were developed by manufacturers and supported, other companies began writing software and in some cases selling it, and manufacturers found themselves locked into a cycle of providing machines that could run old, pre-existing software.

At this point, IBM had a problem -- although it took them a decade to realise how serious it was. Each sector -- mainframe, minicomputer -- was dominated by a hardware architecture and one or two operating systems. These machines all had loyal customer bases; but they were very bad at interoperating. The idea of a common networking protol such as TCP/IP was still a radical laboratory experiment. Computing was dominated by big data centres that ran programs for users sitting at remote terminals. And computers still cost as much to buy and run as trains (or coaches, if you wanted a minicomputer).

By the time of the mid-1980's PC boom, IBM had a noticeable problem. The PC lowered the cost of owning a computer to the price of a car; it also added a new machine that was starved of the information and big business applications running on the corporate mainframes and minicomputers. Organisations that had bought mainframes to run their inventory and ordering system, and minicomputers to run their accounting department, now had PC's on every secretary's desk -- and none of these machines could exchange information!

Around 1986, IBM hatched an ambitious plan to fix this problem. The scheme was to converge all their product ranges on a set of user interface standards, networking protocols, and file formats, so that PC's could talk to minicomputers or mainframes via token-ring network, exchange data via SNA protocols, and provide access to mainframe applications with a menu-driven CUI interface. (The keystrokes of which have given the Windows world a permanent legacy -- Windows 3.1 was designed with CUI compatability in mind.)

IBM poured billions of dollars into the ambitious scheme to allow all their systems to interoperate -- and it failed.

Part of the failure occured because IBM had lost control of the PC market. When they launched the proprietary, PS/2 systems (using the MCA bus, which IBM wanted royalties for) they ignored the realities of the PC industry in which they were now a minority player. They also tried to push the token ring networking system into the industry -- faster than ethernet, but far more expensive. And finally, internicine feuding between minicomputer and mainframe divisions caused their own interoperability problems to mushroom. The sad fact was that in the late 1980's, IBM was effectively a group of four or five companies flying in close formation -- each division, with its own territory (the 370 mainframe division, UNIX RS/6000 workstations, the minicomputer division, and the PC division) was the size of a large company in its own right, and viewed its fellow divisions as a competitor.

By the early 1990's, the SNA convergence idea was a dead letter. OS/2, the operating system that would tie the desktop PC seamlessly to the mainframe and minicomputer, had been dumped by Microsoft, who went their own way (taking OS/2 Release 3, aka Windows NT, with them). Novell had chewed the guts out of the minicomputer division's natural market of file servers, while the minicomputer division with its new AS/400's was locked in combat with the UNIX division with their RS/6000 workstations. (A workstation, ultimately, turns out to be a fancy name for a minicomputer with a framebuffer and a copy of UNIX.) The finale was one of the worst corporate humiliations in history, when in 1994 IBM announced an annual loss of more than six billion dollars, and Microsoft moved centre-stage to dominate the computer industry by acting as a natural focus into which all other business systems must plug.

But what has this got to do with Linux?

IBM is still big. IBM has learned humility -- they're much better at talking to outsiders than they used to be. IBM have learned to make good kit, rather than relying on the aphorism "nobody ever got fired for buying IBM" to sell their goods. IBM learned to love TCP/IP and the web and Java, as tools for delivering the data stored on their mainframes to the users on their PC's. And with that new-found receptiveness to ideas, in 1998 technicians at IBM began playing with the new upstart operating system on their PC's.

I now have a confession to make. When I first heard that some engineers were porting Linux to run on the new, G4-powered IBM Series/390 mainframes, in early 1999, I thought they were bonkers. While it's a glorious example of hacking, what's the point?

I was wrong. There's a deadly serious point, and in the summer of 2000 it became clear.

Linux supports the Java virtual machine adequately. It's free. Linux is portable. It now runs on the S/390, and a port is underway to the AS/400 minicomputer architecture. You can buy Linux on IBM Thinkpad notebook PC's, and you can buy Linux for the S/390 from SuSE and others -- and IBM will support it. More: IBM support running Linux both on the mainframe's bare metal, and as a user process under another operating system such as VM or MVS. So each user gets their own Linux session on the mainframe -- providing high security. (If one user's session is cracked by an intruder, the others are still secure -- it's like a network of virtual machines.)

IBM is fixated on the idea of using those big mainframes and minicomputers as file servers to deliver a deluge of data to the users. Now they're in the process of achieving a position where a single operating system, with a single user interface and networking protocol, runs on every piece of hardware they manufacture. Even the RS/6000's, which run AIX (a flavour of UNIX), are being upgraded to an AIX release specifically designed to make porting Linux software easy.

Moreover, Linux is a Java bytecode platform. Compile a Java application, and you get bytecode that can run on any machine with a Java Virtual Machine (JVM). IBM have open-sourced their own JVM in an attempt to get it into Linux, and by that route ensure that every IBM machine that runs Linux can also run binaries compiled from Java source code.

By 2001, it will be possible for a company with a mixture of mainframes, minicomputers, workstations, and PC's to do everything on a single operating system, and deploy software written in a single language and compiled to a common object format. IBM will have achieved their twenty-year goal of totally integrating all their products, while the rest of the industry has been ignoring them.

The implications for IBM are staggering; Linux has become the holy grail for their marketing people, because they can now offer a one-size-fits-all solution that scales from a single desktop PC up to clusters of mainframes. Meanwhile, they're working at filling in the remaining pieces of the jigsaw. IBM recently announced that they're pumping US $200M into porting applications to Linux. They're open sourcing some of their Java development tools, to encourage application development in that language. They've joined the Linux Software Labs consortium (being set up in conjunction with HP and other big-name companies) to fuel software development and interoperability standards. And they just released a Bluetooth protocol stack under the GPL, for incorporation into the Linux kernel.

(Bluetooth is the new standard for wireless personal-area networking, to allow devices such as keyboards or mobile phones or wrist watches or personal stereos or servers to talk to each other across a range of up to ten metres without needing wires. Bluetooth is going to be very, very, important in a couple of years time, as our concept of a monolithic personal computer is replaced by a cluster of interoperating widgets that surround us with a computing environment, adrift in a sea of wireless data.)

As IBM discovers Linux, IBM is forced to adopt the open source ethos. For IBM, this is not an unnatural move; they've been forced to confront the failures of the early 1990's, and in any case IBM is not a software company -- it's a solutions provider, making money off the combination of hardware and services. It would be a short step for IBM to become a true open source company from top to bottom -- and a natural one, as Linux becomes the vital core of their operations.

The future is Big; the future is Blue ...

What Brian Ken and Dennis did Next

And now for something that isn't Linux.

The other day, a fat box popped through my letter box. (Actually it didn't -- the postman rang the doorbell and heaved it onto my porch, grunting with effort.) On removing the wrapping, I was confronted by a large, garish box titled "Plan 9 goes Open Source". It was, indeed: Christmas had come early, and delivered the most exciting new toy I've had in ages.

(Please pardon me while I pause to get my inner geek back under control. All will be explained in a minute ....)

In the beginning there was a telephone company called AT&T. And AT&T was huge, and its research arm, Bell Labs, was also huge. Given a near-monopoly position, AT&T was in a position to engage in blue-sky research to some extent. And so it was that when a talented programmer wanted to play Space War on a PDP-8 that had no operating system the result, by 1972, was an operating system -- UNIX -- and a typesetting system -- troff.

By the time UNIX squirmed out of Bell Labs and infected universities everywhere, it was of little interest to researchers of the calibre of Ken Thompson, Dennis Ritchie, and Rob Pike. By the mid-1980's they were focussing their attention on new fields; on the design of new programming languages, on networking over the new TCP/IP protocols, and on the use of graphical interfaces (albeit to enable programmers to work more effectively, rather than to make the computer more accessible to non-programmers). Through a process of evolving new tools, their platform had evolved by 1988 or thereabouts into the first edition of Plan 9 from Bell Labs: a work of sublime beauty and elegance or a total mess, depending on your perspective, just like the movie it was named after ("Plan Nine from Outer Space", directed by Ed Wood -- a cult classic and arguably the worst movie in history).

Plan 9 didn't take UNIX and extend it; rather, it took UNIX and tried to throw away the do-everything mentality that dogged the big server operating system that the UNIX OEMs were selling. Plan 9 retained the UNIX philosophy of having a lot of small tools, each of which do just one thing and do it well. It took the use of filesystems and device files and radically extended it; in Plan 9, all system resources are presented as filesystems. And it introduced the idea of a cluster of servers -- file servers (to store files on a networked filesystem), compute servers (to crunch data and do computing tasks), and display servers (to use as graphics terminals). First edition Plan 9 used a thing called a Gnot as a terminal -- a weird homebrew graphics console built in-house at Bell Labs -- but could also run on other hardware. When the first papers about Plan 9 came out it generated considerable technical interest -- but no follow-on commercialisation. AT&T, recognizing their mid-70's mistake in freeing UNIX from their control, decided to sit on Plan 9, and sat on it so thoroughly that they nearly smothered it.

System resources in Plan 9 are names and accessed like files in a hierarchical filesystem -- something of this trend has already shown up on Linux in the form of the /proc filesystem. A networking protocol called 9P provides remote access to these resources, much as NFS provides access to remote files on UNIX -- although NFS won't let you export your Linux system's process table. Finally, and most radically of all, each user process has its own view of the resources accessible to it, which is provided via a hierarchical namespace; for example, every process can see a file called /dev/cons (the current console), which is connected to whatever console device the file is running in. Similarly, all processes can see /dev/mouse -- whether or not the mouse at the other end of the namespace is on the same machine as the process. This level of abstraction makes programming somewhat easier than under a traditional UNIX-like system.

The graphics system rio (and its predecessor 81/2) are essentially terminal multiplexers. Each window you open in rio provides access to the same devices (/dev/cons, /dev/mouse, /dev/bitblt), mapped into the context of the current window; rio itself requires access to these devices, and can run in a window. (Essentially rio was written to provide the same ability as X11 -- to run processes on one host and display their output on another -- but achieves this task far more simply and elegantly.)

Plan 9 revolves around file servers, because just about any system you use on Plan 9 is interfaced with via a filesystem. For example, there's no FTP client; instead, there's an ftp filesystem -- cd to /n/ftp/ and the ftpfs will log into and present you with its root directory contents. Another server, exportfs, lets you export name spaces from one Plan 9 machine to another. The command "import" calls up a remote machine, starts exportfs on it, and grabs a connection to the local namespace. For example, the command:

import foobar /proc
imports the process table from machine foobar, so that you can run a debugger locally on a process running on foobar.

An important point to note is that in Plan 9, when you mount one directory on top of another, the contents of both directories are visible in your name space. Such a mount is called a union directory -- for example, /bin is built as a union mount of /$cputype/bin, /rc/bin, and possibly something like /home/$user/bin if the user wants to add their own directory full of executables. (Who needs a long PATH variable?)

A really important difference from Linux/UNIX is that Plan 9 has abolished the ioctl() system call, used to manipulate the attributes and behaviour of a file or device from which data is read or written. Under Plan 9, every device has two files associated with it -- one from which data can be read or written, and a control device. For example, a serial device /dev/eia1 will be accompanied by /dev/eia1ctl. To set /dev/eia1 to run at 9600 baud, instead of calling ioctl() on it with some weird parameters, you open /dev/eia1ctl and write the text string b9600 to it. (Control of devices by plain text messages is somewhat important in Plan 9, because Plan 9 is intended to run in a heterogenous environment where byte order may differ between the machine a device is physically attached to and the host where the process trying to control the device is running.)

If this isn't extreme enough for you, networking is also handled via the filesystem namespace. Tcp connections are accessed under /net/tcp; this contains connection directories for each active connection, and each of these contains some more virtual files. To create a new network connection, you open /net/tcp/clone; this returns a handle on a new, unused connection control file. You read the connection number from this, which enables you to work out the path to the connection directory. You then open the connection to a specific host by writing a string such as:

to the control file. (This opens a connection to host on port 23 (telnet).) Finally, you can read and write to the "data" file in the directory you got via the control file to talk to the remote host via this connection.

This abstraction is sufficiently high level that you can implement network applications in awk on Plan 9 more easily than you can in Perl on Linux. Which is something of a surprise because on Linux awk is network-dumb, while Perl is network glue.

Now if you'll excuse me, I can feel a cold shower coming on. I've just rambled on at length about why Plan 9 is technically superior to all other operating systems on the planet; quite a surprise for a Linux columnist! Before you shout at me, I ought to put this in some sort of context ...

As mentioned earlier, Bell Labs (now known as Lucent Technologies) nearly strangled Plan 9 at birth. Released in 1995 under very restrictive non- commercial licensing terms, Plan 9 used to cost an arm and a leg. Indeed, it cost so much that to the best of my knowledge nobody succeeded in building a commercial implementation of it. It remained an academic curiosity, popular with some researchers, and a source of good ideas to be periodically raided by the UNIX community -- and nothing more.

All this changed about two or three months ago. First, Bell Labs released Plan 9 under a nearly-but-not-quite open source license. (The main restrictions are that if you modify the source code you are required to contribute your changes back to Lucent Technologies; and if you take legal action against them for damages arising, they can yank your license to use it. Both of which clauses violate the Open Source Definition.) However, for the first time Plan 9 was out of Bell Labs and in the hands of the curious. Next, a small company called Vita Nuova (based in York, in this very country) announced a Plan 9 distribution. Instead of paying US $350 for a very restrictive license and two books, you can now get Plan 9 for about 66 (for the full kit -- just 15 for the CD!) from VitaNuova. Like I said, if you have an ounce of curiosity about operating system design, or just want to see what the original authors of UNIX decided was wrong with it (and how they fixed it), this is a must- buy. It's also a sign of the future -- because I'd be surprised if a whole lot of the concepts in Plan 9 don't make it into Linux sooner or later.

[ Site Index] [ Linux Index] [ Feedback ]