The World is Flat

We used to use punch cards from the original IBM mainframes for making shopping lists. In grade school, I started doing some basic programming, and by high school I was pretty into computers... I graduated in 1991, but in 1989, in the early days of the Internet, a friend gave me a copy of a program he had downloaded onto a floppy disk, called 'Fractint.' It was not pirated, but was freeware, produced by a group of programmers, and was a program for drawing fractals. [Fractals are beautiful images produced at the intersection of art and math.] When the program started up, the screen would show this scrolling list of e-mail addresses for all the scientists and mathematicians who contributed to it. I noticed that the source code was included with the program. This was my first exposure to the concept of open-source. Here was this program that you just downloaded for free, and they even gave you the source code with it, and it was done by a community of people. It started to paint a different picture of programming in my mind. I started to think that there were some interesting social dynamics to the way certain kinds of software were written or could be written-as opposed to the kind of image I had of the professional software developer in the back office tending to the mainframe, feeding info in and taking it out for the business. That seemed to me to be just one step above accounting and not very exciting.”

After graduating in 1991, Behlendorf went to Berkeley to study physics, but he quickly became frustrated by the disconnect between the abstractions he was learning in the classroom and the excitement that was starting to emerge on the Internet.

“When you entered college back then, every student was given an e-mail address, and I started using it to talk to students and explore discussion boards that were starting to appear around music,” said Behlendorf. “In 1992,1 started my own Internet mailing list focused on the local electronic music scene in the Bay Area. People could just post onto the discussion board, and it started to grow, and we started to discuss different music events and DJs. Then we said, 'Hey, why don't we invite our own DJs and throw our own events?' It became a collective thing. Someone would say, 'I have some records,' and someone else would say, 'I have a sound system,' and someone else would say, 'I know the beach and if we showed up at midnight we could have a party.' By 1993, the Internet was still just mailing lists and e-mail and FTP sites [file transfer protocol repositories where you could store things]. So I started collecting an archive of electronic music and was interested in how we could put this online and make it available to a larger audience. That was when I heard about Mosaic [the Web browser developed by Marc Andreessen.] So I got a job at the computer lab in the Berkeley business school, and I spent my spare time researching Mosaic and other Web technologies. That led me to a discussion board with a lot of the people who were writing the first generation of Web browsers and Web servers.”

(A Web server is a software program that enables anyone to use his or her home or office computer to host a Web site on the World Wide Web. Amazon.com, for instance, has long run its Web site on Apache software. When your Web browser goes to www.amazon.com, the very first piece of software it talks to is Apache. The browser asks Apache for the Amazon Web page and Apache sends back to the browser the content of the Amazon Web page. Surfing the Web is really your Web browser interacting with different Web servers.)

“I found myself sitting in on this forum watching Tim Berners-Lee and Marc Andreessen debating how all these things should work,” recalled Behlendorf. “It was pretty exciting, and it seemed radically inclusive. I didn't need a Ph.D. or any special credentials, and I started to see some parallels between my music group and these scientists, who had a common interest in building the first Web software. I followed that [discussion] for a while and then I told a friend of mine about it. He was one of the first employees at Wired magazine, and he said Wired would be interested in having me set up a Web site for them. So I joined there at $10 an hour, setting up their e-mail and their first Web site-HotWired... It was one of the first ad-supported online magazines.”

HotWired decided it wanted to start by having a registration system that required passwords-a controversial concept at that time. “In those days,” noted Andrew Leonard, who wrote a history of Apache for Salon.com in 1997, “most Webmasters depended on a Web server program developed at the University of Illinois's National Center for Super-computing Applications (also the birthplace of the groundbreaking Mosaic Web browser). But the NCSA Web server couldn't handle password authentication on the scale that HotWired needed. Luckily, the NCSA server was in the public domain, which meant that the source code was free to all comers. So Behlendorf exercised the hacker prerogative: He wrote some new code, a 'patch' to the NCSA Web server, that took care of the problem.” Leonard commented, “He wasn't the only clever programmer rummaging through the NCSA code that winter. All across the exploding Web, other Webmasters were finding it necessary to take matters into their own keyboards. The original code had been left to gather virtual dust when its primary programmer, University of Illinois student Rob McCool, had been scooped up (along with Marc Andreessen and Lynx author Eric Bina) by a little-known company in Silicon Valley named Netscape. Meanwhile, the Web refused to stop growing—and kept creating new problems for Web servers to cope with.” So patches of one kind or another proliferated like Band-Aids on bandwidth, plugging one hole here and breaching another gap there.

Meanwhile, all these patches were slowly, in an ad hoc open-source manner, building a new modern Web server. But everyone had his or her own version, trading patches here and there, because the NCSA lab couldn't keep up with it all.

“I was just this near-dropout,” explained Behlendorf. “I was having a lot of fun building this Web site for Wired and learning more than I was learning at Berkeley. So a discussion started in our little working group that the NCSA people were not answering our e-mails. We were sending in patches for the system and they weren't responding. And we said, 'If NCSA would not respond to our patches, what's going to happen in the future?' We were happy to continue improving this thing, yet we were worried when we were not getting any feedback and seeing our patches integrated. So I started to contact the other people I knew trading patches... Most of them were on the standards working groups [the Internet Engineering Task Force] that were setting the first standards for the interconnectivity between machines and applications on the Internet... And we said, 'Why don't we take our future into our own hands and release our own [Web server] version that incorporated all our patches?'

“We looked up the copyright for the NCSA code, and it basically just said give us credit at Illinois for what we invented if you improve it-and don't blame us if it breaks,” recalled Behlendorf. “So we started building our own version from all our patches. None of us had time to be a full-time Web server developer, but we thought if we could combine our time and do it in a public way, we could create something better than we could buy off the shelf-and nothing was available then, anyway. This was all before Netscape had shipped its first commercial Web server. That was the beginning of the Apache project.”

By February 1999, they had completely rewritten the original NCSA program and formalized their cooperation under the name “Apache.”

“I picked the name because I wanted it to have a positive connotation of being assertive,” said Behlendorf. “The Apache tribe was the last tribe to surrender to the oncoming U.S. government, and at the time we worried that the big companies would come in and 'civilize' the landscape that the early Internet engineers built. So 'Apache' made sense to me as a good code name, and others said it also would make a good pun”-as in the APAtCHy server, because they were patching all these fixes together.

So in many ways, Bellendorf and his open-source colleagues-most of whom he had never met but knew only by e-mail through their open-source chat room-had created a virtual, online, bottom-up software factory, which no one owned and no one supervised. “We had a software project, but the coordination and direction were an emergent behavior based on whoever showed up and wanted to write code,” he said.

But how does it actually work? I asked Behlendorf. You can't just have a bunch of people, unmonitored, throwing code together, can you?

“Most software development involves a source code repository and is managed by tools such as the Concurrent Versions System,” he explained. “So there is a CVS server out there, and I have a CVS program on my computer. It allows me to connect to the server and pull down a copy of the code, so I can start working with it and making modifications. If I think my patch is something I want to share with others, I run a program called Patch, which allows me to create a new file, a compact collection of all the changes. That is called a patch file, and I can give that file to someone else, and they can apply it to their copy of the code to see what impact that patch has. If I have the right privileges to the server [which is restricted to a tightly controlled oversight board], I can then take my patch and commit it to the repository and it will become part of the source code. The CVS server keeps track of everything and who sent in what... So you might have 'read access' to the repository but not 'commit access' to change things. When someone makes a commit to the repository, that patch file gets e-mailed out to all the other developers, and so you get this peer review system after the fact, and if there is something wrong, you fix the bug.”

So how does this community decide who are trusted members?

“For Apache,” said Behlendorf, “we started with eight people who really trusted each other, and as new people showed up at the discussion forum and offered patch files posted to the discussion form, we would gain trust in others, and that eight grew to over one thousand. We were the first open-source project to get attention from the business community and get the backing from IBM.”

Because of Apache's proficiency at allowing a single-server machine to host thousands of different virtual Web sites-music, data, text, pornography-it began to have “a commanding share of the Internet Service Provider market,” noted Salon's Leonard. IBM was trying to sell its own proprietary Web server, called GO, but it gained only a tiny sliver of the market. Apache proved to be both a better technology and free. So IBM eventually decided that if it could not beat Apache, it should join Apache. You have to stop here and imagine this. The world's biggest computer company decided that its engineers could not best the work of an ad hoc open-source collection of geeks, so they threw out their own technology and decided to go with the geeks!

IBM “initiated contact with me, as I had a somewhat public speaker role for Apache,” said Behlendorf. “IBM said, 'We would like to figure out how we can use [Apache] and not get flamed by the Internet community, [how we can] make it sustainable and not just be ripping people off but contributing to the process...' IBM was saying that this new model for software development was trustworthy and valuable, so let's invest in it and get rid of the one that we are trying to make on our own, which isn't as good.”

John Swainson was the senior IBM executive who led the team that approached Apache (he's now chairman of Computer Associates). He picked up the story: “There was a whole debate going on at the time about open-source, but it was all over the place. We decided we could deal with the Apache guys because they answered our questions. We could hold a meaningful conversation with these guys, and we were able to create the [nonprofit] Apache Software Foundation and work out all the issues.”

At IBM's expense, its lawyers worked with the Apache group to create a legal framework around it so that there would be no copyright or liability problems for companies, like IBM, that wanted to build applications on top of Apache and charge money for them. IBM saw the value in having a standard vanilla Web server architecture-which allowed heterogeneous computer systems and devices to talk to each other, displaying e-mail and Web pages in a standard format-that was constantly being improved for free by an open-source community. The Apache collaborators did not set out to make free software. They set out to solve a common problem-Web serving-and found that collaborating for free in this open-source manner was the best way to assemble the best brains for the job they needed done.

“When we started working with Apache, there was an apache.org Web site but no formal legal structure, and businesses and informal structures don't coexist well,” said Swainson. “You need to be able to vet the code, sign an agreement, and deal with liability issues. [Today] anybody can download the Apache code. The only obligation is that they acknowledge that it came from the site, and if they make any changes that they share them back.” There is an Apache development process that manages the traffic, and you earn your way into that process, added Swainson. It is something like a pure meritocracy. When IBM started using Apache, it became part of the community and started making contributions.

Indeed, the one thing the Apache people demanded in return for their collaboration with IBM was that IBM assign its best engineers to join the Apache open-source group and contribute, like everyone else, for free. “The Apache people were not interested in payment of cash,” said Swainson. “They wanted contribution to the base. Our engineers came to us and said, 'These guys who do Apache are good and they are insisting that we contribute good people.' At first they rejected some of what we contributed. They said it wasn't up to their standards! The compensation that the community expected was our best contribution.”

On June 22, 1998, IBM announced plans to incorporate Apache into its own new Web server product, named WebSphere. The way the Apache collaborative community organized itself, whatever you took out of Apache's code and improved on, you had to give back to the whole community. But you were also free to go out and build a patented commercial product on top of the Apache code, as IBM did, provided that you included a copyright citation to Apache in your own patent. In other words, this intellectual commons approach to open-sourcing encouraged people to build commercial products on top of it. While it wanted the foundation to be free and open to all, it recognized that it would remain strong and fresh if both commercial and noncommercial engineers had an incentive to participate.

Today Apache is one of the most successful open-source tools, powering about two-thirds of the Web sites in the world. And because Apache can be downloaded for free anywhere in the world, people from Russia to South Africa to Vietnam use it to create Web sites. Those individuals who need or want added capabilities for their Web servers can buy products like WebSphere, which attach right on top of Apache.

At the time, selling a product built on top of an open-source program was a risky move on IBM's part. To its credit, IBM was confident in its ability to keep producing differentiated software applications on top of the Apache vanilla. This model has since been widely adopted, after everyone saw how it propelled IBM's Web server business to commercial leadership in that category of software, generating huge amounts of revenue.

As I will repeat often in this book: There is no future in vanilla for most companies in a flat world. A lot of vanilla making in software and other areas is going to shift to open-source communities. For most companies, the commercial future belongs to those who know how to make the richest chocolate sauce, the sweetest, lightest whipped cream, and the juiciest cherries to sit on top, or how to put them all together into a sundae. Jack Messman, chairman of the Novell software company, which has now become a big distributor of Linux, the open-source operating system, atop which Novell attaches gizmos to make it sing and dance just for your company, put it best: “Commercial software companies have to start operating further up the [software] stack to differentiate themselves. The open source community is basically focusing on infrastructure” (Financial Times, June 14, 2004).

The IBM deal was a real watershed. Big Blue was saying that it believed in the open-source model and that with the Apache Web server, this open-source community of engineers had created something that was not just useful and valuable but “best in its class.” That's why the open-source movement has become a powerful flattener, the effects of which we are just beginning to see. “It is incredibly empowering of individuals,” Brian Behlendorf said. “It doesn't matter where you come from or where you are-someone in India and South America can be just as effective using this software or contributing to it as someone in Silicon Valley.” The old model is winner take all: I wrote it, I own it-the standard software license model. “The only way to compete against that,” concluded Behlendorf, “is to all become winners.”

Behlendorf, for his part, is betting his career that more and more people and companies will want to take advantage of the new flat-world platform to do open-source innovation. In 2004, he started a new company called CollabNet to promote the use of open-sourcing as a tool to drive software innovation within companies. “Our premise is that software is not gold, it is lettuce-it is a perishable good,” explained Behlendorf. “If the software is not in a place where it is getting improved over time, it will rot.” What the open-source community has been doing, said Behlendorf, is globally coordinated distributed software development, where it is constantly freshening the lettuce so that it never goes rotten. Behlendorfs premise is that the open-source community developed a better method for creating and constantly updating software. CollabNet is a company created to bring the best open-source techniques to a closed community, i.e., a commercial software company.

“CollabNet is an arms dealer to the forces flattening the world,” said Behlendorf. “Our role in this world is to build the tools and infrastructure so that an individual -in India, China, or wherever-as a consultant, an employee, or just someone sitting at home can collaborate. We are giving them the toolkit for decentralized collaborative development. We are enabling bottom-up development, and not just in cyberspace... We have large corporations who are now interested in creating a bottom-up environment for writing software. The old top-down, silo software model is broken. That system said, 'I develop something and then I throw it over the wall to you. You find the bugs and then throw it back. I patch it and then sell a new version.' There is constant frustration with getting software that is buggy-maybe it will get fixed or maybe not. So we said, 'Wouldn't it be interesting if we could take the open-source benefits of speed of innovation and higher-quality software, and that feeling of partnership with all these stakeholders, and turn that into a business model for corporations to be more collaborative both within and without?'”

I like the way Irving Wladawsky-Berger, IBM's Cuban-born vice president for technical strategy and innovation, summed open-sourcing up: “This emerging era is characterized by the collaborative innovation of many people working in gifted communities, just as innovation in the industrial era was characterized by individual genius.”

The striking thing about the intellectual commons form of open-sourcing is how quickly it has morphed into other spheres and spawned other self-organizing collaborative communities, which are flattening hierarchies in their areas. I see this most vividly in the news profession, where bloggers, one-person online commentators, who often link to one another depending on their ideology, have created a kind of open-source newsroom. I now read bloggers (the term comes from the word “Weblog”) as part of my daily information-gathering routine. In an article about how a tiny group of relatively obscure news bloggers were able to blow the whistle that exposed the bogus documents used by CBS News's Dan Rather in his infamous report about President George W. Bush's Air National Guard service, Howard Kurtz of The Washington Post wrote (September 20, 2004), “It was like throwing a match on kerosene-soaked wood. The ensuing blaze ripped through the media establishment as previously obscure bloggers managed to put the network of Murrow and Cronkite firmly on the defensive. The secret, says Charles Johnson, is 'open-source intelligence gathering.' Meaning: 'We've got a huge pool of highly motivated people who go out there and use tools to find stuff. We've got an army of citizen journalists out there.'” That army is often armed with nothing more than a tape recorder, a camera-enabled cell phone, and a Web site, but in a flat world it can collectively get its voice heard as far and wide as CBS or The New York Times. These bloggers have created their own online commons, with no barriers to entry. That open commons often has many rumors and wild allegations swirling in it. Because no one is in charge, standards of practice vary wildly, and some of it is downright irresponsible. But because no one is in charge, information flows with total freedom. And when this community is on to something real, like the Rather episode, it can create as much energy, buzz, and hard news as any network or major newspaper.

Another intellectual commons collaboration that I used regularly in writing this book is Wikipedia, the user-contributed online encyclopedia, also known as “the people's encyclopedia.” The word “wikis” is taken from the Hawaiian word for “quick.” Wikis are Web sites that allow users to directly edit any Web page on their own from their home computer. In a May 5, 2004, essay on YaleGlobal online, Andrew Lih, an assistant professor at the Journalism and Media Studies Centre at the University of Hong Kong, explained how Wikipedia works and why it is such a breakthrough.

“The Wikipedia project was started by Jimmy Wales, head of Internet startup Bomis.com, after his original project for a volunteer, but strictly controlled, free encyclopedia ran out of money and resources after two years,” wrote Lih. “Editors with PhD degrees were at the helm of the project then, but it produced only a few hundred articles. Not wanting the content to languish, Wales placed the pages on a wiki Website in January 2001 and invited any Internet visitors to edit or add to the collection. The site became a runaway success in the first year and gained a loyal following, generating over 20,000 articles and spawning over a dozen language translations. After two years, it had 100,000 articles, and in April 2004, it exceeded 250,000 articles in English and 600,000 articles in 50 other languages.

And according to Website rankings at Alexa.com, it has become more popular than traditional online encyclopedias such as Britannica.com.“

How, you might ask, does one produce a credible, balanced encyclopedia by way of an ad hoc open-source, open-editing movement? After all, every article in the Wikipedia has an “Edit this page” button, allowing anyone who surfs along to add or delete content on that page.

It starts with the fact, Lih explained, that “because wikis provide the ability to track the status of articles, review individual changes, and discuss issues, they function as social software. Wiki Websites also track and store every modification made to an article, so no operation is ever permanently destructive. Wikipedia works by consensus, with users adding and modifying content while trying to reach common ground along the way.

“However, the technology is not enough on its own,” wrote Lih. “Wales created an editorial policy of maintaining a neutral point of view (NPOV) as the guiding principle... According to Wikipedia's guidelines, The neutral point of view attempts to present ideas and facts in such a fashion that both supporters and opponents can agree...' As a result, articles on contentious issues such as globalization have benefited from the cooperative and global nature of Wikipedia. Over the last two years, the entry has had more than 90 edits by contributors from the Netherlands, Belgium, Sweden, United Kingdom, Australia, Brazil, United States, Malaysia, Japan and China. It provides a manifold view of issues from the World Trade Organization and multinational corporations to the anti-globalization movement and threats to cultural diversity. At the same time malicious contributors are kept in check because vandalism is easily undone. Users dedicated to fixing vandalism watch the list of recent changes, fixing problems within minutes, if not seconds. A defaced article can quickly be returned to an acceptable version with just one click of a button. This crucial asymmetry tips the balance in favor of productive and cooperative members of the wiki community, allowing quality content to prevail.” A Newsweek piece on Wikipedia (November 1, 2004) quoted Angela Beesley, a volunteer contributor from Essex, England, and self-confessed Wikipedia addict who monitors the accuracy of more than one thousand entries: “A collaborative encyclopedia sounds like a crazy idea, but it naturally controls itself.”

Meanwhile, Jimmy Wales is just getting started. He told Newsweek that he is expanding into Wiktionary, a dictionary and thesaurus; Wikibooks, textbooks and manuals; and Wikiquote, a book of quotations. He said he has one simple goal: to give “every single person free access to the sum of all human knowledge.”

Wales's ethic that everyone should have free access to all human knowledge is undoubtedly heartfelt, but it also brings us to the controversial side of open-source: If everyone contributes his or her intellectual capital for free, where will the resources for new innovation come from? And won't we end up in endless legal wrangles over which part of any innovation was made by the community for free, and meant to stay that way, and which part was added on by some company for profit and has to be paid for so that the company can make money to drive further innovation? These questions are all triggered by the other increasingly popular form of self-organized collaboration-the free software movement. According to the openknowledge.org Web site, “The free/open source software movement began in the 'hacker' culture of U.S. computer science laboratories (Stanford, Berkeley, Carnegie Mellon, and MIT) in the 1960's and 1970's. The community of programmers was small, and close-knit. Code passed back and forth between the members of the community-if you made an improvement you were expected to submit your code to the community of developers. To withhold code was considered gauche-after all, you benefited from the work of your friends, you should return the favor.”

The free software movement, however, was and remains inspired by the ethical ideal that software should be free and available to all, and it relies on open-source collaboration to help produce the best software possible to be distributed for free. This a bit different from the approach of the intellectual commons folks, like Apache. They saw open-sourcing as a technically superior means of creating software and other innovations, and while Apache was made available to all for free, it had no problem with commercial software being built on top of it. The Apache group allowed anyone who created a derivative work to own it himself, provided he acknowledge the Apache contribution.

The primary goal of the free software movement, however, is to get as many people as possible writing, improving, and distributing software for free, out of a conviction that this will empower everyone and free individuals from the grip of global corporations. Generally speaking, the free software movement structures its licenses so that if your commercial software draws directly from their free software copyright, they want your software to be free too.

In 1984, according to Wikipedia, an MIT researcher and one of these ex-hackers, Richard Stallman, launched the “free software movement” along with an effort to build a free operating system called GNU. To promote free software, and to ensure that its code would always be freely modifiable and available to all, Stallman founded the Free Software Foundation and something called the GNU General Public License (GPL). The GPL specified that users of the source code could copy, change, or upgrade the code, provided that they made their changes available under the same license as the original code. In 1991, a student at the University of Helsinki named Linus Torvalds, building off of Stallman's initiative, posted his Linux operating system to compete with the Microsoft Windows operating system and invited other engineers and geeks online to try to improve it-for free. Since Torvalds's initial post, programmers all over the world have manipulated, added to, expanded, patched, and improved the GNU/Linux operating system, whose license says anyone can download the source code and improve upon it but then must make the upgraded version freely available to everybody else. Torvalds insists that Linux must always be free. Companies that sell software improvements that enhance Linux or adapt it to certain functions have to be very careful not to touch its copyright in their commercial products.

Much like Microsoft Windows, Linux offers a family of operating systems that can be adapted to run on the smallest desktop computers, laptops, PalmPilots, and even wristwatches, all the way up to the largest supercomputers and mainframes.