A Brief History of Webmastery

This article was first published in Web Developer magazine, Vol. 3 No.2 Mar/Apr 1997. Written by David Belson.

The World Wide Web is based on ideas that were developed over the past fifty-plus years.

The World Wide Web is based on ideas that were developed over the past fifty-plus years, including “memex” and “Xanadu,” though the Web might have been very different if those ideas had come to fruition. In another 50 years, the Web will likely be very different, based on ideas that are being developed today.

In the Web’s current incarnation, the evolved role of the Webmaster-as a team and a team member-has become crucial to the success of a Web site. The Webmasters’ Guild defines these roles as evangelism, business strategy, domain expertise, content, information architecture, design, and technology. (For the complete story on the roles of the Webmaster, and the Webmasters’ Guild organization, see “The Complex Art of Webmastery” in Web Developer®’s Nov/Dec 1996 issue).

However, if earlier ideas had been implemented, who might the Webmasters have been, and what roles might they have played? With today’s Web server technologies, what responsibilities and challenges do Webmasters face? And with the implementation of technologies currently in development-such as “embedded,” what members will need to be added to the Webmaster team, and what roles will they play?

What The Vision Was

In 1945, Vannevar Bush published “As We May Think,” in which he described the “memex”-a microfilm-based device “in which an individual stores his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility.”

The contents of one’s memex library could be expanded by adding additional microfilm documents, or by “scanning” them directly onto blank microfilm. The owner of the memex could create what Bush termed “trails,” where documents and notes were linked together into a personalized information infrastructure. In a memex-based world, each user would be their own Webmaster, building webs (trails) of information that were personalized to their needs and interests. Each individual would perform the roles of information architect and content provider, and in Bush’s vision, the technology would be standardized, and ubiquitous, so the technical Webmaster and evangelist roles would be far less necessary.

Thirty-five years later, Ted Nelson described Project Xanadu in his book “Literary Machines,” based on the concepts of “hypertext” and “hypermedia” (terms Nelson coined in 1965). The Xanadu FAQ describes Xanadu as “a unifying system of order for all information, non-hierarchical and side-linking, including electronic publishing, personal work, organization of files, corporate work and groupware.” Xanadu systems allowed any portion of one document to be linked to any portion of any other document in the “Xanadu docuverse,” and the systems contained mechanisms for payment of royalties to information owners, whenever their documents were read.

“It is optimized for a point-and-click universe, where users jump from document to document, following links and buying small pieces as they go,” notes the FAQ, presenting a model very similar to what many would like to see the World Wide Web evolve into. Although one of Xanadu’s developers notes that the Web lacks nearly every one of Xanadu’s advanced features, Xanadu Webmasters would likely be very similar to today’s Webmasters. Archivists and librarians would be valuable members of the team, providing domain expertise in the content and information architecture areas. The hypermedia aspect of Xanadu would have required the domain expertise of graphic and interface designers. With payment of content viewing a fundamental underpinning of Xanadu, business strategists would no doubt have their place on the Webmaster team, developing ways for content providers and publishers to increase their earnings.

The Current Reality

In 1989, Tim Berners-Lee and Robert Cailliau proposed a hypertext-based systems for sharing documentation and High Energy Physics (HEP) documents within CERN. The following year, Berners-Lee prototyped Web server and browser software on a NeXT cube, and he and Cailliau decided on “World Wide Web” as a name for the system.

The Web continued to spread within the HEP community, as the Stanford Linear Accelerator Center set up the first Web server in the United States, publishing the contents of an existing database of abstracts of physics papers. The following year, HEP laboratories in Hamburg, Amsterdam, and Chicago set up Web servers. By the end of 1992, there were 50 Web servers in the world.

So who were the Webmasters for these early Web servers? While on-line histories don’t provide a specific answer, we can guess that they were the lab’s system administrators, or those physicists with Internet-connected workstations on their desks, and the expertise to set up the software. They served the primary function of technical Webmaster; though they were also in the roles of providing content and designing an information infrastructure, they simply served up that information (papers, abstracts, etc.) that was readily available to them in digital form, not worrying about ease of navigation.

1993 saw the release of CERN Web server software with basic protection mechanisms and early versions of NCSA HTTPd, as well as the availability of the first generation of graphical browsers, including Viola, Midas, and Mosaic. This growth in server and browser software began to make the Web available to greater numbers of people, moving further out of the HEP community, and further into the academic and commercial worlds, as an additional 200 “early adopters” set up Web servers by the end of the year. Web servers, by and large, had not yet moved into full institutional acceptance, so Webmasters, still often a single person, were challenged to perform a number of functions. Since they likely controlled the resources the servers ran on, they continued to play the role of technical Webmaster.

The Webmaster’s role as evangelist began to develop more fully during this period in the Web’s growth, as Webmasters tried to convince librarians, archivists, and marketers of the value of the Web, in order to get them to provide content with more depth and greater value to users. At this time, coding HTML was considered to be something of a black art, so Webmasters, who were among a select group that had developed proficiency in HTML, also played the role of designer and information architect in creating early Web sites.

By the end of 1994, the number of Web servers had grown ten-fold, to 2500, and server software was available for most major operating systems, including Unix, VM/CMS, VMS, Microsoft Windows, MS-DOS, and MacOS. Growth in the Web at this time was also due in part to media exposure, as popular magazines and newspapers began to run stories introducing the public to this “new technology.”

As a result of all of this, the Web grew in acceptance and importance, and the concept of the Webmaster as a team-working with and supporting the technical Webmaster-began to take shape. New members of the Webmaster team included:

Graphic Designers: The early Webmasters learned enough HTML to create and convert content into a state that was acceptable for Web presentation. The graphic designers moved beyond being mere HTML jockeys, and brought content into agreement with accepted graphic standards, as well as designing sites that were attractive enough to draw users back for a return visit.

Information Architects: The sites developed by early Webmasters were often not well organized, but generally were collections of documents listed on a main index page, with little thought given to ease of navigation. Information architects, who were frequently librarians, archivists, marketers, or information designers, began to make order out of chaos, performing a number of functions that would prove to be increasingly vital to the success of a Web site.

As a discipline, information architecture did not really exist before the Web. Content was re-organized in an intuitive, easy-to-navigate fashion, ensuring consistency of links between documents. In addition, information architects created scalable infrastructures, encouraging information owners to contribute content, and ensuring that there would always be a place for new information. Business strategists and visionaries: As the Web moved from a grass-roots effort to being recognized and supported by institutional resources, the role of the evangelist moved into the spotlight. Business plans and visions for the future of marketing and sales on the Web were needed to sway the skeptics and secure resources.

As the Web left 1994 behind and continued its exponential growth into 1995, several new server technologies were courting additional members to the Webmaster team. The introduction of Secure HTTP (SHTTP) by EIT, and the Secure Sockets Layer (SSL) by Netscape, moved the Web beyond a simple marketing medium, raising the idea of conducting business on the Web from a specter to more of a reality.

With Web-based commerce, however, came a whole new set of issues. The Webmaster team grew to include members of the legal department, who brought with them domain expertise about tax laws, and the complications that would inevitably raise their ugly heads in the world of international commerce. Members of the sales force also joined the Webmaster team, guiding business strategies, providing domain expertise on how to integrate this new sales channel with existing ones, and acting as evangelists within the sales organization.

Continued developments in server technology over the past two years have led to tighter integration with databases, via CGI-driven interfaces, as well as through servers from database companies like Oracle and Lotus. To best use the newly enhanced abilities to collect information about visitors and to make legacy data available to visitors, programmers moved from being unsung heroes of the Webmaster team into mission-critical roles.

Marketing Webmasters and information architects were now relying on the programmers to build Web sites that could collect information on visitors, ultimately providing them with customized, personalized views of the information available on and through the Web site. Additionally, the growth in Intranets meant that the programmer was playing an important role in interfacing internal Web servers with information housed in legacy databases.

While not exactly a new development in Web server technology, the increase in outsourcing of Web server hosting has rearranged some roles within the Webmaster team. The technical Webmaster is freed from the day-to-day responsibilities of the care and feeding of the Web server, and can instead focus on designing host architectures that will scale to meet the site’s needs, and can work more closely with the programmers to incorporate new technologies into the Web site. The role of the evangelists shifts into a mode where they must convince the rest of the team of the value of outsourcing, to allow the team to focus on their core competencies of building and promoting the site. The business strategists have the new responsibility of spending the money saved by outsourcing the hosting and management of the Web site.

What Does The Future Hold?

While the concepts upon which the World Wide Web is based may be over fifty years old, the Web has only grown into a usable medium over the past five years. During this time, the role of the Webmaster has been created, and has grown from the technologically-curious individual into a multi-disciplinary team. Also during this time, Web server technology has grown from simply delivering the texts of HEP papers to complex, database-driven entities that deliver text, applets, and all manner of multimedia, both static and streaming. Where will future developments in Web server technology be occurring, and what do they mean to the Webmaster?

HTTP/1.1, approved to become a proposed standard (RFC 2068), contains a number of interesting new features. One that is certain to have an impact on Webmasters is the enhanced content negotiation available for preferred character sets and languages. Giving users the ability to assign preference levels to the language and character sets they wish to receive content in will likely lead to an increasing need for multiple versions of content. Providing a means for the Web to truly be “world wide,” Webmasters will start including internationalization, translation, and language experts as part of the team.

Thanks to “embedded” technologies being developed by companies like Spyglass, Web servers will soon be finding their way out of desktop workstations, and into such unlikely devices as cars, laser printers, and medical instruments. According to the Spyglass Web site, “The benefits of Web-enabled devices are tremendous: more efficient processes, increased convenience, time and money savings.”

It is interesting to consider what functions the Webmaster of an embedded Web server will perform, if any. These Webmasters will be evangelists, convincing others of the benefits of Web-enabled devices. They will probably also serve the role of designer and information architect, determining what information from the device needs to be served, and how it should look to the user. The technical Webmaster will also continue to play an important role, integrating the embedded Web server technology into a wide variety of electronic devices.

Terry Swack, vice president of the Webmasters’ Guild notes, “If the technology is being embedded, you also have to embed the intelligence.” Swack feels that these tools will be included as part of the Web server technology, possibly as AI components, allowing Webmasters to become agents in these new virtual spaces.

David Belson is an active member of the Webmasters’ Guild. He is an Internet Sales Engineer for BBN Planet, and is pursuing a Master’s Degree in Technical and Professional Writing at Northeastern University in Boston, MA.