Text preview for : 19790710_Pilot_A_Software_Engineering_Case_Study.pdf part of xerox 19790710 Pilot A Software Engineering Case Study xerox sdd memos_1979 19790710_Pilot_A_Software_Engineering_Case_Study.pdf
Back to : 19790710_Pilot_A_Software | Home
Pilot: A Software Engineering Case Study
by Thomas R. Horsley and William C. Lynch
July 10, 1979
ABSTBACT
Pilot is an operating system implemented in the strongly typed language Mesa and produced
in an environment containing a number of sophisticated software engineering and
development tools. We report here on the strengths and deficiencies of these tools and
techniques as observed in the Pilot project. We report on the ways that these tools have
allowed a division of labor among several programming teams, and we examine the problems
introduced within each different kind of development programming activity (ie. source editing,
compiling, binding, integration, and testing).
XEr{OX
SYSTEMS DEVELOPMENT DEPARTMENT
3408 Hillview Ave / Palo Alto / California 94304
2 T. R. HORS[ EY A:-';l) W. C. LYt\CH
Introduction
The purpose of this paper is to describe our experiences in implementing an operating system
called Pilot using a software engineering support system based on the strongly typed language
Mesa [Geschke et of, 1977, Mitchell et 01, 1978], a distributed network of personal computers
[Metcalfe et 01, 1976], and a filing and indexing system on that network designed to coordinate the
activities of a score or more of programmers. In this paper we will present a broad overview of
our experience with this project, briefly describing our successes and the next layer of problems
and issues engendered by this approach. Most of these new problems will not be given a
comprehensive discussion in this paper, as they are interesting and challenging enough to deserve
separate treatment.
lbat the Mesa system, coupled with our mode of usage, enabled us to solve the organi7C1tional and
communication ploblems usually associated with a development team of a score of people. These
facilities allowed us to give stable and non-interactive direction to the several sub-teams.
We developed and used a technique of incremental integration which avoids the difficulties and
schedule risk usually associated with system integration and testing. .
lhe use of a Program Secretary, not unlike Harlan Mills' program librarian, proved to be quite
valuable, particularly in dealing with situations where our tools had weaknesses. We showed the
worth of the program librarian tool, which helped coordinate the substantial parallel activity we
sustained; and we identified the need for some additional tools, particularly tools for scheduling
consistent compilations and for controlling incremental integrations.
We determined that these additional tools require an integrated data base wherein consistent and
correct information about the system as a whole can be found.
Background
Pilot is a medium-sized operating system designed and implemented as a usahle tool rdthcr than as
an object lesson in operdting ~ystem design. Its con~truction WC1,) subjected to the fiscal, schedule,
and performance pressufes normally associated with an industrial enterpi isc.
Pilot is implemented in Mesa, a modular programming system. As reported in [Mitchell et 01,
1978], Mesa supports both definitions and implementing modules (sec below). Pilot is compriscd
of some 92 definitions modules and 79 implementation modules, with an average module size of
approximately 300 lines.
Pilot consists of tens of thousands of Mesa source lines; it was implemented and released in a few
months. The team responsible for the development of Pilot necessarily consisted of a score of
people, of which at least a dozen contributed Mesa code to the final result. The coordination of
fOUf separately managed sub-teams was required.
There arc a number of innovative features in Pilot, and it employs some interesting operating
system technology. However, the structure of Pilot is not particularly relevant here and will be
reported in a series of papers to come [Redell et of, 1979], [Lampson et of, 1979].
Pll OT: A SOFt WAf.( I r~:\GI:\ EfIU"G CASE Sl G"DY 3
Development Environment and Tools
The hardware system supporting the development environment is based on the Alto, a personal
interactive computer [Lampson 1979], [Boggs, et aI, 1979J. Each developer has his own personal
machine, leading to a potentially large amount of concurrent development activity and the
potential for a great degree of concurrent development difficulty. These personal computers are
linked together by means of an Ethernet multi-access communication system [Metcalfe et aI, 1976].
As the Altos have limited disk storage, a file server machine with hundreds of megabytes of
storage is also connected to the communications facility. Likewise, high-speed printers are locally
available via the same mechanism. The accessing, indexing, and bookkeeping of the large number
of files in the project is a serious problem (see below). To deal with this, a file indexing facility
(librarian) is also available through the commun ications system.
lne Alto supports a number of significant wideranging software tools (of which the Mesa system is
just one) developed over a period of years by various contributors. As one might imagine, the
level of integration of these tools is less than perfect, which led to a number of difficulties and
deficiencies in the Pilot project. Many of these tools were constructed as separate, almost stand-
alone system~.
rLne major software tools which we employed are described below.
Mesa is a modular programming language [Geschke el aI, 1977]. The Mesa system consists of a
compiler for the language, a Mesa binder for connecting the separately compiled mudules, and an
interactive debugger for debugging the Mesa programs. Optionally, a set of procedures called the
Mesa run-time may be used as a base upon which to build experimental systems.
rrne language defines two types of modules: definitions modules and implementatlOll modules.
Both of these are compiled into binary (object) form. A definitions module describes an interface
to a function by providing a bundle of procedure and data declarations which can be referenced
by c!ielll programs (cliellts). Declarations are fully type specified so that the compiler can carry out
strong type checking between clients and implementation modules. The relevant type informatioIl
is supplied to the clients (and checked against the implementations) by reading the object modllle~
which resulted from previous compilation(s) of the reIevent definitions module(s). The
implementing modules contain the procedural description of one or more of the functions defined
in some definitions module. Since an implementing module can be seen only through some
definitions module, a wide variety of implementatiuns and/or versions is pussihle without theil
being functionally detectable by the clients. Thus Mesa enforces a form of information hiding
[Parnas, 1972].
The Mesa binder [M itchell et aI, 1978] defines another language, called C/Mesa, which is capable
of defining configurations. These assemble a set of modules and/or sub-configurations into a new
conglomerate entity which has the characteristics of a single module. Configurations may be
nested and used to describe a tree of modules. Configurations were used in the Pilot project as a
management tool to precisely define the resultant output of a contributing development sub-team.
Another software tool is the Librarian. It is designed specifically to index and track the history of
the thousands of files created during the project. In addition to its indexing, tracking, and status
reporting functions, the Librarian is constructed to adjudicate the frequent conflicts arising between
programmers attempting to access and update the same module.
1'. R. HORS! t'( :\:\D W. C. LY:-"CH
Organization, Division, and Control of the Development Effort
The size of the Pilot development team (itself mandated by schedule considerations) posed the
usual organizational and management challenges. With 20 developers, a multi-level management
structure was necessary despite the concomitant human communication and coordination problems.
As described below, we chose to use the modularization power of the Mesa system to address these
problems, rather than primarily providing the capability for rapid interface change as reported in
[Mitchell, 1978]. The resultant methodology worked well for the larger Pilot team. We believe
that this methodology will extrapolate to organizations at least another factor of five larger and one
management level deeper. A description and evaluation of this methodology are the topics of this
section.
Another aspect of our approach was the use of a single person called the Program Secretary, a
person not unlike the program librarian described by Harlan Mills [Mills, 1970] in his chief
programmer team approach. As we shall describe, the Secretary performed a number of functions
which would have been very difficult to distribute in our environmcnt. This person allowed us to
control and make tolerable a number of problems, described below, which for lack of time or
insight we were not able to solve directly.
The Pilot Configuration Tree
We organized Pilot into a tree of configurations isomorphic to the corresponding people tree of
teams and sub-teams. The nodes uf the Pilot tree are C/Mesa configuration descriptions and the
leaves (at the bottom of the tree) are Mesa implementation modules. By strictly controlling the
scope (sec below) of interfaces (through use of the facilities of the configuration language C/Mesa),
different branches of the tree were developed independently. Th~ configuration tree was three to
four layers deep everywhere. The top level configuration implements Pilot itself. Each node of
the next level down maps to each of the major Pilot development teams, and the next lower level
to sub-teams. At the lowest level, the modules themselves were usually the rcsponsibility of one
person. This technique of dividing thc labor in correspondence with the con figurdtioll trcc p rm cd
to be a viable management technique and was supported effectively by Mesa.
Management Of lllterface5