Science of Design: Software-Intensive SystemsA National Science Foundation Workshop
Airlie Center, November 2-4, 2003
Science of Design: Software-Intensive Systems
The National Science Foundation, grant CCR-0346938, supported a workshop on the topic of Science of Design, with a particular emphasis on Software and Software-Intensive Systems. The workshop was held at Airlie Center, 30 miles from Dulles International Airport, November 2–4, 2003. The objectives were to develop community recommendations on proper formulations of such a science, addressing its subject, scope, and methods; to identify major open problems and opportunities; and to identify research priorities for an assumed ten-year time frame.
Advances in software and software-intensive system design over the last fifty years have created deep knowledge and great benefits for our society. Nevertheless, long-standing technical challenges remain, and the march of information technology and other forces continue to widen the gap between software and software-intensive systems in practice and our underlying scientific and engineering knowledge and educational methods. We face hard and interesting problems. Left unsolved, they threaten us. Solving them would greatly improve our lives. Software has enabled miraculous applications—improving health, education, environment, our economy, our security. To harness complex information technologies of the future reliably and economically, however, requires significant advances in knowledge creation and education.
It’s already possible, for example, for a single teenager to cause hundreds of millions of dollars worth of harm, perhaps even billions, by launching a malicious computer program into the Internet. Critical software systems are largely developed by composing third-party software components, but we lack the means to predict and certify their properties alone or in systems. The emergence of significant programming expertise around the world is creating a new need for U.S. employees to operate at a much higher design level than in the past. Business value is increasingly concentrated in software assets, but we lack the means to reason about that value and to design for value. To date, limited and fragmentary knowledge of software and software-intensive system design has sufficed. The future, in which all complex engineering systems will be software-intensive systems, demands a deeper and more unified science of design, with an emphasis on that elements which is now the most central but the least well understood: software.
The goals of such an effort include the following: To enable the design of software-intensive systems as dependable and fit to purpose as mechanical, civil, and other engineering systems typically are today; to ensure that the values and goals of the people impacted by software-intensive system are accounted for rigorously in their design and promoted effectively by their operation; to develop rich behaviors at complex interfaces between software-intensive systems and their human and physical environments; to enable automated tools based on sound science to replace costly and error-prone human activities in software design; to leverage knowledge of design from other fields; to develop design methods for emerging technology platforms; to understand how to design software-intensive systems as social agents; to understand the characteristics of successful designs and how they are achieved; to learn how to represent designs at a much higher-level than code but nevertheless rigorously embody the key constraints; to discover how multi-disciplinary communities of designers and users can communicate to jointly advance design processes; to devise methods to increase the design content of practitioner code and so increase the degree to which working with the code is working at the design level; and, crucially, to educate students in the fundamentals of design in all dimensions, with a particular emphasis on teaching effective software and software-intensive systems design.
The workshop organizers sought to define a scope satisfying several criteria. First, it had to be broad enough to consider the sciences of design, in general, in a sense such as, but not limited to, that advocated by Herbert Simon in his seminal book, The Sciences of the Artificial [1]. Second, the focus had to be on fundamental problems in software and software-intensive systems design, in particular. Third, recognizing that design theory is not a new idea and that it has been treated very seriously in disciplines beyond computer and information science and engineering, the workshop was designed to strongly encourage participation of design scholars from fields such as engineering design, architecture, and the social sciences, including economics.
It was also stipulated that the term design be taken in a design theoretic sense, referring to the whole process of problem framing, and conceiving, representing, evaluating, and evolving solutions; and not in the narrow sense sometimes used in software engineering, as referring merely to program structuring subsequent to specification and prior to coding. The broader, historically established interpretation recognizes the crucial importance of technical issues, but also the central relevance of extra-technical values, goals, conditions, constraints, and creativity.
The workshop objectives were to develop positions on the proper formulations of a science of design, with a clear emphasis on software and software-intensive systems, addressing subject, scope, methods, norms, and the like; to identify major open problems and opportunities; and to develop guidance on research priorities for ten-year time frame.
It was considered vitally important to open the workshop to participants from all relevant disciplines, both within and beyond computer science, and to have a broad range of viewpoints represented. A program committee of distinguished scholars was formed, with representatives from relevant areas of computer science and from engineering design and the social sciences. A call for position papers was disseminated widely, seeking contributions from both U.S. and international participants. Over 140 position papers were submitted. All papers were subject to review by all members of the organizing committee. Invitations were based on committee evaluations of the papers and on diversity considerations. The workshop had over 60 invited participants, in computer science, engineering design, architecture, and economics, coming from the United States, Europe, and Israel. NSF personnel and student volunteers were also present.
The program committee members were Carliss Baldwin, Harvard Business School; Fred Brooks, University of North Carolina, Chapel Hill; Clive Dym, Harvey Mudd College; Cordell Green, Kestrel Institute; Michael Jackson, Consultant; Alan Kay, HP Labs, Viewpoint Research Institute, and the University of California, Los Angeles; Gregor Kiczales, the University of British Columbia; Greg Morrisett, Harvard University; Jakob Nielsen, Nielsen Norman Group; David Notkin, University of Washington; Mary Shaw, Carnegie Mellon University; Kevin Sullivan, University of Virginia (Workshop Chair); and Richard Taylor, University of California Irvine.
An organizing principle of the workshop was to avoid imposing a preconceived structure on the outcome, to allow the represented community to find its own voice. Therefore no attempt was made to impose tracks or categories on the discussion at the outset. Rather, the workshop process was intended first provoke the participants to identify key issues, and then to permit them to develop their positions in a form that could be incorporated into this report.
The workshop began in earnest the morning of November 3 with short talks from selected participants. The talks were expressly not keynotes, but were meant to provoke an initial round of discussions. Following the talks, participants were organized into parallel break-out groups, each given the same charge: formulate the problems and research ideas that will become the results of the workshop. Each group reported its findings in a plenary session. The reports were integrated by the workshop organizers to identify topics for an afternoon round of topic-specific break-out groups. Those break-out groups then reported back in an evening plenary session.
On the second day of the workshop, in a marketplace of ideas, individuals were asked to identify priority areas for research, consistent with the previous day’s results, by holding up banner signs with keywords of their choice, and—reading each others’ signs—to self-organize into affinity groups of up to about four members. Each group was then given several hours to produce a written document articulating the critical problems and research priorities in its area. Those short papers are the main source of the findings presented in the next section of this report.
Individuals were also asked to provide short written statements of their views, independent of their groups, and written dissents from group conclusions were also solicited.