Kevin Sullivan
Associate Professor, University of Virginia and
Visiting Scientist, Software Engineering Institute, Carnegie-Mellon University

151 Engineer's Way
P.O. Box 400740,
Charlottesville VA, 22904-4740 USA
E-mail: sullivan[AT]cs.virginia.edu 
Telephone: +1 (434) 982-2206
FAX at +1 (434) 982-2214

Most recent paper: Kevin Sullivan, Adaptation Architectures, DESRIST 2008.

What do I think about?

One of the big questions I think about is how to relate the design of complex software and software-intensive systems to business goals and value objectives in . general. I have been working in particular on how to understand the economic value of modular (software) design architectures. Architectures create value in at least two ways. First, they can support the delivery of certain properties for which people are willing to pay. Second, they create options to make follow-on investments to adapt or evolve a design to deliver even more valuable properties. Figuring out to put a value on these options--on the flexibility to vary or adapt a design--is a real challenge, especially to assign a value when benefits are uncertain. How much should one invest in architecture? The emerging field of real options had the potential to provide an intellectual and mathematical handle on such questions. That said, traditional real options techniques (e.g., using Black-Scholes or binomial options pricing techniques) cannot be used directly because they make deep assumptions about the nature and measurability of the underlying uncertainties. These assumptions are generally not valid in the setting of design of unprecedented systems. A lot of what I think about today is how develop and validate options-like models for valuing investments in modular design architectures without falling into one of these traps. I was also a co-author of the recent report from the Software Engineering Institute on Ultra-Large-Scale Systems. Related questions of efficient evolution of very complex systems are central in that report.

CS 101 - Fall 2007

• Welcome 101 students! I will teach CS 101 in the Fall of 2007. There are three distinct sessions and versions of this course. For more information, please see the CS101 web site.

Recent News

• I appear as #24 in a ranking of the "Top 50 Software Engineering Scholars" internationally in an article in the June, 2007 Communications of the ACM (article is here). This article presented a publication-based framework for assessing impact, focusing in particular on publications in the recognized top journals and conference proceedings in the field.
• My colleagues at UC San Diego, Vanderbilt University and Michigan State University and I are planning an NSF Industry-University Collaborative Research Center on Ultra-Large-Scale - ULS - Software-Intensive Systems. In a related vein, along with Doug Schmidt at Vanderbilt and Dick Gabriel at Sun Labs, I was an outside member of the expert and author teams for the report, Ultra-Large-Scale (ULS) Systems: The Software Challenge of the Future. I continue to work with the Software Engineering Institute as a consulting Visiting Scientist on the ULS Systems project.
• Here's a Software Engineering Radio interview (with me etc.) on ULS Systems:: http://se-radio.net/index.php?post_id=172537.
• I'm the organizing chair for the 2007 NSF Science of Design PI meeting.
• I'm organizing two workshops for ICSE: Economics of Software and Computation, and Ultra-Large-Scale Systems.
• Here are the slides for my OOPSLA 2006 Tutorial on Modular Aspect-Oriented Programming with XPI's. (Permission is granted to copy and view for non-commercial research and teaching purposes; and OOPSLA tutorial T13 attendees may make one copies and a second for backup purposes; all other rights reserved; Copyright 2006.)

Biographical Sketch:

Kevin Sullivan received his undergraduate degree from Tufts University in 1987 and the MS and PhD in Computer Science and Engineering from the University of Washington in 1994. His PhD advisor was David Notkin. He is now Associate Professor and Virginia Engineering Foundation (VEF)  Endowed Faculty Fellow in computer science at the University of Virginia, where he has worked since 1994. Kevin's research  interests are in software-intensive systems, in general, and in software engineering and languages. more specifically. He recently served as associate editor for the Journal of Empirical Software Engineering and the ACM Transactions on Software Engineering & Methodology, and on the program and executive committees of conferences including the ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE), the International Conference on Software Engineering (ICSE), Aspect-Oriented Software Development (AOSD) and ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL).

Research Interests:

My students and I are broadly interested in the design and engineering of software-intensive systems, with an emphasis on the need for a value-based theory and practice of system design.  We have two relatively independent areas of research activity under this theme.

• We are particularly interested in value-based software engineering, with a particular focus on the nature and value of modularity in design. In this regard, we are working on formal theories and representations of modularity, models of the option-like economic value of modularity, technical models of software components and component integration, large-scale reuse, design evolution, and novel approaches to modular design, including aspect-oriented development.
• We are also interested in improving our ability to assure the trustworthiness of software-intensive systems. In this regard, we collaborate with Joanne Dugan in the area software tools for reliability modeling and analysis of fault-tolerant computer-based systems. The experimental evaluation of formal methods of software specification and verification is a key software engineering component of that project, with efforts in specification-based testing as a particular emphasis.

I also lead a major new (2004) multidisciplinary initiative funded by the Commonwealth of Virginia and the University of Virginia's Vice President for Research. While the specific formulation of the center theme is evolving, it can be said that in general the focus is on failure avoidance in next-generation, software-intensive systems, with a particular emphasis on anticipating, avoiding, and managing failures due to malicious environments.

Selected Recent Professional Activities:

• PI, planning grant to establish and direct NSF Industry/University Collaborative Research Center for Ultra-Large-Scale Systems (unofficial, 2007)
• Program Committee, International Conference on Generative Programming and Component Engineering (GPCE'07), Salzburg, Austria
• Program Committee Program Committee, IEEE 2007 International Conference on Web Services (2007)
• Organizing Committee, co-Founder, ICSE Workshop on the Economics of Software and Computation (2007)
• Organizing Committee, co-Founder, ICSE Workshop on Ultra-Large-Scale Software-Intensive Systems (2007)
• Program Committee, Software Engineering Properties of Languages and Aspect Technologies (AOSD 2007)
• Breakout Chair/Co-Organizers, Program Committee, NSF Cyberphysical Systems Workshop, December 2006
• Lockheed Martin Software Wind Tunnel Workshop, Invited Outside Expert, December 2006
• NSF/EPSCOR pre-proposal reviewer, December 2006
• Program Committee, National Science Foundation Workshop on Cyber-Physical Systems, Oct 2006
• Organizing Committee, PI, National Science Foundation Science of Design PI Meeting, 2007
• NSF HCSS Planning Workshop on Cyber-physical Systems, Invited Participant, 2006
• Workshop on Ultra-Large-Scale Systems, co-organizer, OOPSLA 2006
• Panel on Ultra-Large Scale Systems, panelist OOPSLA 2006
• Panel on Modularity, panelist, OOPSLA 2006
• Author Team, Ultra-Large-Scale Systems: The Software Challenge of the Future, Special Report of the Software Engineering Institute, 2006
• Tutorial, Modular Aspect-Oriented Programming with Crosscut Programming Interfaces (XPIs), OOPSLA 2006
• Program Committee, Dependencies and Interactions Workshop, European Conference on Object-Oriented Programming (ECOOP), 2006
• Program Committee, Principles of Programming Languages (POPL), 2007
• Program Committee, International Conference on Software Engineering (ICSE), 2007
• Program Committee, .NET Technologies 2006, May 29 - June 1, 2006, Prague
• Program Committee, Software Engineering Properties of Aspect Languages and Technologies, Bonn, March 2006
• Research Proposal Review Committee, Science Foundation of Ireland, 2005, 2006
• Steering Committee, 2006 Interoperability Workshop, National Coordination Office for Networking and IT R&D (NITRD), Software Design and Productivity
• Expert, Ultra-Large-Scale Systems Study, Software Engineering Institute, a study commissioned by Hon. C. Bolton, Assistant Secretary, U.S. Army
• Program Committee, Emerging Results Track, International Conference on Software Engineering (ICSE) 2006
• Workshop Co-Chair (with Rick Kazman), 8th Annual Workshop on Economics-Driven Software Engineering Research (ICSE) 2006
• Invited Keynote Presentation, Foundations of Object-Oriented Languages/Workshop on Object-Oriented Design (at POPL 2006)
• Invited Keynote Presentation, Software Engineering Today a SIGS/Datacomm Conference (SET) 2006
• Associate Editor, ACM Transactions on Software Engineering and Methodology (expired 2005)
• Associate Editor, Journal of Empirical Software Engineering (expired 2005)
• Executive Committee, 2005 International Conference on Software Engineering
• Program Committee, 2005 International Conference on Software Engineering
• Program Committee, 2005 Conference on Aspect-Oriented Software Development
• Program Committee, 2005 Workshop on the Foundations of Aspect-Oriented Languages
• Workshop Chair, 7th International Workshop on Economics-Driven Software Engineering Research
• Interim Director of Virginia Laboratory for Dependable and Secure Computing (tentative name)
• University of Virginia Committee for Review of Scientific and Engineering Research Proposals
• School of Engineering and Applied Science Committee on Graduate Studies
• Computer Science Department Assistant Director of the Graduate Program

Selected Software Projects:

•     Galileo: Software Engineering of Software Tools for Engineering
•     Eos: Improved Abstractions for Aspect-Oriented Development

NSF Workshop on the Science of Design:

In November 2003 I ran an NSF workshop on Science of Design. The results, including position papers, are available here. We are setting up a new web site for the science of design community. Visit it here. It includes suggested readings.

Selected Publications:

Books and Book Chapters

• Sullivan, K.J., “Designing Models of Modularity for Component Software,” in Heineman and Councill, eds., Component-Based Software Engineering, Addison-Wesley, to appear, 2001.

• Sullivan, K.J., P. Chalasani, S. Jha and V. Sazawal, “Software Design as an Investment Activity: A Real Options Perspective,” in Real Options and Business Srategy: Applications to Decision Making, L. Trigeorgis, consulting editor, Risk Books, November, 1999.

• Sullivan, K.J., Salman, M., and Van Evera, S. ``SIOP: A Computerized Nuclear Exchange Model for Civilian Defense Analysts,'' in Eden, L. and Miller, S., eds., Nuclear Arguments, (Ithaca: Cornell University Press), 1989.

• Salman, M., Sullivan, K.J., and Van Evera, S. ``Analysis or Propaganda: Measuring American Strategic Nuclear Capability, 1969-1988,'' in Eden, L.  and Miller, S., eds., Nuclear Arguments, (Ithaca: Cornell University Press), 1989.

Refereed Journal Publications

• G. Murphy, D. Notkin and K. J. Sullivan, Software Reflection Models: Bridging the Gap between Design and Implementation, IEEE Transactions on Software Engineering, vol. 27, no. 4, April 2001, pp. 364-380.

• J.B. Dugan, K.J. Sullivan and D. Coppit, “Developing a Low Cost High Quality Software Tool for Dynamic Fault Tree Analysis,” IEEE Transactions on Reliability, Vol. 49, No. 1, March 2000, pp. 49 – 59. 

• B. Boehm and K.J. Sullivan, “Software Economics: Status and Prospects,” Special Millenium Issue, Information and Software Technology, January 2000.

• G. Murphy, D. Notkin and K.J. Sullivan, “Software Reflexion Models,” IEEE Transactions on Software Engineering 27,4, 2001, pp. 364-380.

• K. Vemuri, J.B. Dugan and K. Sullivan, “Automatic Synthesis of Fault Trees for Computer-Based Systems,'” IEEE Transactions on Reliability, December 1999, pp. 394 – 402.

• K.J. Sullivan, M. Marchukov and J. Socha, “Analysis of a Conflict Between Aggregation and Interface Negotiation in Microsoft's Component-Object Model,” IEEE Transactions on Software Engineering, July/August 1999, pp. 584–599.

• I.J. Kalet, J.P. Jacky, M.M Austin-Seymour, S.M. Hummel, K.J. Sullivan and J.M. Unger, ``Prism: a New Approach to Radiotherapy Planning Software,'' International Journal of Radiation Oncology, Biology and Physics, 36, 2, 1996, pp. 451--461.

• K.J. Sullivan, I.J. Kalet, and D. Notkin, “Evaluating The Mediator Method: Prism as a Case Study,” IEEE Transactions on Software Engineering, 22, 8, August, 1996, pp. 563--579.

• Sullivan, K.J. and Notkin, D., ``Reconciling Environment Integration and Software Evolution,'' ACM Transactions on Software Engineering and Methodology 1, 3, July, 1992, pp. 229-268.

Refereed Conference Papers

• H. Rajan and K. Sullivan, “Aspect Language Features for Concern Coverage Profiling,” to Appear, Fourth International Conference on Aspect-Oriented Software Development (AOSD), 2005.

• H. Rajan and K. Sullivan, “Classpects: Unifying Aspect- and Object-Oriented Program Design,” to appear, International Conference on Software Engineering (ICSE), 2005.

• K. Sullivan, J. Yang, D. Coppit, S. Khurshid, D. Jackson, “Improving Software Assurance by Bounded Exhaustive Testing,” 2004 International Symposium on Software Testing and Analysis (ISSTA), Boston, July, 2004.

• Hridesh Rajan and Kevin Sullivan, “Eos: Instance-Level Aspects for Integrated System Design,” Proceedings of Joint European Software Engineering Conference and ACM Symposium on the Foundations of Software Engineering (ESEC/FSE) 2003, Helsinki, Finland, November 2003.

• D. Coppit, R. Painter, and K. Sullivan, “Shared Semantic Domains for Computational Reliability Engineering,” International Symposium on Software Reliability Engineering, Denver, Colorado, November, 2003, pp. 169--180.

• J.C. Knight, E. Strunk, and K.J. Sullivan, “Toward a Rigorous Definition of Survivability,” DISCEX 2003, April 22-24, 2003.

• D. Coppit and K. Sullivan, “Sound Methods and Tools for Engineering Modeling and Analysis,” International Conference on Software Engineering, Portland, OR, May, 2003.

• K.J. Sullivan, Yuanfang Cai and Lin Gu, “Nonmodularity in Aspect Languages: AspectJ as a Case Study,” First International Conference on Aspect-Oriented Software Development,” 2002.

• K.J. Sullivan, W.G. Griswold, Y. Cai and B. Hallen, “The Structure and Value of Modularity in Software Design,” Joint Proceedings of the European Software Engineering/ACM SIGSOFT Foundations of Software Engineering Conference (ESEC/FSE), Vienna, September 2001.

• K.J. Sullivan and A. Saxena, “An Architectural Aspect for the Emerging Computational Tapestry,” 2001 International Conference on Software Engineering,” May 2001.

• K.J. Sullivan, M. Shaw, B. Boehm, D. Notkin and W. Harrison, “Third International Workshop on Economics-Driven Software Engineering Research,” Proceedings of the 23^rd International Conference on Software Engineering (ICSE), May 2001, p. 770.

• D. Coppit, K.J. Sullivan and J.B. Dugan, “Formal Semantics of Models for Computational Engineering: A Case Study on Dynamic Fault Trees,” 2000 International Symposium on Software Reliability Engineering, October, 2000, pp. 270-282.

• H. Zhu, S. Zhou, J.B. Dugan and K.J. Sullivan, “A benchmark for Quantitative Fault Tree Reliability Analysis,” 2001 Proceedings of the Annual Reliability and Maintainability Symposium, January, 2001, pp. 86 – 93.

• D. Jackson and K.J. Sullivan, “COM Revisited: Tool-Assisted Modeling and Analysis of Complex Software Structures,” SIGSOFT 2000, ACM SIGSOFT International Symposium on the Foundations of Software Engineering, November, 2000.

• Coppit, D. and K.J. Sullivan, “Mass-Market Packages as Software Components,” Proc. 2000 International Conference on Software Engineering, Limerick, June 5–8, 2000.

• K.J. Sullivan, “The 2^nd International Workshop on Economics-Driven Software Engineering Research,” Proceedings of the 22^nd International Conference on Software Engineering, June 4 – 11, 2000, p. 811.

• B. Boehm and K.J. Sullivan, invited paper, “Software Economics: A Roadmap,” 22nd International Conference on Software Engineering, June, 2000.

• J.C. Knight, K.J. Sullivan, M. Elder, “Survivability Architectures,” Proceedings of DARPA Information Survivability Conference and Exposition (DISCEX), January 25—27, 2000.

• J.B. Dugan, K.J. Sullivan and D. Coppit, “The Design of a High Quality Fault Tree Analysis Tool,” Proceedings of the International Symposium on Software Reliability Engineering, November 1999.

• M.A. Copenhafer and K.J. Sullivan, “Tool Supported Exploration of Large Component Properties,” Proceedings of the Automated Software Engineering Conference, pp. 7—14, October, 1999.

• K.J. Sullivan, D. Coppit, and J.B. Dugan, “The Galileo Fault Tree Analysis Tool,” Proceedings of the 29th International Symposium on Fault Tolerant Computing, pp. 232--235, June 1999.

• K.J. Sullivan, J.C. Knight, X. Du and S. Geist, “Information Survivability Control Systems,” Proceedings of the 21^st International Conference on Software Engineering, pp. 184--193, May 1999.

• R.W. Manian, D. Coppit, K. Sullivan and J.B. Dugan, “Bridging the Gap Between Fault Tree Analysis Modeling Tools and the Systems Being Modeled,” Proceedings of the Reliability and Maintainability Symposium, January, 1999.

• K.K Vemuri, J.B. Dugan and K. Sullivan, “A Design Language for Automatic Synthesis of Fault Trees,” Reliability and Maintainability Symposium, January 1999, pp. 91--96.

• D. Coppit and K.J. Sullivan, Formal Specification in Collaborative Design of Critical Software Tools,” Proceedings of the High Assurance Systems Engineering Symposium, 1998, pp. 13--20.

• R. Manian, J.B. Dugan, D. Coppit and K.J. Sullivan, “Combining Various Solution Techniques in Dynamic Fault Tree Analysis,” Proceedings of the High Assurance Systems Engineering Symposium, 1998, pp. 21--28.

• Sullivan, K and J.C. Socha, “Using Formal Methods to Reason about Architectural Standards,” Proceedings of the 19th International Conference on Software Engineering, Boston, May 17--24, 1997, pp. 503--513.

• Sullivan, K and Knight, J.C., “Experience Assessing an  Architectural Approach to Large-Scale Systematic Reuse,” Proceedings of the 18th International Conference on Software Engineering, Berlin, March, 1996, pp. 220--229.

• Murphy, G., Notkin, D. and Sullivan, K., “Software Reflexion  Models: Bridging the Gap between Source and High-Level Models,” Proceedings of the Conference on the Foundations of Software Engineering (SIGSOFT 95), October, 1995.

• Notkin, D., Garlan, D., Griswold, W.G., and Sullivan, K., “Adding Implicit Invocation to Languages: Three Approaches,” Proceedings of the JSSST International Symposium on Object Technologies for Advanced Software, (November 1993).  (Also appears as a Springer-Verlag Lecture Notes in Computer Science volume \#742.)

• Sullivan, K.J. and Notkin, D., “Reconciling Component Independence and Environment Integration,” Proceedings of SIGSOFT90: Fourth Symposium on Software Development Environments, 1990, Irvine, CA.

A Few Interesting Old Reads

• Vannevar Bush, "As we may think," Atlantic Monthly, July, 1945.
• Herbert Simon, "The architecture of complexity," Proceedings of the Americal Philosophical Society, Vol. 106, No. 6, December, 1962.
•