Monday, August 24, 2009
Ross Gore
Chair: Worthy Martin; Jack Davidson; Kevin Sullivan; Steve Turner and Preston White
Advisor: Paul Reynolds
OLSSON 228E, 09:00:00
A Ph.D. Proposal
Understanding Unexpected Software Behaviors
ABSTRACT
Software for exploration and insight has entered the mainstream of critical public policy and research decision-making practices. Unfortunately, methods for gaining insight into unexpected outcomes in that software, as related to design, implementation and use, have not kept pace. Often users must explore whether unexpected behaviors reflect an error or an unexpected, but valid, behavior of the system. Common practice is to apply classic debugging techniques to identify the program statements and interactions that lead to the unexpected behavior. Subsequently an explanation for the behavior is formed and code is modified. The user iterates this process until satisfied. This process is manual and time-consuming. Considering the constituent parts of the process: (1) hypothesizing about relationships among program states, (2) altering and experimenting with program and data assumptions, and (3) determining the causal chains that lead to unexpected behaviors, we argue the process could benefit from a significant degree of automation notcurrently employed. Thus, we propose INSIGHT, a novel automated methodology facilitating explanation of unexpected behaviors in deterministic and stochastic exploratory and insight producing software. INSIGHT leverages work in the debugging, program comprehension and program understanding communities to produce novel methods for analyzing deterministic and stochastic exploratory and insight producing software. INSIGHT draws on technologies such as semi-automated search, causal inferencing and program slicing. It will bring automation to each part of the process of hypothesizing, experimenting and drawing causal conclusions to support explanation of unexpected behaviors in exploratory software. The effectiveness of INSIGHT will be evaluated using established methods for evaluating human productivity tools. Evaluation methods to be used include human user studies. INSIGHT's effectiveness will be compared to that of the leading debugging, program comprehension and program understanding tools and it will be analyzed by a small set of application experts for effectiveness, usability and practicality.