CS551/651: Advanced Modeling and Simulation

 

 

 

 

 

 

 

 

 

 

 

Schedule and Assignments, Spring 2006 

version 2006034030.1501

 

There are two types of assignments:  reading, and projects.

Reading will be assigned weekly.  Please bring the reading materials, or at least your notes on those materials, to class on the day(s) we intend to discuss them.

Projects will be assigned four times during the semester and you will typically have a couple of weeks to complete them. Your four projects will form a sequence, based on a common theme.

You are expected to have read the web material listed for a given class and tested your skills before class.  The quality of class discussion (and of your final evaluation) will depend on your preparation.

This page will be changing as I add material for the rest of the semester.  Track the version number above.

Note: Reading is to be completed by the day it is listed below.

Where are we?

 

Week 1 (18 Jan): Introduction

 

     What you should know to be in this class

     Overview of the course

 

     Reading:    None - first class

 

 

Week 2 (24 Jan & 26 Jan): Challenges overview.

     Three years ago there was a workshop to identify the grand challenges facing the modeling and simulation community.  As with any workshop the product has a strong bias towards the research interests of the participants.  Even so, the Dagstuhl report covers the bulk of important M&S issues and remains quite relevant three years later.  Read through this report and be prepared to discuss it.  Yes, it's 75 pages long.  I suggest you read the entire report, focusing on portions that interest you the most.  They can be the parts you expect to discuss actively in class.

     More recently, Paul Davis of Rand Corporation has ruminated on new paradigms and challenges in modeling and simulation.  Paul's focus is primarily in military simulation, but his observations are universally applicable.  His paper is here.

A couple of additional sites regarding grand challenges in M&S that are of interest:  LLNL (here's a pdf of the site for easy printing) and simguy.

 

Week 3 (31 Jan & 2 Feb): Model Semantics and Composability

     Model semantics play a role in just about everything we want to do with models: compose them, federate them, operate them at multiple levels of resolution...   Explore the website for DAML, an ongoing (soon to end) project to develop a structure for ontologies for the web.  DAML is not focused specifically at M&S, but you'll find links proposing that it be used for describing model semantics.  Also, there's a subpage with a whole collection of DAML ontologies --items such as "agents" and "robots" apply immediately to our interests.  There's a link to a paper in Scientific American by Tim Berners-Lee that you may want to look through.  Your objective when browsing the DAML webpages is to determine the suitability of the DAML effort for describing model semantics (so that users of DAML semantic descriptions could perhaps automate plug-and-play of multiple simulations, or at least provide a means for describing and later understanding everything important about a model).

     Also, look at the W3C site, in particular, OWL.  OWL is a follow-on to DAML.  As with the DAML site, there's a lot there to consume.  Read for a basic understanding of what OWL is about, and how the technology could apply to the description of model and simulation semantics (meaning).

Next is Composability, beginning with  

     DMSO overview https://www.dmso.mil/public/warfighter/cmse/,

     Davis (long, so skim) report  www.rand.org/publications/MG/MG101/MG101.pdf  and

     Petty Taxonomy:    http://www.cs.virginia.edu/~rgb2u/03S-SIW-023.doc)  

Here are more papers, addressing the complexity and theory of composability. First one provides the background.  Fox and Bartholet are recent papers out of the UVa MaSTRI group.


     http://www.informs-cs.org/wsc99papers/078.PDF

     http://www.cs.virginia.edu/~rgb2u/03S-SIW-054.doc

     http://www.cs.virginia.edu/~mrm/publications/petty_validity_of_models.pdf

     Michael Fox

     Rob Bartholet

 

 

 

Week 4 (7 Feb & 9 Feb):  Multi-resolution modeling

 

       On to Multi-Resolution Modeling.  Most of the papers we'll read have been authored by Paul Davis of Rand Corporation.  Dr. Davis has done more for the field of MRM than anyone else on the planet. 

The following papers will give you an introduction to multi-resolution, multi-perspective, variable resolution modeling.  The Reynolds paper has a section on reasons popular alternatives are insufficient/inadequate, and goes on to discuss conditions that must be met in order to achieve multi-resolution models. 

                Reynolds:  Consistency Maintenance in Multiresolution Simulations

The follow-on publication for this work was a 308 page dissertation by Natrajan on how to build consistent MRM simulations.  I give you a presentation on the results but won't ask you to read the dissertation.  If you're interested, let me know.  We released it as a technical report.

     The Davis papers are lengthy but chock full of great insights:

                Davis #1:   Exploratory Analysis and a Case History...

                Davis #2:   http://www.rand.org/publications/MR/MR1570/index.html

     We will discuss the Reynolds paper first, followed by the Davis papers in the order listed.  We'll probably only get through the Reynolds and first Davis paper this week.  We'll do Davis #2 next week.

 

Project 1 Assignment:  Due at beginning of class on 21 February.

     Read this paper A Case Study of Context Assumptions for Simulation Composability and Reusability and do the following:  Describe how you would define the semantics of the falling body problem using OWL/RDF.  Include a sufficient number of examples using OWL/RDF to demonstrate your intended approach clearly. You do NOT have to do an exhaustive capture of the semantics of the model in OWL/RDF.  Just convince me your approach could lead to one.

 

 

Week 5 (14 Feb & 16 Feb): Multi-resolution modeling cont.

         We'll be covering this paper on Tuesday of this week:

                Davis #2:   http://www.rand.org/publications/MR/MR1570/index.html

                        (already assigned above)

 

 

 

Week 6 (21 Feb & 23 Feb): COERCE, Parallel and distributed simulation overview

Project 1 due at beginning of class on Tuesday.

Here are some papers that introduce the concept of COERCE, being developed by the UVa MaSTRI group:

             Sarah Waziruddin et al: " The Process for Coercing Simulations "

             Joseph C. Carnahan et al: " Simulation-Specific Characteristics and Software Reuse "

             Paul Reynolds et al: "  Capturing Scientists’ Insight for DDDAS  " (submitted to ICCS 7 Feb 2006))

             Sarah Waziruddin et al:  " Selecting Optimization Techniques in Support of Simulation Transformation  "

             Lingjia Tang:  " Assessment of Optimization Methods for Chained Explorations "  (coming very soon)

 

  This paper is a respectable survey of parallel and distributed simulation:

             Vee, Hsu: Parallel Discrete Event Simulation: A Survey

  

 

 

 

 

Week 7 (28 Feb & 02 Mar):   HLA, Object management, Time management

 

   We have optimistic time management in parallel simulations to finish this week on Tuesday, and then we'll start on the DoD High Level Architecture, also on Tuesday.

       To begin studying the HLA, there are three papers to review.  It's over 150 pages of reading, so be smart about it!  The objective is for you to learn about the HLA - what it is and why it exists - and important pieces of it. The first paper (Dahmann) is an overview of the High Level Architecture.  Your goal is to get a general sense of the HLA, and to see where Object Model Templates (OMT), Simulation Object Models (SOM) and Federation Object Models (FOM) fit in.  I recommend you read this paper in detail.

        The second paper is the precursor (version 1.3) to an IEEE standard (1516.2) for Object Model Templates.  It will give you a deeper view of OMT and SOMs.  Get a feel for how the OMT works.

        The third paper is a masters thesis from the Naval Post Graduate School that addresses using OMT and SOM rules to build a SOM for a venerable DoD model.  You don't need to focus on Janus, the model, when reading through this thesis.  Get a feel for how SOMs are used to capture simulation semantics (and how they are deficient).

     Here's more on SOMs/FOMs and Object Management:    HLA_1516_M1_P1.pdf,  (HLA_1516_M1_P2.pdf),  (HLA_1516_M1_P4.pdf),  (HLA_1516_M1_P5.pdf),  and (HLA_1516_M1_P6.pdf)

% % % % % % % % % % % % % %

     After SOMs/FOMs/MOMs and OMT we'll discuss time management in distributed simulations, as reflected in the HLA approach.  Since we studied TM in parallel simulation ingeneral last week, we won't need to dwell on this topic too long.  Cal Chico offers two modules on HLA time management, both in the advanced topics section.  Read over TimeManagement1.pdf  and  TimeManagement2.pdf.  Also read Fujimoto's Simulation paper:  (PDF 84K).   If you're interested in issues the design team struggled with, look at this document  HLA-TM-1.0.pdf.

As you read through the TM material, ask yourself if you agree with the design decision that placed the bulk of TM responsibility into the federates.  Alternatively, should the federates have been given even more of the burden?  Work through an imaginary scenario where an environment simulator and an instrumented human are federated and consider the costs associated with maintaining correct ordering of events between them using an HLA approach.

 

Project 2 assigned.  See this write-up.

 

SPRING BREAK  3 - 12 March

 

Week 8 (14 Mar & 16 Mar): HLA Data distribution management, Midterm

 

     So we have a little catching up to do.  Tuesday's topic will be HLA FEDEP, OMT and object management.  We'll see if we get into HLA time management before the midterm.  Reading for Tuesday's topics is listed above.

 

    Here are links to some documents that can bring you up to date on the IEEE 1516 standard for the HLA and other recent events:

    *  The Lutz paper on 1516 upgrades to the OMT

    *  The late 2005 paper on transitioning to the 1516 standard

    *  The review for NASA on using the HLA

    *  The Chronos effort in the gaming community

    *  The DMSO webpage where verification efforts are tracked

    Our last HLA topic is on data distribution management.  Cal Chico offers two modules on distribution management, both in the advanced topics section.  Read over DDMPart1.pdf   and DDMPart2.pdf.   Also read (Cohen):  Ayani.pdf,  Srinivasan:  Srinivasan.pdf   and the Mak DDM overview. 

Be prepared to answer these questions:

               1.  Explain the difference between region based and cell (grid) based DDM.

               2.  Given the Ayani work, why do you suppose MAK chose a region based approach?

               3.  What happens on the grid approach if cells are really large?  Really small?  Does this suggest hybrid region/cell approach that might be better than either one alone?

               4.  Given a federation description, do you believe it's possible to build a tool that automatically determines the best way to do DDM for that federation?

   

 16 March:  MIDTERM.  Take home, open book.   Exam will be available at classtime on the 16th.

 

 

Week 9 (21 Mar & 23 Mar):  DDDAS

 

Midterm exam due at beginning of class on Tuesday

 

 On Tuesday we will discuss the questions on the midterm and your thoughts about them.

 

Project 2 due at beginning of class on Thursday.

 

  On Thursday we will finsh DDM in the HLA and then start on DDDAS.  DDDAS is a program run by Dr. Frederica Durema at the National Science Foundation.

Here's a pair of summaries of DDDAS from 2000 and 2004.  Also, here are some slides from a talk she gave recently.  Here's a paper that makes the goals of DDDAS fairly clear.  This  paper provides some insight into a DDDAS application.

 

Week 10 (28 Mar & 30 Mar):  Qualitative Simulation

 

On Tuesday we'll finish discussing DDM in the HLA, and we'll take a quick look at DDDAS and where it's going.  Related papers were assigned for last week's reading.

 

Qualitative simulation, which will be the topic for the rest of the week, falls in-between AI and simulation.  An outgrowth of qualitative reasoning work that was conducted 20 years ago, qualitative simulation attempts to bridge the gap between the exactness that is generally required of quantitative simulation and the imperfect understanding of the modeled system that we typically possess.  I see qualitative simulation applying to the goals of motivated meta-modeling, exploratory analysis and COERCE.  Of course it has merit in its own right, too.  A good starting paper is written by the researcher who started it all, Ben Kuipers.  In more recent times Ingalls has explored methods for applying qualitative simulation to discrete events.  This Ingalls paper appeared first, and more recently this one.  Other variations on qualitative simulation appear in these papers:  Apt, Yan and Erignac.   I have additional papers which I'd gladly make available to you if you're interested.

 

Thursday plan:   We'll finish the small part of DDDAS we didn't complete last time.  Michael Spiegel will present his analysis of the Yucca Mountain modeling challenge.  You'll get to learn a lot about the issues associated with a large, reality based question that can only be answered with modeling and simulation (When you ask questions along the lines of "Will it work for 10,000 years?" lab based experiments reach certain limits.)  Michael hopes to learn about how to present a case study to a class that has learned about multi-resolution modeling, model semantics, and the like, with the goal of helping students relate the key M&S issues to real problems. In the time left I'll start in on qualitative simulation.

 

 

Week 11 (04 Apr & 06 Apr):  Verification, validation and accreditation

 

Tuesday Plan:  Joe Carnahan, one of my senior PhD students will discuss qualitative simulation applied to discrete event simulation (Ingalls papers).

 

Thursday plan:  We cover VV&A.  I've always found the literature in this area a little discouraging, as far as the immensity of the challenge and the capabilities we have to address it.  The literature reflects the gap, so I don't like to spend a lot of time on it.  If you have ideas about how to improve VV&A we'll certainly allocate time to discuss them.

 V,V&A: This is a topic that's been worked over pretty well.  It's about as well mined as the more general fields of formal verification and software testing in the software engineering community.  One of the early researchers in the area is Osman Balci of Virginia Tech.  His 2002 paper is a good place to start.  (I also recommend his website.)   Some good recent references appear in the Foundations V&V conference that took place Fall '02.  Another good starter paper is Girardot and Jacquart that appeared at the foundations conference.   This paper by Pace and Sheehan is focused on desirable attributes in SMEs.  It offers good insight into the human side of V&V.

    Here's a rather lengthy white paper on Verification, Validation and Certification of data.

 

 

 

 

Week 12 (11 Apr & 13 Apr):  Agent based modeling and simulation

 

   Agent based simulation: Where to start?  What's come to be known as agent based simulation has roots in cellular automata, Conway's Game of Life, behavioral modeling, artificial life ("A-life"), a DARPA program called "Ants", and others.

 

Here are some introductions to the topic:

One of the hot approaches to ABMS is SWARM:  http://www.swarm.org/wiki/Main_Page

Some other tools for ABMS:  http://repast.sourceforge.net/ ;   http://www.cougaar.org/

 

A topic of some importance is validation of ABMSs.  Read through these papers.  Ross will lead discussion on the topic.

Other ABMS sites of interest:

 

  You've probably heard about Wolfram's controversial book  (Save $45.00.  Read this review first!) that came out a few years ago. 

The Journal of Artificial Societies and Social Simulation has a good collection of articles. Check out recent issues for some insight into the challenges of social modeling. 

 

 

 

Week 13 (18 Apr & 20Apr):  Human behavioral modeling

 

Project 3 assigned.  See this write-up.

 

 Human Behavioral Modeling/Representation: There are a lot of papers out there.  Here's a small set that provides a good overview of the state of the art and challenges:

Here's a useful recent document that summarizes the state of the art in HBR.  I'll be using it as lecture material Tuesday.

Additional presentations that will be drawn upon in class:

 

 

Week 14 (25 Apr & 27Apr): Genetic algorithms & Neural networks; response surfaces

 

    Next we'll cover approximating simulation behavior (Neural Networks, Genetic Algorithms, Response Surface Networks)

The readings are broken up into groups for RSM and NN. The first reading is a general overview of approximation techniques. The readings are ordered by importance, with the first reading in each section a must-read. (Note: NN readings are after RSM readings, but not less important)

General Reading: The reading is from Burgeon group and is Chapter 2 of their report. Please look over the document, with Section 2.3 being the focus.

    RSM

    NN

 

 

 

Week 15 (02 May): Simulation Based Acquisition

 

Third project, due 3 May.

     Simulation Based Acquisition is primarily a DoD activity, at least by that name.  Good starter papers are: http://www.nap.edu/html/tech_21st/msc.htm and Johnson (especially starting in section 4).  This one offers a good critique of SBA:   http://www.dau.mil/pubs/pm/pmpdf99/brownjf.pdf.   This one has a number of suggested use cases for testing SBA:  http://www.pti-usa.com/white_papers/32-GUMP.doc. Here's a useful presentation by Bob Lutz of HLA OMT fame.  Lutz talk.

More on SBA:

http://www.dtic.mil/ndia/2002sba/dahmann.pdf (Dahmann presentation)
https://www.msiac.dmso.mil/sba/generalinfo.asp ("SBA Roadmap Executive Summary" and "The New SBA - Revisited" was released 24 FEB 04).  It's probably not necessary to read the entire roadmap (it's huge), but the Executive Summary does a pretty good job hitting all of the high points.
http://www.informs-cs.org/wsc00papers/140.PDF (White Paper)
http://www.dau.mil/pubs/pdf/pmpdf99/brownjf.pdf (Brown Paper)

There are many other interesting sites out there that deal with this topic. Among the best are:

http://www.sgi.com/industries/government/modeling.html
http://www.amso.army.mil/smart/
http://www.peostri.army.mil/PRODUCTS/SBA/
http://www.abm.rda.hq.navy.mil/navyaos/content/view/full/171/
http://www.atl.external.lmco.com/overview/papers/980-9946.pdf
http://www.boeing.com/news/releases/2001/q4/nr_011004n.htm
http://www.boeing.com/commercial/777family/pf/pf_computing.html
http://www.informs-cs.org/wsc00papers/140.PDF

http://www.dau.mil/pubs/mfrpts/pdf/sba.pdf
https://www.dmso.mil/public/library/efms/1999/isg/olson.pdf