Information Systems that Really Support Decision-making
Gio WiederholdStanford University Computer Science Department. Gates Computer Science Building 4A Stanford CA 94305-9040 650 725-8363 fax 725-2588
Abstract. A decision maker in an enterprise is expected to make decision which have a positive effect on its future. Information systems should support their activities. Today, databases and web-based resources, accessed through effective communications, make information about the past rapidly available. To project the future the decision maker either has to use intuition or employ other tools, and initialize them with information obtained from an information system to such tools. An effective information system should also support forecasting the future. Since choices are to be made, including the case of not doing anything, such a system must also support the comparative assessment of the effects of alternate decisions. We recommend the use of an SQL-like interface language, to access existing tools to assess the future, as spreadsheets and simulations. Making results of simulations as accessible as other resources of integrated information systems has the potential of greatly augmenting their effectiveness and really support decision-making.
Today rapid progress is being made in information fusion from heterogeneous resources such as databases, text, and semi-structured information bases [WiederholdG:97]. Basic database systems are growing into broader information systems to encompass the communication and analysis capabilities that are now broadly available. In many cases, the objective of the investment in those systems is to support decision-making. However, the decision maker also has to plan and schedule actions beyond the current point-in-time. Databases make essential past and near-current data available, but do not support the essence of decision-making, namely to assess the effect of alternate future courses that can be initiated [Knoblock:96].
To assess the effect in the future of the decisions to be made diverse tools come into play. These tools range from back-of-the envelope estimates, via spreadsheets, to business-specific simulations. The information they provide is complementary to the information about the past provided by databases, and helps in selecting the best course-of-action. The need has been clearly recognized in military planning. Quoting from "New World Vistas, Air and Space Power for the 21st Century"[McCall:96]: The two ‘Capabilities Requiring Military Investment in Information Technology are:
Tools for planning and simulation have benefited from substantial research efforts [DeanW:91].
Today planning research includes distributed planning, focusing on interaction with remote participants [SandewallB:94]. The use of databases and or simulations is not well integrated in this research, the focus is on coordination of plans that are put forward by the participants[LindenG:92]. Once the information on costs, delays, and benefits is known, these systems will analyze the alternatives and present optimal plans. Except in handcrafted situations their use has been limited, since gathering all the required information on past and future situations is tedious. The effect is that participants tend to prune the choices prior to the actual planning phase, limiting greatly the alternatives to be analyzed, and reducing the need for the planing algorithms themselves [TateDK:92].
Simulation is a major area of research as well, and much of this work is used in practical settings, as a tool by staff that supports decision-making. Simulation tools often deal explicitly with risks and provide measures of certainty as part of their outputs. The simulation results are analyzed by support staff, and summaries are forwarded to the actual decision-maker. It is rare that the decision maker has the time to go back and request analyses of variations of the choices that have been presented. The predictive requirements for decision-making have been rarely addressed in terms of integration and fusion [Orsborn:94].
Most simulation tools are too complex for direct use by decision-makers. Specifically, many war-gaming simulations are very costly and most are impossible to reuse, so that they are not available in actual emergencies [Zyda:97]. The primary tool actually used by decision-makers is the spreadsheet. Once the formulas are defined, alternate assumptions or allocations are plugged in to provide estimates of future effects. The results are copied out to paper documents. Some 3-D spreadsheets allow on-line flipping through alternatives that have been computed and retained.
Technology has made great strides in accessing information about past events, stored in databases, object-bases, or the World-Wide Web. Data warehouses that integrate data into historic views are becoming broadly available [Kimball:96]. Access to information about current events is also improving dramatically, with real-time news feeds and on-line cash registers. We must still expand the temporal range to project the effect of candidate events into the future. Decision-making in planning depends on knowing past, present, and likely future situations, as depicted in Figure 1. To assess the future we must access simulations which employ a variety of technologies [Burnstein:96]. Many simulations are available from remote sites [FishwickH:98]. Simulation access should handle both local and remote, and distributed simulation services.
Distributed simulations communicate intensely with each other, using interactive protocols [IEEE:98], but are rarely accessible to general information systems [SinghalC:95]. If the simulation is a federated distributed simulation, as envisaged by the HLA protocol, then one of the federation members may supply results to the decision-making support system. To mediate the difference in granularity, such a server must aggregate data from the detailed events that occur many times per second in distribute simulations to the level of minutes or hours that are typical for initiating planning interactions.
A high-level information system is one that supports decision-making by providing past and future data to a client program. The client program aids the decision maker by incorporating analysis and planning algorithms that assess the value of alternate decisions to be made now or at points in the near future.
During that process effective access is needed to past and current information, and to forecasts about the future, given information about current state and decisions that may be made, typically about resource allocations: money, people, and supplies.
Data covering past information has to be selected, aggregated and transformed to be effective in decision-making. Because of the volume of resources mediating modules are often required. A mediator is a software module that exploits encoded knowledge about certain sets or subsets of data to create information for a higher layer of applications. A mediator module should be small and simple, so that it can be maintained by one expert or, at most, a small and coherent group of experts. The results are most effectively represented by a timeline [DasSTM:94]. The common assumptions is that there is only one version of the past, and that it can be objectively determined.
Recent data may be reported by messages, especially where databases cannot be updated instantaneously. The database paradigm is strong on consistency, but that may mean that recent information, that is not complete or yet verified, may not be retrievable from a formal database. To the decision-maker, however such information is valuable, since any information can lower the uncertainty when one has to project into the future.
Today most simulations are performed in a planner’s mind. That is, the planner sketches reasonable scenarios, mentally developing alternate courses-of-action, focusing on those that had been worked out in earlier situations. Such mental models are the basis for most decisions, and only fail when the factors are complex, or the planning horizon is long. Human short-term memory can only manage about 7 factors at a time [Miller:56]. That matches the number of reasonable choices in a chess game, so that a game as chess can be played as by a trained chess master as well by a dumb computer with a lot a memory. When situations become more complex, tools should be used for pruning the space of alternatives, presentation, and assessment. Today, the pruning is mainly done intuitively, the presentations use whiteboards and the available tools are video-conferences and communicating smartboards, perhaps augmented by results that participants obtain from isolated analysis programs. For instance, a participant may execute a simulation to assess how a proposed decision would impact people and supply resources. Financial planners will use spreadsheets to work out alternate budgets.
Computer-based simulations for planning assess resource consumptions, benefits, risks and the like at points in the future. Risks for alternatives may be obtained by mining past data for similar situations. Simulations that incorporate gaming may automatically investigate allocations that can be made by the enemy, select their best choice and continue the projection for multiple plies. Simulations that deal with natural events will incorporate probabilities of weather, floods, or earthquakes. Derived effects, as flight delays, road closures, and the like might also be computed. There are always multiple futures to be considered in decision-making. The combination is depicted in Figure 1.
Fig. 1. Components of Effective Information Systems
A composed system as we propose has many interfaces. There is the interface to the decision-maker and the tools employed in the process, and there are the interfaces to the databases, their mediators, the message systems, and, finally, to the simulations.
Information system – to - decision maker
To make the results obtained from an information system clear and useful for the decision maker that interface must use a simple model. Computer screens today focus on providing a desktop image, with cut and paste capability, while relational databases use tables to present their contents, and spreadsheets a matrix. Graphic displays using a time-line are effective and allow visualization of many past events in a coherent format [DeZegherGeetsEa:88]. Using modern technology such displays can be augmented with clickable icons for expansion for underlying information [AberleEa:96].
Data resources – to - information system
For database access SQL is the prime candidate. SQL requires that data are well structured, and this requirement simplifies validation and analysis. Modern versions of SQL provide also remote access [DateD:93]. Often SQL language queries are embedded in applications for data validation and analysis. For semi-structured data the dust has not yet settled down, but it appears that XML is a prime candidate [XML], although it will be a long time before thr domain-specific data descriptions (DTD) settle down and HTML documents are replaced. For object-oriented access CORBA and DCOM protocols are now available. These are intended for embedding in application software and not as flexible as SQL and HTML.
Note that access languages as SQL are interface languages only, they are not the language in which to write a database system; those may be written in C, PL/1, or Ada. The databases themselves are owned and maintained by specialists, as domain experts and database administrators.
Simulation – to - information system
To provide the missing link to simulation systems we have developed a prototype simulation access language, SimQL [WiederholdJG:98]. It mirrors that of SQL for databases. Since we wish to integrate past information from databases with simulation results we started with the relational model. However the objects to be described have a time dimension and also an uncertainty associated with them.
The simulations themselves will be written in their own specialized languages or in common programming languages as Fortran or Ada [INEL:93]. Users of spreadsheets are completely unaware of their source languages.
An ability to access simulations as part of an information system adds a significant new capability, by allowing simultaneous and seamless access to factual data and projections (e.g., logistics data with future deployment projections). Interfaces such as SimQL should adhere closely to emerging conventions for information systems. For instance, they might use a CORBA communication framework, and ‘Java’ for client-based services. Such use of COTS technology will facilitate the integration of an SimQL interface into analysis applications that also employ access to diverse non-predictive data resources.
5. SimQL Implementation
SimQL provides an interface for accessing information about future events. There are two aspects of SQL that SimQL mimics:
Using similar interface concepts for data and simulation result access will simplify the understanding of customers and also enable seamless interoperation of SimQL with database tools in supporting advanced information systems. We focus on accessing pre-existing predictive tools.
Components of the system include four types of software
Note that there are significant differences in accessing past data and computing information about the future:
Wrappers are used to provide compatible, robust, and ‘machine-friendly’ access to their model parameters and execution results [HammerEa:97]. Our wrappers also convert the uncertainty associated with simulation results (say, 50% probability of rain) to a standard range ( 1.0 - 0.0 ) or may estimate a value if the simulation does not provide a value for its uncertainty.
Despite the structural similarity, the SimQL language is different from SQL in several ways, among which the following are the most prominent
Our experiments used diverse simulations. They were wrapped to provide information to a SimQL interface.
A customer application can invoke multiple SimQL simulations. Our experiments only combined simulations b) and c), selecting the forecast based on data with minimal uncertainty over a wide range. Still, these experiments with a few real-world simulation convinced us of the applicability of the SimQL concep to a range of settings and provides a foundation for further research in this direction. Details of the prototype are given in [WiederholdJG:98] and on our webpages.
6. Use of Simulation for Current Status
We have focused on using simulation to assess the future. There is however an important task for SimQL in assessing the present state. Databases can never be completely current. Some may be a few minutes behind, others may be several days behind in reporting the state of sales, inventory, and shipments. Information about competitors often lags even further behind, although it is the most crucial element in decision-making.
The traditional, consistency preserving approach in database technology is to present all information at the same point in time, which reduces all information to the worst lag of all sources. It would be better to use the latest data from each source, and then project the information to the current point-in-time. In fact, we are certain that decision maker today will take recent, even if inconsistent data into account when faced with data of varying times of validity. SimQL can support this approach since tot provides an interface that is consistent over databases (assumed to have data with probability 1.0) and simulations, as shown in Figure 2.
Fig. 2. Even the present needs SimQL
Extrapolation of last know database states to the current point-in-time will help in providing a nearly-consistent, somewhat uncertain picture of, say, where the supply trucks are now. This situational information will be more valuable to a decision maker than a consistent picture that is a week out of date, and would not reflect recent sales.
7. Research Opportunities
The importance of rapid, ad hoc, access to data for planning is well conceptually understood, but not served adequately by existing tools. Our work on SimQL provides the interfaces for tools, but did not extend to implementations of the vision for modern information systems that motivated our research. There are many research opportunities to help bring the vision about, from tool development for understood issues to conceptual research to deal with new issues in computation and representation that will arise.
The focus of traditional database technology has been highly reliable and consistent services for operational systems. As decision-making support has become more important, the constraints due to this emphasis have restricted the services needed for effective decision-making support.
Specifically, information systems should not be limited to reporting of historic data. Already, when historic records are a bit out-of-date, planners routinely make undocumented projections to extrapolate to the current situation and obtain an approximate current picture. Extrapolating further into the future increases the uncertainty. Furthermore, alternate decisions, or acts-of-nature, lead to alternate future scenarios. When the simulations incorporate alternate assumptions, they will produce alternate futures, so that an information model that supports planning must not only incorporate uncertainty, but also alternatives.
Interoperation with past information is required. Information systems must integrate past, present, and simulated information, providing a continuous view. The source data will have different temporal granularities, and simulated results must be computed to coincide with expected future events. Furthermore, the representation must indicate what information is valid when.
Temporal issues also arise when dealing with databases that are not fully up-to-date. The time-of-validity capability alone, while modest, can be of great value to decision-makers. It also provides the initial time-point for forecasting from the past, through now, into the future.
Important research into uncertainty processing has not been applicable in the traditional database model [Pearl:88]. There have been multiple definitions of uncertainty and their range of applicability is not clear [BhatnagarK:86]. The information systems that process forecast results will have to take uncertainty explicitly into account, so that the decision-maker can weigh risks versus costs. By supplying data about the future that have intrinsic uncertainties developers will be forced to deal with the issue explicitly, and we expect that new research will evolve to deal with the scalability and robustness of such applications.
The data stored in many databases are also not always certain. Mediators may report such data, using their own knowledge, once analysis tools have reached the maturity to deal with uncertainty over large datasets [GarciaMolinaBP:92].
The information systems must support comparison of the results for multiple courses-of-action (CoAs). These CoAs branch out, although sometimes the results of two distinct sequences of CoAs may merge at a future point. Queries directed to a future point in time will hence provide a list of value-sets. Each set is valid, although the system may impose that the some of the certainties of the sets be equal to 1.0. Labeling of the branches, so that their provenance is clear is open research issue.
As time passes, opportunities for choosing alternatives disappear, so that the future tree is continuously pruned as the now marker marches forward. At the same time, the uncertainties about future events should reduce, so that the tools that provided the information about the future should be re-invoked.
Keeping the results in an information models for planning up-to-date requires continuous re-evaluation. It makes little sense to warehouse all future CoAs and their results. Keeping the system content current is unlikely to happen without tools that automate the integration of information about the future into decision-support systems.
We have described issues that decision makers face when using the current information processing tools that computer scientists, system designers, and implementors provide. We find that integration of databases and forecasting tools is poor. To investigate the feasibility of improving the situation we defined and implemented a new interface language, SimQL. This language provides access to the growing portfolio of simulation technology and predictive services maintained by others. We have some early results, indicating that highly diverse predictive tools may be accessed in an integrated fashion via a language as SimQL.
We expect that interfaces as SimQL will enable information systems to become much more effective and really support realistic decision making processes. Most importantly, having a language interface will break the bottleneck now experienced when predictions are to be integrated with larger planning systems. Because of the importance of forecasting to decision-making, we expect that concepts as demonstrated will in time enter large-scale information systems and become a foundation which will make a crucial difference in the way that spreadsheets, simulations and computational web resources will be accessed and managed.
This research was supported by DARPA DSO, Pradeep Khosla was the Program Manager; and awarded through NIST, Award 60NANB6D0038, managed by Ram Sriram. The original SQL compiler was written by Mark McAuliffe, of the University of Wisconsin - Madison, and modified at Stanford by Dallan Quass and Jan Jannink. James Chiu, a Stanford CSD Master’s student, provided and wrapped the gas station simulation. Julia Loughran of ThoughtLink provided useful comments to an earlier version of this paper [WiederholdJG:98].
[AberleEa:96] Denise Aberle, John Dionisio, Michael McNitt-Gray, Rickt Taira, Alfonso Cardenas, Jonathan Goldin, Kathleen Bown , Robert Figlin, and Wesley Chu: "Integrated Multimedia Timeline of Medical Images and data for Thoracic Oncology Patients"; Imaging and Therapeutic Technology, Vol. 16 No.3; May 1996, pp.669-681.
[BhatnagarK:86] Bhatnagar and L.N. Kanal: "Handling Uncertain Information: A Review of Numeric and Non-numeric Methods"; in Kanal and Lemmer(eds.): Uncertainty in AI, North-Holland publishers, 1986.
[Burnstein:96] M. Burnstein & D. Smith: "ITAS: A Portable Interactive Transportation Scheduling Tool Using a Search Engine Generated from Formal specifications"; in B. Drabble (ed.): Proceedings of the Third International Conference on Artificial Intelligence Planning Systems,: AAAI Press: 1996.
[DasSTM:94] A.K. Das, Y. Shahar, S.W. Tu, and M.A. Musen: "A temporal-abstraction mediator for protocol-based decision support"; Proceedings of the Eighteenth Annual Symposium on Computer Applications in Medicine, Washington DC, 1994, pp.320-324.
[DateD:93] C.J. Date and Hugh Darwen: A Guide to the SQL Standard, 3rd ed; Addison Wesley, June 1993.
[DeanW:91] Thomas Dean and Michael Wellman: Planning and Control; Morgan Kaufmann, 1991.
[DeZegherGeetsEa:88] IsabelleDeZegher-Geets, Andrew G. Freeman, Michael G. Walker, Robert L. Blum, and Gio Wiederhold: "Summarization and Display of On-line Medical Records"; M.D. Computing, Vol.5 No.3, March 1988, pp.38-46.
[FishwickH:98] Paul Fishwick and David Hill, eds: 1998 International Conference on Web-Based Modeling & Simulation; Society for Computer Simulation, Jan 1998,http://www.cis.ufl.edu/~fishwick/webconf.html.
[GarciaMolinaBP:92] Hector GarciaMolina, D. Barbara, and D. Porter: "The Management of Probabilistic Data"; IEEE Transactions on Knowledge and Data Engineering,Vol. 4, No. 5, October 1992, pp. 487-502.
[HammerEa:97] J. Hammer, M. Breunig, H. Garcia-Molina, S. Nestorov, V. Vassalos, R. Yerneni: "Template-Based Wrappers in the TSIMMIS System"; ACM Sigmod 26, May 1997.
[IEEE:98] P1561, Draft IEEE Standard for Modeling and Simulation (M&S) High Level Architecture (HLA); IEEE, 1998.
[INEL:93] Idaho National Engineering Laboratory: "Ada Electronic Combat Modeling"; OOPSLA’93 Proceedings, ACM 1993.
[Knoblock:96] Craig A. Knoblock: "Building a Planner for Information Gathering: A Report from the Trenches"; Artificial Intelligence Planning Systems: Proceedings of the Third International Conference (AIPS96), Edinburgh, Scotland, 1996. .
[LindenG:92] Ted Linden and D. Gaw 1992: "JIGSAW: Preference-directed, Co-operative Scheduling," AAAI Spring Symposium: Practical Approaches to Scheduling and Planning, March 1992
[Kimball:96] Ralph Kimball: The Data Warehouse; Wiley, 1996.
[McCall:96] Gene McCall (editor): New World Vistas, Air and Space Power for the 21st Century; Air Force Scientific Advisory Board, April 1996, Information Technology volume, pp. 9.
[Miller:56] George Miller: "The Magical Number Seven± Two"; Psych.Review, Vol.68, 1956, pp.81-97.
[Orsborn:94] Kjell Orsborn: "Applying Next Generation Object-Oriented DBMS for Finite Element Analysis"; ADB conference, Vadstena, Sweden, in Litwin, Risch Applications of Database’, Lecture Notes In Computer Science, Springer, 1994.
[Pearl:88] Judea Pearl: Probabilistic Reasoning in Intelligent Systems; Morgan-Kaufman, 1988.
[SandewallB:94] E. Sandewall and C. Backstrom (eds.): Current Trends in AI Planning, IOS Press, Amsterdam, 1994.
[SinghalC:95] Sandeep K. Singhal and David R. Cheriton: "Exploiting Position History for Efficient Remote Rendering in Networked Virtual Reality"; Teleoperators and Virtual Environments, Vol.4 No.2, pp.169-193, Spring 1995.
[TateDK:92] Tate,A; Drabble,B; Kirby,R : O-Plan 2: the Open Planning Architecture 1992, AAAI Spring Symposium.
[WiederholdG:97] Gio Wiederhold and Michael Genesereth: "The Conceptual Basis for Mediation Services"; IEEE Expert, Intelligent Systems and their Applications, Vol.12 No.5, Sep-Oct.1997.
[WiederholdJG:98] Gio Wiederhold, Rushan Jiang, and Hector Garcia-Molina: "An Interface for Projecting CoAs in Support of C2; Proc.1998 Command & Control Research & Technology Symposium, Naval Postgraduate School, June 1998, pp.549-558.
[Zyda:97] Michael Zyda, chair: Modeling and Simulation, Linking Entertainment and Defense; Committee on Modeling and Simulation, National Academy Press, 1997.