Darden, Lindley (1998), "Anomaly-Driven Theory Redesign: Computational Philosophy of Science Experiments," in T.W. Bynum and J.H. Moor, The Digital Phoenix: How Computers are Changing Philosophy. New York: Blackwell Publishers, pp. 62-78. 
ANOMALY-DRIVEN THEORY REDESIGN:
COMPUTATIONAL PHILOSOPHY OF SCIENCE EXPERIMENTS

LINDLEY DARDEN

Introduction

I have been asked to discuss how computers have affected my work in philosophy. This paper discusses the use of artificial intelligence (AI) models to investigate both the representation of scientific knowledge and reasoning strategies for scientific change. The focus is on the reasoning strategies used to revise a theory, given an anomaly, which is a failed prediction of the theory.

Discoveries of theories often occur incrementally over a period of time, not in one "aha" moment. Previous work in history and philosophy of science (Darden 1991) traced the modular development of the theory of the gene from 1900 to about 1930. When using historical methods, I feel like an archeologist or a natural historian, unearthing shards from the history. Early in my research career, I naively believed that one could use scientists1 papers and notebooks to determine what reasoning strategies they actually used in their discoveries. However, an adequate description of scientists1 actual reasoning strategies is almost always underdetermined by available historical evidence. T. H. Morgan, one of the principal architects of the theory of the gene, cleaned out his files every five years. Even in cases where abundant unpublished material exists, historians often cannot decide between competing interpretations, such as the debate about Darwin1s use of the analogy to domestic selection in his discovery of natural selection (Ruse 1979).

Consequently, wearing my philosopher's hat, as opposed to the historian's, I have proposed "hypothetical" reasoning strategies that could have accounted for the actual historical changes that did occur in the development of the theory of the gene. (For more discussion of this methodology, see Darden 1991, 15-17, 34-39.) Strategies exemplified in the genetics case may be grouped into three categories: strategies for producing new ideas (e.g., reasoning by analogy), strategies for theory assessment (e.g., prediction testing), and strategies for anomaly resolution and change of scope (Darden 1991). The first two

-62-

categories have been much discussed in philosophy of science under the too-dichotomous headings of the "logic of discovery" and the "logic of justification" (e.g., Nickles 1980a; 1980b; for arguments about their interrelation, see Josephson 1994, 9). Discussion of strategies for learning from mistakes and incrementally improving a theory (or experimental plan) over time have been neglected but are becoming important current topics of research (e.g., Darden and Cook 1994; Darden 1995; Mayo 1996).

As perceptive critics (e.g., Vicedo 1995) have urged, the status of these hypothetical reasoning strategies needs elucidating. If they are not descriptive of reasoning strategies actually used and if they do not have the prescriptive force of, for example, Popper1s (1965) logically grounded view of falsification, what claims can be made for them? I would like them to function in an "advisory" capacity (Nickles, 1987): given this type of problem, this strategy might be fruitful. I would like to find good strategies that scientists would find of use in their work (Darden 1987). For the strategies to be advisory, their efficacy as general problem-solving strategies needs to be tested.

There are at least four ways of testing the adequacy of reasoning strategies. First, cognitive science experiments provide human subjects with problems, record their reasoning protocols, and examine the use of reasoning strategies (e.g., Clement 1988; Klahr et al. 1990). A challenge in such work is to ensure that the reasoning strategies used in lab-amenable exercises are applicable to, and adequate for, actual scientific problems. Secondly, one can teach reasoning strategies to students, put them in problem-solving situations, and see if they are able to profit from the knowledge of the reasoning strategies. It is of course a challenge to design good experiments to demonstrate the efficacy of particular strategies in educational settings (Dunbar 1993). Third, one can play the role of a participant-observer, working with scientists to investigate the use of strategies in as yet unsolved problems (e.g., Darden and Cook 1994). A variant on this third method is that scientists use published strategies without the presence of the philosopher (Raquel Sussman, personal communication).

The first three methods of strategy testing require working with human subjects. It is difficult to control for all the components that the subject may be using in the problem-solving episode. Is a particular strategy used alone to produce the result or are other factors (maybe even unconscious ones) playing a role? The final method for testing strategies aims at avoiding this problem: program the strategies in an AI system and test their adequacy.

AI systems allow philosophers to experiment with knowledge and reasoning. One is no longer a natural historian unearthing artifacts in the historical record nor an armchair philosopher speculating about reasoning strategies. Computational philosophy of science (Thagard 1988) is experimental epistemology. It investigates methods for repre-

-63-

senting knowledge and for modeling reasoning strategies that can manipulate that knowledge. Once one has a working system one can experiment with "what if" scenarios, adding or removing strategies to determine their effects. This method has challenges of its own--finding adequate knowledge representation methods, devising good implementations of strategies, being careful not to tailor the system to produce one desired result. More pragmatic challenges for a philosopher engaging in such research are to learn about AI and programming techniques and to obtain access to the necessary computing facilities. One can either become a programmer oneself, as, for example, Paul Thagard and John Josephson have done, or one can collaborate with others (as I have done).

My interest in using AI to investigate scientific discovery was sparked by an exciting paper by Bruce Buchanan entitled "Steps Toward Mechanizing Discovery" (presented in 1978; published in 1985). Buchanan discussed the first expert systems, which he helped to develop at Stanford. DENDRAL formed hypotheses about the structures of compounds, given data from a mass spectrograph (Lindsay et al. 1980; 1993).

Buchanan articulated a new research program:
 

In 1980, I spent my sabbatical with Buchanan1s group at Stanford, then called the Heuristic Programming Project. In subsequent years I learned more about the work in AI on scientific discovery (e.g., Langley et al. 1987; Kulkarni and Simon 1990; Shrager and Langley 1990). Simultaneously, I continued my historical and philosophical work on the genetics case and began exploring methods for implementing parts of that case. Collaborators and I explored rule-based systems (in which knowledge is represented in if-then rules) and frame based systems (in which knowledge is represented by entities with properties grouped hierarchically) (e.g., Darden and Rada 1988a; 1988b). However, these systems required tremendous time and effort for small results in modeling general reasoning strategies for scientific discovery. The problem of finding an adequate representation of the components of the theory of the gene loomed large. Finding an ade-

-64-

quate knowledge representation system is itself a subject of experimentation.

Anomaly-driven Theory Redesign in TRANSGENE

An entirely new approach proved much more fruitful. B. Chandrasekaran, John Josephson, and their colleagues at the Laboratory for Artificial Intelligence Research (LAIR) at Ohio State University developed what they call a "functional representation" (FR) language (Sembugamoorthy and Chandrasekaran 1986). It is useful for representing the functional components of a device, e.g., a chemical manufacturing plant (Keuneke 1989), or a biological system, e.g., the immune system (Sticklen and Chandrasekaran 1989). FRs support diagnostic reasoning by allowing one to find a faulty functional component and they support reasoning in redesigning a faulty component of the system to improve its functioning. Diagnosis and redesign are examples of "generic tasks" for which the LAIR group has developed specific information processing strategies (Josephson 1994, 50-62). If one can identify a problem, e.g., anomaly resolution, as an example of a generic task, e.g., diagnosis, then one can use the knowledge representation and inference procedures developed for that task.

Collaborators and I at LAIR built the TRANSGENE system (Darden et al. 1991; Darden et al. 1992). The TRANSGENE project has been guided by two analogies. First, a scientific theory can be viewed as analogous to a device, a device with the functions of prediction and explanation. Such an analogy allows the FR language, initially developed for computationally representing devices, to be used to represent the transmission (Mendelian) genetic mechanism posited by the theory of the gene (hence the name TRANSGENE, which has nothing to do with more modern transgenic manipulations). The second guiding analogy is that between anomaly resolution and diagnosis/redesign. The reasoning in localizing a failure of a theory is like the reasoning in diagnosing a failure of a device. Reasoning in fixing a theory to remove an anomaly is like reasoning in redesigning the device to remove the failure. Such an analogy allows methods that have been developed in AI for diagnosis and redesign to be applied to the task of scientific theory revision.

An anomaly is generated when an observation or experimental result does not agree (to some specified degree of accuracy) with a prediction of a theory. One might be able to resolve the anomaly by showing that there is a problem with the data. At the other extreme, one might decide that the anomaly is sufficiently serious to require completely abandoning the theory (e.g., Kuhn (1970) discusses crisis anomalies that lead to a paradigm change). These responses are not modeled in TRANSGENE. The TRANSGENE system requires the assumption that the data supplied is accurate; furthermore, it proceeds

-65-

on the assumption that the theory can be modified to resolve the anomaly. (For stages in anomaly resolution, see Darden 1991, 269; Darden 1992.)

The reasoning in making an incremental change to remove an anomaly marks a path through a space of possible changes. This reasoning constitutes the cognitive process of interest in the TRANSGENE project. The cognitive "agent," navigating the theory change problem space, is here thought of as facing, first, a diagnosis problem to localize the fault, and, then, an incremental redesign problem to change the theory. These reasoning tasks are called "anomaly driven redesign tasks." A more precise, information-processing statement of the anomaly driven redesign task is as follows:
 

In order to carry out this anomaly-driven redesign task, a computational framework is needed to do the following: (1) Represent the theory. (2) Detect an anomaly. (3) Trace the fault to a specific component of the theory, that is, tentatively localize the fault. (4) Use redesign strategies to propose a fix at the fault site. (5) Test the proposed redesign. (6) If the test fails, return to a previous step to select another localization or propose an alternative way to fix the theory, then retest.

TRANSGENE.1, 2, and 3 were three versions of the TRANSGENE system (each an improvement over the previous one) developed to carry out the above tasks. They show improvements in representing components of the theory of the gene in the FR language. Furthermore, they explore increasingly powerful methods for carrying out the tasks of diagnosis and redesign.

TRANSGENE.1 focused on the issues of theory representation and support of generation of fault site hypotheses. In transmission genetic theory, the processes of interest are inheritance mechanisms and the observations to be predicted are patterns in the distribution of inherited characteristics in offspring. An abstract representation of the theory was constructed, consisting of a set of causal steps with variables. When supplied with an anomaly, the system was able to generate all the possible fault sites and provide queries to a user to aid in the localization of failing steps in genetic processes (Darden 1990; 1992; Moberg and Josephson 1990).

Added to TRANSGENE.2 and TRANSGENE.3 were the abilities to simulate genetic crosses and to carry out theory revision automatically. The systems predict the result of a breeding experiment

-66-

by simulation, given the initial conditions of the experiment. The predicted result is compared to an observed result, which is provided by the user. If an anomaly is detected, the systems try to localize the fault to one or more specific components of the theory. Having localized the fault, the systems attempt to modify the theory. Resimulation is done using the modified theory and predicted results are once again compared with the experimental results. This loop is repeated until there are no more anomalies.

Figure 1.

The redesign methods used in TRANSGENE.2 were weak; little knowledge was used in narrowing the search for changes. Changes were made using a random generate and test method (Darden et al. 1991). TRANSGENE.3 incorporated changes in the representation of genetic theory that enabled the difference between a predicted and observed result to be used to guide localization and redesign. (For a similar implementation of anomaly characterization, see Karp 1990.) An important strategy in anomaly resolution is to extract as much information as possible from the nature of the anomaly for use in making the changes. For example, if the theory predicts a 1AA:2Aa:1aa ratio but the data show a 2Aa:1aa ratio, then the missing set of AAs must somehow be accounted for in the redesign process. Different modifications would be made if the anomaly had been 1:0:1. The characterization of the difference between the prediction and the observed data provides guidance for localization and redesign.

-67-

Figure 2.

TRANSGENE.3 has two parts, as shown in Figure 1: the transmission genetics simulation model and the hypothesis forming, testing and refining model. Figure 2 shows the simulator represented in the FR language, with sequential steps in a genetic breeding experiment. The parents1 genotypes (their genes) and phenotypes (their visible characteristics caused by the genes) are inputs to the system. The simulator generates the types of gametes (germ cells: sperm and eggs), unites them in the step of fertilization, and calculates the expected ratios of genotypes and phenotypes of the offspring. That output constitutes the theory1s prediction of the results of the breeding experiment. The system is then supplied data from an actual historical breeding experiment. It determines whether the prediction matches the data to a specified degree of statistical accuracy. When it detects an anomaly, the anomaly resolution part of the system is called, which is

-68-

the top box depicted in Figure 1. The information-processing tasks that TRANSGENE.3 carries out are shown in Figure 3.

Figure 3. Information Processing Tasks in Anomaly Resolution

The difference between the prediction and observation is calculated. This characterization of the anomaly is used to guide the step of finding possible malfunction sites in the simulator. Once an anomaly is characterized, the next subtask is to localize a malfunction site in the theory. The FR model of the theory enables one to generate all the possible fault sites by explicitly representing the modular steps involved in the process. A step in a process, represented by a state-transition in the FR formalism, can go wrong in any one of these three ways: (1) because something occurs before a given state and alters its inputs, and thus a new prior state needs to be added; (2) something about the description of the state transition itself is wrong; (3) the outputs of the step are correct, but something occurs afterward to alter the process's output, and thus a new post-state needs to be added. Consequently, for each state in the FR, three malfunction site hypotheses are gener-

-69-

ated: before the state, in the state, or after the state. TRANSGENE.2 generated all such possible fault sites. For large FRs, even the number of single-fault hypotheses becomes large rapidly. TRANSGENE.3 used the nature of the anomaly to limit this search.

After one or more sites are chosen as the location of a malfunction, then those components must be changed to remove the malfunction. The weak, random generate-and-test method used in TRANSGENE.2 for the theory redesign subtask was replaced in TRANSGENE.3 with dependency directed backtracking. After characterizing the nature of the anomaly, TRANSGENE.3 backtracked through the steps in the FR representation from output toward the input to localize and fix the faulty module. This provided a much more guided redesign process, based on the nature of the anomaly. For example, if TRANSGENE.3 predicted 1AA:2Aa:1aa but the data showed 2Aa:1aa, then the system backtracked through its simulation to determine where the AA gametes were generated and suggested ways to eliminate them. After a fix hypothesis is generated and added to the system, the simulator is run again with the redesigned component to determine if it now makes the correct prediction. (For more details of the TRANSGENE.3 system, see Darden et al. 1992.)

In TRANSGENE.2 and .3 the redesign consisted in changing parameters that represented proportions of types of gametes and zygotes. Additional knowledge, outside the original representation of the theory, would be needed in a significant redesign process for the system to be able to generate hypotheses as to the causal mechanisms producing such changed proportions. Two ways of extending the implementation to make use of additional knowledge have occurred to us: use knowledge of generic biological processes (compare Goel 1989) and use knowledge from closely related scientific fields (Darden and Maull 1977). For example, when one finds none of an expected type of gametes, one might invoke generic processes for how biological things disappear, e.g., they migrate, they are digested by something else, they die. The missing AA gametes are a lethal gene combination that causes the death of offspring receiving AA; hence, a system that could invoke dying to explain missing biological items could conjecture the hypothesis of lethal genes. Goel (1989) represented a library of generic processes, characterized by the functions they achieve in an engineering domain. He used those processes in redesign tasks to accomplish desired functions. A subsequent version of TRANSGENE could implement such a library of biological processes for use in biological theory redesign to produce more creative fixes to the theory.

The system should be able to learn from its anomaly resolution episodes. After an anomaly has been resolved, the newly constructed theory should be saved for use in similar cases in the future. A proliferation of alternative versions of the theory result. They constitute

-70-

the explanatory repertoire of Mendelian genetics (Darden 1991,195-199). Methods for storing the newly constructed versions and choosing the appropriate one in a future problem-solving episode are needed. A typical method for choosing among a set of problem-solving patterns is to classify the patterns according to the type of anomaly they can solve. Then a classification procedure can be used to test to see whether a pattern exists for a given type of anomaly; if it does, then that version of the theory can be used to explain the anomaly. For example, if additional genetic results produced 2:1 anomalies, then a lethal gene pattern could be chosen from the explanatory repertoire and instantiated for the case.

Storing of patterns for reuse and categorizing them as to the kinds of anomalies they solve is analogous to "classification-based diagnosis." For example, if the patient has symptoms 1,2,3, conclude disease X. If no appropriate pattern is already in the look-up table, then the usual anomaly resolution methods of localization and redesign can be used. These anomaly resolution methods are analogous to "model-based diagnosis" (Chandrasekaran and Mittal 1983; Chandrasekaran et al. 1989; Davis and Hamscher 1988).

Implications of the AI Research

This AI work provides alternative views to traditional analyses of the nature of scientific theories and of reasoning in theory change. The logical empiricists analyzed theories as formal axiomatic systems (Suppe 1977 criticizes this). A more recent descendant of this method is Kitcher's (1993) view of theories as abstract argument patterns. Explanation and prediction are analyzed as necessitating deductive arguments. Philosophers, enamored of the Duhem-Quine thesis (for a clear statement of this thesis, see Quinn 1974), have often despaired of localizing and fixing faults in theories. Philosophers have also been pessimistic about analyzing reasoning in the formation of new hypotheses (e.g., Popper 1965).

This AI work suggests alternative analyses. Some scientific theories can be represented via an abstraction of core causal processes (compare Darden and Cain 1989). A representation may be at multiple levels of abstraction and specified at one or several hierarchical levels of detail. (Philosophers trained in logic typically discuss only two levels: a variable and its value. They miss the lesson from AI about multiple hierarchical levels of abstraction.) The abstraction consists of modular steps in the causal process. The modularity is crucial in anomaly resolution; the amount of modularity versus interconnectedness of components determines the ease of localization of problems in single components. The more separable and independently accessible the modules, the easier it will be to avoid a Duhem-Quine holism.

-71-

Such an abstract representation allows a theory to be implemented as a simulator. Given the initial conditions of an experiment, its output can be viewed as a prediction of the outcome of the experiment. The process of making a prediction involves "running" the simulator "forward." Alternatively, the formalism can support explanation, by allowing one to "traverse" the formalism backward to see how a particular state is reached. (Although TRANSGENE was not explicitly used for explanation, humans could use it that way.)

Although perhaps, in principle, any component of one1s theoretical system can be changed (a concession to Duhem-Quine), in practice, scientists do localize anomalies and successfully fix theoretical components. This analogy to diagnosis/redesign shows feasible reasoning strategies for this process. Characterization of the nature of the anomaly and dependency-directed backtracking can provide guidance for localization. The modularity of the steps proved essential in anomaly resolution, allowing localization of faults and a focus for redesign efforts. Historical work had shown that anomalies often cause previously implicit assumptions to be made explicit in order to localize an anomaly (Darden 1991). This implementation allowed steps before or after the explicitly represented states to be conjectured to exist.

Hypotheses about the localization of an anomaly could be tested by experiment, by, for example, doing experiments to determine if a given causal step (e.g., formation of gametes) has occurred. Thus, a system like TRANSGENE could be coupled to a laboratory to help in focusing on the most informative scientific experiments for localizing a problem.

Redesign hypotheses can be either simple tweaking of numbers or they may require the generation of more elaborate hypotheses. TRANSGENE has shown that an AI system, guided by a numerically characterized anomaly (or the qualitative information that a set is missing), can propose quantitative fixes, the tweaking of parameters. However, when more creative hypotheses are needed, requiring knowledge outside the system itself (e.g., via interfield connections), then the task is more difficult. The TRANSGENE work has suggested ways of investigating creative redesign strategies, such as the use of generic libraries of abstract, functionally characterized processes (Goel 1989), and the use of interfield relations (Darden and Maull 1977; Darden 1991).

Doing computational philosophy of science requires precision and completeness not required in non-implemented studies of history and philosophy of science. The nature of knowledge representation methods from AI force a reconsideration of the scientific case to find appropriate details. When developing an AI implementation, items that were previously glossed over must be specified in detail. If key steps are omitted, the program will not run. Thus, one result of this work is an increase in precision and completeness in analysis of a scientific

-72-

case study. (Of course, doing an implementation can also introduce artifacts, items necessary to make the system run but irrelevant to the actual scientific case, so care must be taken.)

Once a system is working, experimentation with representation and reasoning is possible. For example, the changes in TRANSGENE demonstrated the efficacy of dependency directed backtracking versus random trial and error. Furthermore, computational philosophy of science work provides the potential for philosophers to contribute to actual scientific discovery, either by informing scientists of strategies whose efficacy has been demonstrated or by assisting in building computational systems to make discoveries and resolve anomalies.

Conclusion

The TRANSGENE project has investigated reasoning in the redesign of a scientific theory to improve it to remove an anomaly. The work draws on a detailed historical case study of the development of Mendelian (transmission) genetics. The theory of the gene is represented in a computational form that supports simulation, prediction, and explanation. Anomaly resolution is viewed as a diagnosis/redesign task. When presented with an anomalous observation, the TRANSGENE system generates fault site hypotheses, chooses among them, proposes fix hypotheses at the fault site, and tests them by rerunning the simulation. These methods are general ones for representing and improving mechanistic, causal scientific theories.

ACKNOWLEDGMENTS

This material is based on work supported by the National Science Foundation under Grant No. RII-9003142. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect those of the National Science Foundation. Lindley Darden1s work was also supported by a General Research Board Award from the Graduate School of the University of Maryland, College Park. The TRANSGENE system was designed in collaboration with John Josephson and Dale Moberg, with helpful comments by Susan Josephson. The TRANSGENE.2 system was implemented by Satish Nagarajan. The TRANSGENE.3 system was implemented by Sunil Thadani. Thanks to Nancy Hall, Susan and John Josephson, Robert Skipper, Frederick Suppe, and Alana Suskin for comments on an earlier draft of this paper.

-73-

REFERENCES

Buchanan, Bruce (1985), “Steps Toward Mechanizing Discovery,” in K. Schaffner (ed.), Logic of Discovery and Diagnosis in Medicine. Berkeley: University of California Press, pp. 94-114.

Chandrasekaran, B. and Sanjay Mittal (1983), “Deep versus Compiled Knowledge Approaches to Diagnostic Problem-solving,” International Journal of Man-Machine Studies 19:425-436.

Chandrasekaran, B., J. Smith, and J. Sticklen (1989), “'Deep' Models and Their Relation to Diagnosis,” in K. S. Zadeh (ed.), Artificial Intelligence in Medicine, Vol. 1. Tecklenberg, Germany: Burgverlag, pp. 29-40.

Clement, John (1988), “Observed Methods for Generating Analogies in Scientific Problem Solving,” Cognitive Science 12:563-586.

Darden, Lindley (1987), “Viewing the History of Science as Compiled Hindsight,” AI Magazine 8(2):33-41.

Darden, Lindley (1990), “Diagnosing and Fixing Faults in Theories,” in J. Shrager and P. Langley (eds.), Computational Models of Scientific Discovery and Theory Formation. San Mateo, California: Morgan Kaufmann, pp. 319-346.

Darden, Lindley (1991), Theory Change in Science: Strategies from Mendelian Genetics. New York: Oxford University Press.

Darden, Lindley (1992), “Strategies for Anomaly Resolution,” in R. Giere (ed.), Cognitive Models of Science, Minnesota Studies in the Philosophy of Science, Vol. 15. Minneapolis: University of Minnesota Press, pp. 251-273.

Darden, Lindley (1995), “Exemplars, Abstractions, and Anomalies: Representations and Theory Change in Mendelian and Molecular Genetics,” in James G. Lennox and Gereon Wolters (eds.), Concepts, Theories, and Rationality in the Biological Sciences. Pittsburgh, PA: University of Pittsburgh Press, pp. 137-158.

Darden, Lindley and Nancy Maull (1977), “Interfield Theories,” Philosophy of Science 44:43-64.

-74-

Darden, Lindley and Roy Rada (1988a), “Hypothesis Formation Using Part-Whole Interrelations,” in David Helman (ed.), Analogical Reasoning. Dordrecht: Reidel, pp. 341-375.

Darden, Lindley and Roy Rada (1988b), “Hypothesis Formation Via Interrelations,” in Armand Prieditis (ed.), Analogica. Los Altos, California: Morgan Kaufmann, pp. 109-127.

Darden, Lindley and Joseph A. Cain (1989), “Selection Type Theories," Philosophy of Science 56:106-129.

Darden, Lindley, Dale Moberg, Satish Nagarajan, and John Josephson (1991), “Anomaly Driven Redesign of a Scientific Theory: The TRANSGENE.2 Experiments,” Technical Report 91-LD-TRANSGENE. Laboratory for Artificial Intelligence Research, The Ohio State University, Columbus, Ohio.

Darden, Lindley, Dale Moberg, Sunil Thadani, and John Josephson (1992), “A Computational Approach to Scientific Theory Revision: The TRANSGENE Experiments,” Technical Report 92-LD-TRANSGENE, Laboratory for Artificial Intelligence Research, The Ohio State University, Columbus, Ohio.

Darden, Lindley and Michael Cook (1994), “Reasoning Strategies in Molecular Biology: Abstractions, Scans and Anomalies,” in D. Hull, M. Forbes, and R. M. Burian (eds.), PSA 1994, v. 2. East Lansing, Michigan: Philosophy of Science Association, pp. 179-191.

Davis, Randall and Walter C. Hamscher (1988), “Model-based Reasoning: Troubleshooting,” in H. E. Shrobe (ed.), Exploring Artificial Intelligence. Los Altos, California: Morgan Kaufmann, pp. 297-346.

Dunbar, Kevin (1993), “Concept Discovery in a Scientific Domain,” Cognitive Science 17:397-434.

Goel, Ashok (1989), Integration of Case-Based Reasoning and Model-Based Reasoning for Adaptive Design Problem Solving. Ph.D. Dissertation. The Ohio State University, Columbus, Ohio.

Josephson, John R. and Susan G. Josephson (eds.) (1994), Abductive Inference: Computation, Philosophy, Technology. New York: Cambridge University Press.

-75-

Karp, Peter (1990), “Hypothesis Formation as Design,” in J. Shrager and P. Langley (eds.), Computational Models of Scientific Discovery and Theory Formation. San Mateo, California: Morgan Kaufmann, pp. 275-317.

Keuneke, Anne (1989), Machine Understanding of Devices: Causal Explanation of Diagnostic Conclusions. Ph.D. Dissertation. Department of Computer and Information Science, The Ohio State University, Columbus, Ohio.

Kitcher, Philip (1993), The Advancement of Science: Science without Legend, Objectivity without Illusions. New York: Oxford University Press.

Klahr, David, Kevin Dunbar, and Anne L. Fay (1990), “Designing Good Experiments to Test Bad Hypotheses,” in J. Shrager and P. Langley (eds.), Computational Models of Scientific Discovery and Theory Formation. San Mateo, California: Morgan Kaufmann, pp. 356-402.

Kuhn, Thomas (1970), The Structure of Scientific Revolutions. 2nd Edition. Chicago: The University of Chicago Press.

Kulkarni, Deepak and Herbert Simon (1990), “Experimentation in Machine Discovery,” in J. Shrager and P. Langley (eds.), Computational Models of Scientific Discovery and Theory Formation. San Mateo, California: Morgan Kaufmann, pp. 255-273.

Langley, Pat, Herbert Simon, Gary L. Bradshaw, and Jan M. Zytkow (1987), Scientific Discovery: Computational Explorations of the Creative Process. Cambridge, Massachusetts: MIT Press.

Lindsay, Robert K., B. G. Buchanan, E. A. Feigenbaum, and J. Lederberg (1980), Applications of Artificial Intelligence for Organic Chemistry: The DENDRAL Project. New York: McGraw Hill.

Lindsay, Robert K., B. G. Buchanan, E. A. Feigenbaum, and J. Lederberg (1993), “DENDRAL: A Case Study of the First Expert System for Scientific Hypothesis Formation.” Artificial Intelligence 61:209-261.

Mayo, Deborah G. (1996), Error and the Growth of Experimental Knowledge. Chicago: University of Chicago Press.

-76-

Moberg, Dale and John Josephson (1990), “Diagnosing and Fixing Faults in Theories, Appendix A: An Implementation Note,” in J. Shrager and P. Langley (eds.), Computational Models of Scientific Discovery and Theory Formation. San Mateo, California: Morgan Kaufmann, pp. 347-353.

Nickles, Thomas (ed.) (1980a), Scientific Discovery: Case Studies. Dordrecht: Reidel.

Nickles, Thomas (ed.) (1980b), Scientific Discovery, Logic and Rationality. Dordrecht: Reidel.

Nickles, Thomas (1987), “Methodology, Heuristics, and Rationality,” in J. C. Pitt and M. Pera (eds.), Rational Changes in Science. Dordrecht: Reidel, pp. 103-132.

Popper, Karl (1965), The Logic of Scientific Discovery. New York: Harper Torchbooks.

Quinn, Philip (1974), “What Duhem Really Meant,” in R. S. Cohen and M. Wartofsky (eds.), Methodological and Historical Essays in the Natural and Social Sciences, Proceedings of the Boston Colloquium for the Philosophy of Science 1969-1972, Vol. 14, Dordrecht: Reidel, pp. 33-56.

Ruse, Michael (1979), The Darwinian Revolution: Science Red in Tooth and Claw. Chicago: University of Chicago Press.

Sembugamoorthy, V. and B. Chandrasekaran (1986), “Functional Representation of Devices and Compilation of Diagnostic Problem-solving Systems,” in J. Kolodner and C. Reisbeck (eds.), Experience, Memory, and Reasoning. Hillsdale, New Jersey: Lawrence Erlbaum Associates, pp. 47-73.

Shrager, J.  and P. Langley (eds.) (1990), Computational Models of Scientific Discovery and Theory Formation.  San Mateo, California: Morgan Kaufmann.

Sticklen, J. and B. Chandrasekaran (1989), “Integrating Classification-Based Compiled Level Reasoning with Functional-Based Deep Level Reasoning,” Applied Artificial Intelligence 3:275-304.

Suppe, Frederick (ed.) (1977), The Structure of Scientific Theories. 2nd Edition. Urbana: University of Illinois Press.

-77-

Thagard, Paul (1988), Computational Philosophy of Science. Cambridge, MA: MIT Press.

Vicedo, Marga (1995), “How Scientific Ideas Develop and How to Develop Scientific Ideas,” Biology & Philosophy 10:489-499.

-78-