Source: http://www.isis.vt.edu/~dlegg/lectopcs.html Lesson: Case-Based Reasoning Background Information: The Russian wheat aphid has been a chronic insect pest of winter wheat in some areas of southeast Wyoming. Accidentally introduced into North America in the late 1970s or early 1980s, this insect has had a major economic impact on winter wheat production, beginning with the 1987 crop. Winter wheat is a crop that usually provides small economic profits to Wyoming farmers. Faced with raising and selling a relatively low-valued cash crop, Wyoming winter wheat producers try to increase their margin of profit by keeping production costs as low as possible. This translates into reducing or even eliminating several production practices. One such practice they try to do without is an insecticide application for controlling the Russian wheat aphid. The population dynamics of the Russian wheat aphid are such that, if the environmental conditions are just right, the level of infestation can rapidly increase from next to nothing to well over 40% infested tillers. Worse, this increase can occur within two or three weeks, leaving producers little time to conduct an emergency Russian wheat aphid management effort. Situation: Al had been watching his winter wheat very closely since the first of April. It had been a warm, dry winter, and he knows that, the Russian wheat aphid has only been a problem after warm, dry winters. Al was remembering the previous years that his wheat has suffered economic levels of Russian wheat aphid infestations. "Let's see", Al reflected, "in 1987, we had a warm, dry winter and the winter wheat was droughtstressed. That was the first year we had the aphid, and we didn't know what happened to the wheat until it was too late. That crop was a bad one, yielding only 17 bushels per acre." "In 1988, we were in another drought year, with another warm, dry winter. That was a bad year for the aphid, too", Al said. "This year, we were ready for the aphid and applied 16 ounces of Lorsban per acre. That took care of the aphid but it sure was an expensive application, costing almost $7.00 per acre to make." "In 1989", Al continued", "we had a continuation of the drought, with another warm, dry winter. Some of my neighbors had pretty high infestations last fall. Those neighbors always plant real early, though, and last year they ended up spraying both in the fall and in the spring" Al thought. "That cost them about $14.00 per acre. "I had to spray again this year for the aphid, and it cost me $7.00 per acre. The University of Wyoming has conducted some studies on other ways to manage the aphid, and one of them is not to plant real early in the fall. Farmers in Laramie County, near Salem Cemetery, always start planting real late and end their planting late, and they have never had to spray for the aphid", Al mused. "They have real heavy soil, though, and they don't have to worry about the wind erosion as much as we do here in the Goshen Hole, where the soil is real light." "In 1990, I again had to spray for the Russian wheat aphid, but the infestations weren't as bad as in previous years. That crop was planted three weeks late because I couldn't get a part to fix my tractor in time to plant when I normally plant. I was lucky that we didn't have big winds last winter", Al muttered. "Hm, maybe there's something to this planting later than normal for reducing aphid infestations" Al said. "In fact, the wheat looked so good at the first joint growth stage that I was considering not spraying at all until this guy from the University of Wyoming did a sampling effort on my farm and found out I had about 15% infested tillers. He whipped out a new laptop computer which had on it this new computer program that calculated the level of infestation I would need at full head emergence to economically justify an insecticide application now. We ran the program and that level of infestation was 15%. Shoot, since I was already at the 15% level, I decided to go ahead and make an application and boy, am I glad I did" Al proclaimed out loud, not noticing he was talking to himself. "This same guy from the University talked me into putting out a little experiment at the end of one of my wheat strips. There, I put out four big plots; 1) Lorsban at 16 ounces per acre, 2) Lorsban at 12 ounces per acre, 3) Cygon at the recommended rate, and 4) an untreated control. Now, just four weeks later, the wheat in the untreated plot looks really bad, the wheat in the Cygon-treated plot looks better, and the wheat in the two Lorsban-treated plots looks the best. What's better is that the wheat in the 12 ounce plot looks as good as the wheat in the 16 ounce plot" Al thought. "I'm glad that I took a gamble and sprayed all of my wheat this year at the 12 ounce rate. That cut my insecticide cost by 25%. I guess those guys from the University aren't so bad after all!" "In 1991, we had this monster snowstorm in late November. That storm left about three feet of snow on the ground, and it stayed there all winter", Al remembered. "And when it melted the next spring, we had puddles of water everywhere. Luckily for the wheat", Al said, not noticing that he was talking out loud to himself again, "it all soaked in after a day. That was the first year I didn't have to spray for the Russian wheat aphid. That guy from the University showed up again early that spring and told me they had found out that the aphid doesn't like lots of snow on the ground for weeks at a time and they don't like lots of moisture either" Al remembered. "That guy must have been right because I didn't have hardly any aphids on my wheat that year." "Nineteen ninety-two was a different story, though" groused Al. We had another warm, dry winter and lots of aphids that next spring. Shoot, by the end of the first week in April, the wheat looked so bad from aphid infestation that I thought It may be a complete disaster. I considered not wasting my money on an insecticide application. Well, I thought better of that solution and went ahead and sprayed it anyway, using the Lorsban 12 ounce per acre treatment. After a few days, the wheat began to look better, and after about a week and a half, the wheat looked pretty darned good" Al reflected. "I'm glad that I didn't give up on my wheat that year." "In 1993, we had a terribly dry winter and early spring. What was worse", Al muttered to himself, "was the terrible winds we had in late winter and early spring. Those winds just roasted the wheat on the hilltops and on the west sides of the hills", Al remembered. "And what was odd about that year was that we didn't have an economic aphid infestation! That was the second time I didn't have to spray for the aphid. That guy from the University came around again that year", Al remembered, "and I asked him why we didn't have a problem with the aphid. He reminded me that the soil was so dry the previous fall that early- and average-planted wheat didn't emerge until very late in the fall. Lateplanted wheat emerged about the same time. Then winter hit. He said that the aphid didn't have enough time to colonize those late-emerging fields and get their populations built up to good levels to make it through the winter", al said to himself, not noticing that he was beginning to talk to himself more and more. "Hm", al said to himself, "maybe there's something to this planting later than normal after all. Still, I can't risk losing the soil on this farm by planting late all the time", Al concluded. "In 1994, we had plenty of snow over the winter, and had lots of spring rains. We didn't see hardly an aphid all summer" Al said. "That was the year that we suffered that tremendous hail storm, which wiped out our crop", Al groused. "Good thing I had insurance! That was the year I bought another farm. I also got to thinking about my weed and aphid management programs, looked at my old spray rig, and decided I couldn't get all the spraying I might have to do done if I was rushed by the Russian wheat aphid", Al said to himself. "That was when I bought a huge sprayer so I can finish my spraying in a little less than a week." "In 1995, we again had a warm, dry winter. Worse, the wheat came up when it normally did, and I didn't try to plant later than normal. That spring, the guy from the University was out in early March and didn't find many aphids. He came back in late March and said that he found about a 4% level of infestation. I was watching my wheat too and, in the first week in april, began to see spots of severe Russian wheat aphid-infested wheat. Well, when I saw those, I began spraying for weeds and Russian wheat aphids. About that same time, the fellow from the University showed up and said that I had about a 40% level of infestation. This year, I was farming three and a half sections and got all of them sprayed just before it started raining. Then it rained for the next two months. Then it quit and turned hot and dry. That was the best wheat crop I've had. I sold most of it at $5.00 per bushel, too! My neighbors waited on the spraying until it was too late, though", Al remembered out loud, "and they either didn't get the aphid sprayed because it was too wet or had to have the custom applicator fly it on for them at triple the cost", Al said. "They later said that they could really see where the aphid had been. I guess that University guy was right again, when he said that rain will only stop the growth of the infestation, not reduce it." "In 1996, we had another warm dry winter. The wheat, however, was planted into dry soil and had a delayed emergence. We also had bad winds late that winter and early that spring. The winds killed about 1/3 of the wheat. I was watching my wheat all spring for signs of aphids, and hadn't seen much. But I remembered the 1993 crop, when we had a delayed emergence and a warm, dry winter. The aphid wasn't bad that year. Luckily, that guy from the University showed up again in the middle of April, and he said that the infestation in my wheat at that time was about 1%. The wheat was between tillering and first joint stage, and he had a new version of that computer program with him. The program forecasted aphid infestations given different amounts of moisture and temperature. I asked him to make some runs at normal moisture and temperatures. When he did that, he came up with potential levels of infestation that ranged from 30 to 50% infestation at head emergence. OW! I asked him to make some runs with some extra rain (about 4 inches extra) and he came up with levels of infestation that ranged from 4 to 11%", Al remembered. It is now time to decide on whether or not to spray for the Russian wheat aphid. Questions for the decision-maker: 1) How likely is the current level of Russian wheat aphid infestation to increase to or past the economic injury level?, 2) What is the economic injury level?, 3) What is the forecasted late spring and early summer weather: a) warmer temperatures and less moisture than normal, b) normal temperatures and moisture, or c) cooler temperatures and greater moisture than normal? Information needed to answer the questions: 1) Russian wheat aphid cases from past years 2) Al's projected wheat yield, value of wheat, cost of Russian wheat aphid control 3) Forecasted late spring and early summer weather Lesson: Case-Based Reasoning Reference: Kolodner, J. 1993. Case-based reasoning. Morgan Kaufmann Publishers, San Mateo California. 668 pp. Case-Based Reasoning - an Overview What's so Special About Case-Based Reasoning? Case-based reasoning is not the first artificial intelligence method to combine reasoning and learning, but it makes learning little more than a by-product of reasoning. That feature makes it unique among artificial intelligence methods. In other words, a case-based reasoner that remembers its experiences also learns as it reasons; feedback from early experiences gives it insight into solving problems at a later date. What the Heck is a Case? A case is a contextualized piece of knowledge representing an experience that teaches a lesson fundamental to achieving the goals of the reasoner. The basis for case-based reasoning is that situations recur with regularity; what was done in one situation is likely to be applicable in another situation of similar context. In essence, we begin the case-based reasoning effort by starting with what worked in a previous situation that is similar to the present. Artificial Intelligence and Case-Based Reasoning Some scientists, who have been trying to explain how the human mind thinks, are putting their efforts into something called artificial intelligence. In some areas of artificial intelligence, a great effort first goes into developing a model of how a thought or decision-making processes works. Then, that model is generally applied to all problem situations involving that particular process. Take the Russian wheat aphid, for example. One may formulate a model for managing that insect on one farm in southeast Wyoming. Efforts may then be made to use that model on all farms in Wyoming plus Colorado, Kansas, Nebraska, Oklahoma, and Texas. This generalizes the Russian wheat aphid model, and the knowledge contained in that model is called general knowledge. General knowledge has some important advantages, one being the economy of storage that comes through minimizing the amount of knowledge needed to solve the problem. General knowledge also allows people to deal with uncertainty; we may not know exactly how Russian wheat aphid infestations change in response to temperature and moisture, but we can approximate that change with statistical models. General knowledge, however, has certain disadvantages. One being something called perationalization, which is the difficulty we encounter when we try to operate something specific given general rules. Sometimes, we need to have specific knowledge when performing specific problemsolving tasks. Another disadvantage of general knowledge is that, although it covers the normal, it doesn't tell us how to reason about situations that are different from normal. For example, one Russian wheat aphid management model, developed for use in a drought-prone area that is susceptible to Russian wheat aphid infestations, may simply involve preparing for an insecticide application in early February, and then spraying the insecticide in early March. However, this doesn't help those producers deal with the abnormal situation when late winter and early spring weather is unusually wet, preventing Russian wheat aphid infestations from increasing to economic levels. So, What do Cases Represent? Cases represent specific knowledge that is associated with specific situations. They represent knowledge that is at the operational level. They make that knowledge explicit for performing a task or using a piece of knowledge or using a strategy when trying to accomplish a goal. Cases also represent an experienced situation. That situation, when remembered later, forms a context in which the knowledge embedded in the case is presumed applicable. What does a Case Contain? Cases come in many shapes and sizes. They may cover a situation that evolves over time or they may represent a situation that must be solved at one point in time. In addition, they may represent problemsolving episodes or they may associate the description of a situation with a particular outcome, or they may do both at the same time. Which Cases are Worth Remembering? Normally, tasks that are routinely performed are not considered as cases worth remembering. These are the normal, daily, routinely performed tasks (i.e., they represent the "norm"). We shall call these schemata. Those that are not routinely performed, however, are usually worth remembering. These are experiences that are different from the "norm", and we shall call these "cases". Stated in another way, cases that are worth remembering are experiences that are different in some way from what was expected. They record major variations from the norm. What constitutes a "major variation from the norm"? Well, researchers have formulated a general rule which states, "If the difference is instructive such that it teaches a lesson for the future that could not have been inferred easily from the cases already recorded, then record it as a case." A lesson is earning the normal way of doing something if it is currently unknown how to do it. Therefore, cases represent current differences from the current norm. A Summary of the Discussion thus far: 1. 2. 3. 4. 5. A case represents specific knowledge tied to a context. It records knowledge at an operational level. Cases can come in many different shapes and sizes, covering large or small time slices, associating solutions with problems, outcomes with situations, or both. A case records experiences that are different from what is expected. Not all differences are important to record, however. Cases worthy of recording as cases teach a useful lesson. Useful lessons are those that have the potential to help a reasoner achieve a goal or set of goals more easily in the future or that warn about the possibility of a failure or point out an unforeseen problem. Each case serves two purposes when it is recalled: a) it provides a suggestion on how to solve a problem, and b) it provides a context for understanding or assessing a situation. Primary Processes Required for Case-Based Reasoning Case retrieval is the first primary process of case-based reasoning. This is not done blindly, as only those cases that partially to completely match the context of the problem are retrieved. Retrieving cases involves two intensive efforts: 1) searching the cases and 2) matching cases with the current problem. This is done by a computer program that rapidly scans the library of cases on hand, selects those that at least partially match the context of the present problem, and places them into nonpermanent memory. A ballpark solution is then proposed. This is done by extracting the solution from some of the retrieved cases and proposing it as the solution to the new case. Note that, at this stage of case-based reasoning, only a general solution is proposed; hence use of the term "ballpark" (i.e., within the general area). Ballpark solutions are sometimes selected by the case-based reasoner when it determines that certain portions of the ballpark solution are appropriate for the current problem. Alternatively, the case-based reasoner can select ballpark solutions by determining if one portion of the ballpark solution is more appropriate for solving the problem than others. A third way that a case-based reasoner can select a ballpark solution is to "check" with the goals of the decision-maker to determine which portion or portions of a potential ballpark solution are consistent with those goals. Next, adaptation or justification of the proposed solution is done. When using case-based reasoning to solve problems, adaptation is normally used. This is accomplished by suggesting small changes in the ballpark solution to propose a different solution that is better-adapted to solving the specific problem. Two steps are taken in adaptation: 1) identifying what needs to be changed, and 2) making the changes. When using case-based reasoning to interpret a ballpark solution, justification is employed. In casebased reasoning, justification is the process of creating an argument for the proposed solution by comparing and contrasting the new situation with prior cases. The reasoner then looks both for similarities between the new situation and the selected cases that justify the desired result and differences between the new situation and the selected cases that imply that other factors must be taken into account. Criticism of the proposed solution is then done as a critique before the solution is applied to the new problem. When all the knowledge necessary to evaluate the proposed solution is known, this step is called validation. When all such knowledge is not available to conduct a validation, alternate cases are called from memory and their proposed solutions are compared and contrasted with one another as well as with the currently-considered solution. This provides the decision-maker potential scenarios through which the proposed adapted solution may succeed - or fail. Evaluation of the solution is then conducted after it has been selected, applied, and carried through to completion. This step includes feedback about how the solution was implemented, what went right, as well as what went wrong. It also includes brief explanations about what could have been done to prevent some of the complications when implementing the solution. Of all the steps of the case-based reasoner, this is the most important because it provides the information necessary for the case-based reasoner to learn. It allows the case-based reasoner to "notice" the consequences of its reasoning. In essence, evaluation is the process of judging the goodness of a proposed solution. Storage of the new case is then accomplished by the computer program. Remember, if this new case meets the criteria of representing an appreciable departure from the current norm, and in its solution it teaches a lesson, and also satisfies one or more goals of the reasoner, then it should be saved and indexed for future use. Perhaps the most important part of the memory storage process is the indexing. Decisions on how to index this case has a direct bearing on how efficient the searching and matching algorithms will be. Decision-makers use case-based reasoning when they remember previous situations that are similar to the present for helping them solve a problem. Case-based reasoning is commonly used for: 1) adapting an old solution to a new problem, 2) identifying possible failures when adapting old solutions to new problems, and 3) interpreting the present situation by comparing it to many past situations. Some Generalities about Case-Based Reasoning Case-based reasoning is really a model of one kind of reasoning that makes use of problem solving, understanding, and learning. Further, it connects or integrates these three processes within something called memory. Five Premises of the Case-Based Reasoning Model A premise is something that has been proposed - and proved - as a basis for argument. 1) Referencing or recalling the circumstances of old situations or cases is advantageous when dealing with new situations. In addition, such referencing is usually necessary to understand, interpret, and/or solve problems that are involved with novel (new) situations. Therefore, remembering a case to use in a later problem-solving endeavor, and integrating that case with what is already known, is a necessary learning process. 2) Interpreting - (understanding) - the details of a problem is necessary for its solution because descriptions of problems, past and present, are rarely complete. Such interpretation needs to be done by the decision-maker before case-based reasoning can begin. As the decision-maker begins to understand the various aspects of the problem at hand, then similar cases can be recalled to help her or him solve it; as more details are understood, more cases can be recalled, evaluated, and either kept or discarded. 3) Adapting the solution of an old case to a new situation is almost always necessary because old cases are almost never identical to new situations. In essence, adaptation compensates for the differences between old cases and new situations. 4) Learning occurs as a natural consequence of reasoning and experience. When a novel solution is derived when solving a complex problem, and if that solution is correct, a connection is made between that new solution, the set of circumstances involved, and the set of cases used to come up with that solution. These circumstances and connections and the new solution are stored in memory and indexed so they can be retrieved when they are needed. On the other hand, if that particular solution is not a correct solution, that set of connections to the circumstances and past cases is indexed so the decisionmaker can be warned that potential troubles can arise if these solutions are used. Such learning (i.e., the processes of making connections, remembering past cases, understanding details of the present case, indexing the solutions and such and storing and recalling those solutions) occurs in increments. 5) Evaluating the reasoning process is useful to the learning process. This is usually done through something called "feedback" and "analysis". Feedback is the objective evaluation of the case-based reasoning process, determining what went "right" and what went "wrong". The analysis of that feedback is accomplished by conducting alternate procedures/recalls, running them through the mental process, and determining potential outcomes. Such evaluation is a useful part of the reasoning/learning cycle. These premises strongly suggest that the quality of reasoning when a reasoner uses case-based reasoning depends on the following: 1) its breadth and depth of experience, 2) its ability to understand new situations and correctly interpret them in relation to those old experiences, 3) its ability to adapt old solutions to new problems, 4) its ability to conduct the feedback and evaluation processes and make "repairs" when the solution is not working out well, and 5) its ability to make the correct connections between the new situation, old experiences, old solutions, and new solutions (learning). Recall Starts the Case-Based Reasoning Process After the decision-maker understands the details of the new problem, she or he must begin the reasoning process by selecting a set of representative cases. This set of cases must involve the goals and sub-goals of the decision-maker. Using these selected cases, past successful attempts to solve the problem are proposed as solutions to the new problem. Also when using these selected cases, past failed attempts serve to warn the decision-maker of a potential failure to meet those goals. Interpretation: Understanding a New Problem in Terms of Old Experiences Interpretation is the process of comparing the new situation to the recalled experiences. This is done by the case-based reasoner when it compares and contrasts the old experiences with the situation of the new problem; this is an interpretation of the new problem. Interpretation includes inferred knowledge about the new problem and, sometimes, a classification of the new problem. (To infer something is to come to some state of judgment or conclusion based on facts or information that are known or gathered about the subject.) When solutions to the new problems are compared to solutions of past but similar problems, the case-based reasoner gains an understanding of the positive and negative consequences of choosing some solutions over others. Interpretation is generally used when the problem is not well understood and there is a need to criticize the solution. In contrast, when a problem is well understood (i.e., there is little to no missing information), there is little need for the interpretive process. Adaptation: Fixing Old Solutions to Solve New Problems Fixing solutions that were used to solve previous but similar problems is called adaptation. Adaptation is necessary because no old situation is exactly the same as a new one. Adaptation can happen during the formulation of a solution or after feedback that is derived from evaluating the results of an old solution to a similar problem. Generally, adaptation can be accomplished by substitution or transformation. Substitution is accomplished by substituting, for some part of an old solution, a replacement action or substance. Often this is deemed necessary because of ethical, environmental, logistic, economic, or other constraints. Currently, there are six different recognized methods of substitution (see Kolodner 1993 for more information on these). Transformation methods are useful for changing an old solution into one that will work for the new problem. There are two forms of transformation methods, one called commonsense heuristics and the other called model- or principle-guided repair. The former method involves a small set of general heuristics, or rules, that use knowledge about the relative importance of different parts of a solution to determine whether deletions or substitutions should be made. The latter method is used when the decision-maker has knowledge of the cause and effect "connections" that exist in the system. Improving the Performance of the Case-Based Reasoning Process Artificial intelligence programs that use case-based reasoning to solve problems must have the ability to learn from their experiences. This is done through feedback, evaluation, and repair. Without these, the program would get faster at generating solutions but not better. In other words, they or they would repeat their mistakes. There are two ways that performance is improved through the use of case-based reasoning: 1) becoming more efficient, and 2) becoming more competent. The former occurs when, in the reasoning process, old solutions are simply remembered and then adapted to solve new problems rather than going through the entire case-based reasoning process when solving each new problem. The latter occurs when the reasoner indexes each problem situation, indicating whether or not the solution was effective. Over time, the depth and breadth of problem situations grows and the accompanying indexing system is quite useful in sorting through acceptable and unacceptable solutions for similar problems. In light of these performance-enhancing characteristics, it should be no surprise that performances of case-based reasoners improve when they accumulate new cases and indexes. Accumulating new cases will provide additional familiar contexts for solving problems or providing evaluations. Accumulating new indices will allow the "reasoner" to fine-tune their or its recall of relevant cases. Some Advantages and Disadvantages to Problem Solving with Case-Based Reasoning Advantages: 1) Case-based reasoning allows the reasoner to propose solutions to problems quickly, avoiding the time necessary to derive those answers from scratch. For example, a farmer can remember what was done to solve a present problem with the Russian wheat aphid in years past. Although criticism is necessary before any solution is taken, the farmer is far ahead when the old solution is remembered and she or he is not forced to go through the entire reasoning process, as if the Russian wheat aphid had never before been deal with. 2) Case-based reasoning allows a reasoner to propose solutions in knowledge areas that are not completely understood by the reasoner. Problem situations in some knowledge areas are impossible to fully understand because the solution often depends on as yet unpredictable facets, such as the price of winter wheat in the middle of the next winter. In addition, some knowledge areas are not yet understood but the decision-maker must make a decision anyway. This happens when wheat farmers are forced to make management decision on the Russian wheat aphid when it is not known if the weather will change from being hot and dry to being cool and wet. Under these circumstances, Case-based reasoning allows the decision-maker to make assumptions and predictions based on what worked in the past without having a complete understanding of the knowledge area, such as weather and grain futures forecasting. 3) Remembering previous experiences is particularly useful when warning the decision-maker about potential problems that have occurred in the past, alerting a reasoner to take actions to avoid repeating those past mistakes. Remember, cases can also be helpful in pointing out incorrect solutions. How do they do this? Through the evaluative process. Remember that feedback and analysis of what went "wrong" with a solution to a problem situation is also recorded and stored in memory. In future situations, when the case-based reasoner calls up that case, it reviews the solution as well as the feedback and analysis parts of the case. All of these are taken into account when proposing a new solution to the present problem situation. 4) Cases help a reasoner focus its reasoning on important parts of a problem by simply pointing out what features of a problem are important. This advantage rests on the premise that what was important in similar, past cases, will be important in current problem situations. Be aware that important parts of cases include those that contributed to successes as well as failures. For example, some winter wheat farmers in 1995 delayed making an insecticide application to Russian wheat aphid infestations, when those infestations were already in excess of the economic injury level, in the hopes that the weather would change and become cool and wet and reduce the levels of infestation. Well, the weather did change but the levels of infestation did not decrease; they simply remained steady at that point for the balance of the cropping season. They were therefore forced to pay a commercial applicator more to apply the insecticide - when it wasn't raining - or suffer economic losses and not make an expensive extra pesticide application that year. The evaluative process for that particular year would have indicated that, if the winter wheat crop had not yet developed to the first joint stage, and the current level of Russian wheat aphid infestation is at or in excess of the economic injury level, then an emergency insecticide application should made as soon as possible. Disadvantages: 1) A case-based reasoner might be tempted to use old cases blindly, relying on previous experience without criticizing or validating it context of the new situation. 2) A case-based reasoner (or decision-maker) might allow cases to bias them when solving a new problem. 3) A case-based reasoner may not be able to identify and thus consider the most appropriate cases when solving the present problem with case-based reasoning. These all are potential weaknesses of case-based reasoners. They should serve as a warning to decision-makers everywhere that they should not shirk their duty of using their own minds and experiences to criticize and evaluate potential solutions of case-based reasoners. Remember, casebased reasoners are tools that produce information to be considered when we are trying to make decisions about difficult problems. They are not in themselves, decision-makers.