Data - The Next Quality Disaster American Society for Quality, Baltimore Section Thomas C. Redman, Ph.D. Navesink Consulting Group 732-933-4669 www.dataqualitysolutions.com December 10, 2002 ASQ-DQ Disaster © Navesink Consulting Group My messages Bad data are at the root of the events of national and international importance. They capture our interest and won’t let go. Fortunately, most data quality issues don’t make the newspaper. Unfortunately they are insidiously hidden in everyday work, increasing cost, causing customer dissatisfaction, and making it harder to make decisions. © Navesink Consulting Group Outline • • • • • • Data Quality Defined Data Quality Disasters in the News Everyday DQ Disasters COPDQ We Know What To Do About It Final Remarks: A Clarion Call © Navesink Consulting Group What Makes Quality Data? Data are of high quality if they are fit for their intended uses (by customers) in operations, decisionmaking, and planning (after Juran). Data that’s fit for use free of defects: - accessible - accurate - timely - complete - consistent with other sources - etc. possess desired features: - relevant - comprehensive - proper level of detail - easy-to-read - easy-to-interpret - etc. Customers are the ultimate arbiters of quality!! © Navesink Consulting Group Poor Data Quality in the News “Wall Bursts and Water Pours In, Trapping 9 Miners 240 Feet Down” Though Pennsylvania state regulations require a 200 foot safety wall between adjoining mines, outdated maps of area mines failed to show the miners’ proximity to an existing mine. After 77 difficult hours, all nine men were freed. The New York Times, July 26, 2002, Francis X. Clines © Navesink Consulting Group Poor Data Quality in the News “Right Answer, Wrong Score: Test Flaws Take Toll” The testing industry is coming off its three most problem-plagued years. Its missteps have affected millions of students who took standardized proficiency tests in at least 20 states. An examination of recent mistakes and interviews with more than 120 people involved in the testing process suggest that the industry cannot guarantee the kind of error-free, high-speed testing that parents, educators and politicians seem to take for granted. New York Times, May 20, 2001, Diana B. Henriques © Navesink Consulting Group Poor Data Quality in the News “Indecision 2000: The Ballad of the Ballots” The 2000 Presidential Election in the United States held our attention for weeks as we tried to determine: • the accuracy of various counts and recounts • how the end-to-end "election chain" is supposed to work • the final result and consequences of the above. The Associated Press, November 19, 2000, Ted Anthony and Jerry Schwartz © Navesink Consulting Group Poor Data Quality in the News “Voting: What is, What Could Be” Four to six million votes, a number which is double the population of Chicago, were lost in the 2000 presidential election due to problems with ballots, equipment, registration or at the polling place, according to a joint analysis by experts at the California Institute of Technology and the Massachusetts Institute of Technology. MIT News, July 16, 2001 © Navesink Consulting Group Poor Data Quality in the News “Primary Day in Florida Goes On, and On, and On” The primary election results for the Democratic nomination for governor of Florida were held up for 3 days while Miami-Dade election officials tried to sort out problems at polling locations – despite the new $32 million voting system which replaced the one blamed in the 2000 Presidential Election. The New York Times, September 14, 2002, Adam Nagourney © Navesink Consulting Group Poor Data Quality in the News "Holes in System Hid Links in Sniper Attacks" . According to the authors, "In the month before the sniper attacks that left 10 dead and terrorized in the Washington area, John Muhammad and Lee Malvo are believed to have killed or wounded seven people in four states. But because they kept on the move and the police gave only routine attention to what seemed like run-of-the-mill crimes, investigators say, Mr. Muhammad and Mr. Malvo were able to slip through the cracks, gaps and blind spots where jurisdictions meet and crime-fighting databases end." New York Times, November 29, 2002 © Navesink Consulting Group Poor Data Quality in the News “CIA Fires Officer Blamed in Bombing of Chinese Embassy” Using out-dated information, the CIA selected the address of an armory for a bombing target. At the time of the bombing, however, the building housed the Chinese Embassy. The New York Times, April 9, 2000 © Navesink Consulting Group Poor Data Quality in the News “Mail from INS Stuns Flight School” As the result of mismanaged documents, a Florida flight school received notice form the INS of approval of student visas for two of the September 11th terrorists – six months to the day after the attack on America. USA Today, March 13, 2002, Kevin Johnson © Navesink Consulting Group Poor Data Quality in the News “FBI Director Details Blunder on McVeigh Records” The FBI disclosed that it committed a "serious error" when it failed to turn over thousands of pages of interview reports in the 1995 Oklahoma City bombing case to lawyers for Timothy J. McVeigh, who was sentenced to death for the crime. The New York Times, May 17, 2001, David Johnston © Navesink Consulting Group Data Quality In the News • United Nations Resolution 1441 mandated that Iraq declare any current holdings of weapons of mass destruction • Does anyone seriously think that a 12,000 page document, supported by 4 CD-ROMS will be errorfree? © Navesink Consulting Group Poor Data Quality in the News “The Wrong Foot, and Other Tales of Surgical Error” At least 500 times since 1996, surgeons in hospitals in this country have operated on the wrong arm, leg, eye, kidney or other body part, or even on the wrong patient. The figure does not include nearmisses (when surgeons started to operate on the wrong site or patient because no one collects this information. The New York Times, December 11, 2001, Lawrence K. Altman, MD © Navesink Consulting Group Poor Data Quality in the News “Right Drug, Wrong Patient” In 1997, a nationwide survey conducted by the trade publication Drug Topics, 53 percent of pharmacists admitted having made errors in the preceding two months. A June 1996 survey of 3361 pharmacists in California and Oregon revealed that errors occurred at an annual rate of 324 per pharmacy – nearly one a day! Reader’s Digest, September 1998 © Navesink Consulting Group Poor Data Quality in the News “To Err Is Human: Building a Safer Health System” This report, initiated by the Institute of Medicine, found that: • Adverse events occurred in 2.9 percent of hospitalizations in Colorado and Utah and 3.7 percent in New York. • 8.8 percent of adverse events led to death in Colorado and Utah, compared with 13.6 percent in New York hospitals. • Medication errors alone, occurring either in or out of the hospital, are estimated to account for over 7,000 deaths annually. • More people die in a given year (between 44,000 and 98,000) as a result of medical errors than from motor vehicle accidents, breast cancer or AIDS. • The national costs of preventable adverse events are estimated to be between $17 billion and $29 billion. “To Err is Human,” Committee on Quality of Health Care in America, Institute of Medicine, November 1999. © Navesink Consulting Group Poor Data Quality in the News “Accounting Wars” Between 1997 and 1999, long before the current financial reform outcry, at least 364 companies restated financial statements. Business Week, September 25, 2000, p.158 © Navesink Consulting Group Poor Data Quality in the News “The Great Divide” The Enron scandal, on the other hand, clearly was about us. It told us things about ourselves that we probably should have known, but had managed not to see. I predict that in the years ahead Enron, not Sept. 11, will come to be seen as the greater turning point in U.S. society.” The New York Times, January 1, 2002, Paul Krugman © Navesink Consulting Group Everyday Data Quality Disasters • Pricing errors on the internet – United Airlines sold about 1,000 tickets for all flights from and through Chicago and Denver for $5 – W Hotels – New Years Eve in NYC for $25/night instead of $259. – Hilton Hotels listed rooms for $0 at least 3 times in 6 months. – Amazon.com sold a memory board valued at $329.99 for $9.99 and another worth $999.99 for $10.77 • Hyundai over-reported horsepower ratings by as much as 9.6% on some vehicles sold since 1992. • Bear, Sterns ordered $4 billion stocks sold – order should have been for $4 million. © Navesink Consulting Group © Navesink Consulting Group Cost-of-Poor-Data-Quality COPDQ = + + + + + + Cost to Prevent Errors Cost to Find and Fix Errors Cost to Make Good for Customers Cost of Bad/Delayed Decisions Cost of Lost Opportunities Cost of Damage to Reputation + + Can (usually) be estimated* (Usually) more important *Doing so can be tricky. If the organization will respond to COPDQ, we recommend: – Doing so at the “project level” first, – Then expanding to a supplier or information chain – Working up to the entire company. © Navesink Consulting Group COPDQ – The Evidence • Proprietary studies: “easy-to-measure” (finding and fixing errors) COPDQ are 10-20% of revenue. • The Data Warehousing Institute: estimates that, in the US, $611 billion a year is lost as a result of poor customer data (name and address data). • As a result of the embassy bombing, the US paid the Chinese government $27 million. But the real cost is much higher. • As a result of the 2000 Presidential Election, Florida spent $32 million on a new system. The federal government will spend $3.9 billion in the next 3 years to help states replace outdated voting machines and train personnel. • The error on W Hotel’s website cost $175,266 in bookings to one hotel. The web error was posted for only 7 days. © Navesink Consulting Group Data Underlie Everything We Do Perhaps they are “hidden in plain sight” © Navesink Consulting Group A Database is Like a Lake To Clean Up The Lake, One Must First Eliminate The Sources Of Pollutant © Navesink Consulting Group It is so easy for accountability to shift downstream!!! Here’s how you do number 3, son cos2(x) + sin2(x) = 1 © Navesink Consulting Group Lead, follow, or risk being acquired Leading organizations have improved by two orders of magnitude and reduced COPDQ by two-thirds. Best-practice: • Adopt the philosophy of prevention • Leadership • Management Accountability at the points of data creation • Focus on customer • Measurement, control, improvement • Data Culture © Navesink Consulting Group A Clarion Call Poor quality data are the norm and no one is immune to their effects. Most issues are mundane, but their impact is stunning. They: • Increase cost – approximately 20% of revenue • Anger customers • Make decision-making more difficult • Make it harder to implement new technologies • Put company image at risk © Navesink Consulting Group