Regulation and Markets in a Certification Society: The Case of Financial Reporting Karim Jamal and Shyam Sunder Carnegie Mellon University August 25-26, 2006 Summary Examination of audit certification in the broader context of certification activities in the economy Extent and nature of certification services – Ubiquity of certification and conflicts of interest – Fineness of certification categories (2-100) – Merits of fineness of categorization Field study of unregulated market for certification of baseball cards – Twenty three firms – Market value and strictness of grading services Implications for corporate governance, auditing Regulatory Attempts to Improve Auditing Seven decades of regulation under the SEC Major changes in regulation in 2002 (SOX) – Ban on certain consulting services for audit clients to minimize conflict of interest – Corporate governance reforms to promote auditor independence – Transfer of responsibility for setting auditing standards to a government agency (PCAOB) – Transfer of responsibility for oversight of auditing to a government agency (PCAOB) Controversy over whether these reforms improve auditing (Kinney et al., 2004) A Broader Context for Examination of Auditing Certification services are widely available in many sectors of the economy (Power 1994) Try to understand audit market as a special case of markets for certification services Great variations on the range and characteristics of certification services – 2-100 point scales – Cross-selling of services by certifiers – Assessing the quality of certification service – Competition in markets for certification services Implications for regulation of audit services, and recent reforms Extent of Certification Services In 1996, US Government documented 93,000 national standards developed by 80 government and 604 private agencies (Jamal and Sunder 2006) National and international standards cover virtually all aspects of the economy (Jamal and Sunder 2006) To what extent are standards accompanied by availability of services to certify compliance? The Audit Society “Audit explosion” in society (Power 1994): education, healthcare, government supported activities Demand for audit fueled by political demands for accountability and control (Power 1999) U.S. audit firms attempted to expand and re-label their “assurance” services into other sectors of the economy in the 1990s (Elliott 1998) Retail customer as the new client for services Audit firms failed in attempt to expand into e-commerce assurance market: could not compete with the business models of BBB Online and TRUSTe (Jamal, Maier and Sunder 2003) Attempt to empirically assess the extent of availability of certification services Sample Selection Selected a sample of 817 items sold online and offline during June 12-July 25, 2004 – 400 items from eBay.com (online) – 358 items from BLS Producer Price Index – 59 items from BLS Consumer Price Index Types of Certification Services Expert opinion about compliance with formal written standards (91%) Expert opinion in absence of formal standards (7%) Ratings given by lay people (no standards) 1% Popularity/activity meters (best-seller lists for books, music, films, TV, etc.) 0.5% Total: 99.5% have some kind of certificate All four kinds of certification available for 40% Table 1: Certification Services for Products Sold Online and Offline in the US PANEL A EBay (N=400) PPI/CPI (N=417) Total (N= 817) % of Total (817) Expert Opinions Based on Written Standards 344 399 743 91% Expert Opinions Without Written Standards 45 14 59 7% Lay People Ratings 5 3 8 1% Meter 3 0 3 0.5% No Certification / Rating 3 1 4 0.5% All Certification Available (Expert Opinions Based on Written Standards, Expert Opinions Without Written Standards, Lay People Ratings, and Meters) 221 101 322 40% Expert Opinions Based on Written Standards only 4 110 114 14% Expert Opinions Based on Written Standards, Lay People Ratings and Meter 33 57 90 11% Expert Opinions Based on Written Standards and Expert Opinions Without Written Standards 47 34 81 10% Expert Opinions Based on Written Standards, Expert Opinions Without Written Standards, and Lay People Ratings 17 34 51 6% Expert Opinions Based on Written Standards, and Meter 8 28 36 4% Expert Opinions Based on Written Standards, Expert Opinions Without Written Standards, and Meter 11 15 26 3% Expert Opinions Based on Written Standards, and Lay People Ratings 3 20 23 3% All other combinations 56 18 74 9% Panel B: Detailed Breakdown of Certification Activities Extent of Certification Results suggest support for Power’s characterization of ours as a “audit society” AICPA failed in its attempt to fill what they thought were empty spaces in the market for assurance services Although Power (1994) focused his study on the public sector, the certification is also ubiquitous in the private good space Demand for certification for private goods must be driven by broader economic forces, not just politics Data Obtained from One Official Partner of eBay (XYZ) 128 experts Offer opinions on 1,850 separate items $9.95 fee for basic service $29.95 for enhanced service Experts do not follow any written standards Opinions do not reference any written standards Provides descriptions of experts (accreditation by professional institutes, education, related business, relevant experience): two examples Appraiser Information Name: Location: # of Appraisals Completed: Thomas Joyce ILLINOIS, UNITED STATES 978 Appraiser in 75 category(s) Category(s) Books, Maps, Manuscripts and Periodicals About Thomas Joyce Thomas has been a professional rare book dealer for nearly thirty years. He is the regular book/map/manuscript appraiser for "The Appraisal Fair" on HGTV. Organizations Member: Antiquarian Booksellers Association of America, International League of Antiquarian Booksellers, The Midwest Antiquarian Booksellers Association Appraiser Information Name: Location: Website: # of Appraisals Completed: Anita Bartlett-Picarella NY, UNITED STATES Bartlett and Picarella 275 Appraiser in 144 category(s) Category(s) Fine art, paintings, sculptures, etc. About Anita Bartlett-Picarella Anita is the president and senior certified appraiser in the firm Bartlett and Picarella. She received her Bachelor of Science degree from St. John's University in NY. She continued her education taking Sotheby's American Decorative Art program and continued her studies with the Bard Graduate School of Decorative arts attending symposiums, seminars and doing independent study. Anita is an avid attendee at numerous workshops and private study groups relating to art and appraising. She lectures nationwide on the subject of "Standards and Methodology of Fine Art Appraisals" Certifications Board Member, Thomas Paine National Historic Association State Certified by the New York and New Jersey Departments of Economic Development Certified Better Business Bureau Member, Bronxville Chamber of Commerce Senior Certified Member, Appraisers Association of America Organizations IRS Qualified Appraiser Appointed Neutral Appraiser by Supreme Court, County of Westchester Self-Assessment of XYZ’s Experts 128 experts as of July 12, 2004 39% had formal accreditation from a professional body 25% formal relevant educational credentials 70% ran a business in the products for which they provided certification Only 9% reported themselves to be hobbyists Table 2: Qualifications of Experts Providing Opinions on XYZ Website Accreditatio n Educatio n Business Hobby 1. Art and Antiquities (n=14) 64% 50% 71% 7% 2. Books, Maps, Manuscripts (n=5) 20% 20% 80% 20% 3. Clocks, Watches and Timepieces (n=4) 25% 25% 100% 0% 4. Clothing, Linens, Rugs and Quilts (n=6) 33% 17% 100% 0% 5. Coins, Stamps, Numismatics (n=7) 29% 14% 100% 0% 6. Electronics (n=3) 33% 0% 67% 33% 7. Famous People (n=4) 25% 0% 100% 0% 8. Furniture and Accessories (n=8) 50% 0% 75% 13% 9. Glass (n=5) 40% 20% 100% 0% 10. Guns, Knives and Swords (n=5) 60% 0% 100% 0% 11. Jewelry (n=6) 50% 33% 83% 0% 12. Knick Knacks and Collectibles (n=7) 14% 28% 86% 0% 13. Music (n=4) 25% 25% 75% 0% 0% 100% 100% 0% 15. Photography, Cameras, Projectors (n=3) 33% 67% 67% 33% 16. Porcelain, Ceramic and Pottery (n=10) 50% 20% 60% 20% 17. Silver (n=7) 43% 29% 71% 14% 18. Sports (n=6) 17% 17% 67% 33% 19. Tools, Kitchenware & Equipment (n=3) 33% 0% 100% 0% 20. Toys, Dolls, Games (n=9) 44% 11% 78% 0% 21. Transportation (n=4) 25% 25% 25% 25% 22. Wine (n=2) 50% 50% 50% 0% 23. General Appraisers (other) (n=6) 33% 67% 83% 17% Total (n=128) 50 (39%) 14. Nature’s Treasures (n=1) 32 (25%) 101(79% 12 (9%) Fineness of Ratings/Certification Auditor reports are essentially pass/fail Auditor gains a broad understanding of the quality of internal controls, governance, accounting policies, estimates, and disclosure The coarse pass/fail grading does not attempt to convey this detailed information to the client No incentives for companies to report more Shouldn’t a finer report be more informative to the clients Implications for Auditing Conflict of interest appears to be pervasive in certification services in general (79%), not confined to auditing Confirm Moore et al. (2006) Do we (the academic researchers) fall in this category? Conflict of interest as a norm, not exception Unacceptability of consulting services to audit clients in a regulated market (Francis 2004) contrasts sharply with this data from unregulated domains – Clients pay cash for opinions/assessments of self-proclaimed experts who have known conflicts of interest – Is it possible that clients have other means of protecting themselves from self-serving advice, and find it worthwhile to pay for such services? – Could clients of audit services do the same? – Could the prohibition of consulting services to audit clients be a case of regulatory overkill? Economic Theory Dubey and Geanakoplos (2005): In general, an optimal grading scheme should have an intermediate level of fineness – Coarse reports have less information and do not motivate the reporting agent to exert more effort – Fine reports magnify the consequences of measurement errors – 3-10 point grading scale may be optimal Grading scheme should create a small elite Absolute grading schemes dominate relative Data on the Fineness of Government Standards Visited websites of 80 departments of the federal government that set standards (Toth 1996) We able to access standards of 64 (80%) of these agencies Examined the standards for fineness 53/64 (83%) set minimum requirements (pass/fail only) 11/64 (17%) use finer grading (e.g., AAA beef) Given lack of funds and authority, agencies my be induced to use finer scales of quality SEC’s shortage of staff/funding does not seem to induce finer scales from that agency DG: Most federal agencies use absolute standards, but do not set elite grades or use intermediate fineness scales (FDA is an exception) Greater Variability in Fineness of Scales in Private Standards Pass/fail (e.g., UL) Multiple seals (TRUSTe for e-commerce privacy, Jamal, Maier, Sunder 2003, 2005) 10-100 point rating scales (baseball cards, Consumer Reports on cars) Often characterized by competition among rival standards and scales Rating scales as a dimension of competition among services (Jin, Cato and List 2004) Market for Baseball Card Certification Services Jin, Cato and List (2004) – Dominant player is PSA: 1991, 10-point scale with 1 point inc., not graded on curve, 10% cards get 10 (no super elite grade) – BGS: 1999, 10-point scale with ½ point inc., numerical and qualitative labels (mint), subgrades for corners, centering, surface, edges; 0.1% cards get 10, publishes grade distribution on-line; three brands of service (BVG, BGS, and BCCG) – SGC: 1999, smaller market share, 100-point scale (confusion, conversion table to 10-point scale) – 20 other smaller players (Table 3), largely consistent with DG predictions (no curve, intermediate fineness, super elite) Name Year Found ed Scale Catego ries Gradin g Cost Guara ntee Cro sssell er Feature 1. Accugrade (ASA) 1988 13 $5-15 No No Invented 10-point scale + First online rating agency 2. Professional Sports Authentication (PSA) 1991 10 $9-50 Yes Yes Membership Fee ($99 – get 6 cards graded free) + Largest Market Share 3. KSA 1996 14 $12-19 No No Canadian 4. American Authentication (AAI) 1996 10 $12-22 No No 5 card minimum per order 5. Finest Grading (FGS) 1997 14 $5-25 No Yes Value Pricing Card Shows 6. Map Industries 1998 14 $8 No No Free Shipping 7. Beckett Grading Service (BGS) –Vintage Grading (BVG) –Collectors Club (BCCG) 1999 19 $8-25 $9-26 $5 Yes Yes Price guide + provide subgrades for centering, corners, edges, and surface 8. Sportscard Guaranty (SGC) 1999 18 $7- 50 No Yes Started with 100point scale 9. The Final Authority (TFA) 1999 19 $5-13 Yes No Value Pricing 11. Advanced Grading (AGS) 2000 11 $9-25 No No Top view holders 12. Mint Grading Services 2000 18 $6-20 Yes No Customer chooses grade. If MGS disagrees, pay $1 only 13. CTA Grading Experts 2000 11 $10-30 No No Computer grading 14. Bear Stats Grading (BSGS) 2002 14 $10-20 No No Value Pricing 15. Global Authentication (GAI) 2002 19 $6-20 No Yes Former PSA Experts + Dealer Focused 16. Pro Sports Grading (PRO) --- 22 $9-50 No No 2 experts grade each card 17. Professional Grading Service (PGS) --- 19 $5-35 Yes No Help to post on eBay 18. World Class Grading (WCG) --- 20 $5-20 Yes No Value Pricing 19. Champs Grading Service (CGS) --- 10 $2-3 Yes No Racing Specialists 20. Grade Tech --- 21 $14-50 No No Computerized grading 21. Premier Grading (PGI) --- 20 $6-20 No No Value 22. Gem Trading - Gem Elite --- 9 $8-15 $12-19 No No Value 23. Sports Memorabilia --- 19 $6-8 No No Value / Older Cards Comparison with the Market for Audit Services Only binary pass/fail reports, little further detail permitted Possible bases for auditor reputation: size, care in client selection, monitoring GAAP compliance, premium services If the purpose of SOX is to convey information about internal controls, will switch from pass/fail to graded report on IC, governance, quality of accounting methods and governance give better value to investors? The current system suppresses the detailed knowledge gained by the auditor; more consistent with govt. not private markets Value of Certification in Sports Card Market Prospective sellers of sports cards can hire a certification agency to grade and certify their merchandise before selling Issuers of securities are required to hire auditors to add credibility to their FSs Independence and audit quality are difficult to observe in the audit market Baseball card market includes pure and multi-service certifiers, allows a cleaner measure of “audit quality” in grading strictness, observation of an unregulated market and dimensions of competition (e.g., value pricing, computer grading, multiple experts grading the same card, letting customers choose their own grade) Anxiety about race-to-the-bottom and independence in accounting (Dye and Sunder 2001) Data for Assessing the Value of Certification Services 321,045 cards traded on eBay during August 19September 3, 2004 Partitioned cards by decades of origin, singlerookie, graded-ungraded (Table 4) Randomly selected 1,000 rookie and 1,000 singles cards from graded cards Market shares of six major firms (Table 5) in the sample of graded cards (PSA 78 in singles, 39 rookie; BGS 6 singles, 34 rookie) Tables 5A, B Table 4: Baseball Card Certification Services (Baseball Cards Traded on eBay, N=321,045) Singles (272,399 cards) Issue Date Grade d UnGrade d Pre1930s 1,935 2,933 1930s 1,376 1940s Total Rookies (48,646 cards) %Grade d Grade d UnGrade d Tota l %Grade d 4,868 40% ---- ---- ---- ---- 1,689 3,065 45% ---- ---- ---- ---- 417 1,448 1,865 22% 20 218 238 8% 1950s 6,160 22,869 29,029 21% 59 606 665 9% 1960s 7,291 28,007 35,298 21% 161 750 911 18% 1970s 6,858 14,171 21,029 33% 303 1,221 1,52 4 20% 1980s 3,692 8,533 12,225 30% 5,470 6,378 11,8 48 46% 1990s 2,693 25,163 27,856 10% 3,669 9,578 13,2 47 28% 2000s 1,356 135,808 137,16 4 1% 2,608 17,605 20,2 13 13% Total 31,778 240,621 272,39 9 11% 12,290 36,356 48,6 46 25% Assessing Strictness of Grading Jin, Cato and List (2005) gave the same 212 cards to PSA, BGS and SGC online and to three offline dealers Average scores: 8.5 for BGS and two dealers; 8.7 for PSA and one dealer, and 8.9 for SGC (BGS tighter cut-offs, precise); Table 5C Tables 5A and B: empirical frequencies in our sample (selfselection bias): BVG’s premium service stands out (GAI, SGC) Becket’s value brand (BCCG) is lower priced, also has relaxed standards GEM gives the top grade of 10 to 73% percent of the singles cards In rookie Table 5B: roughly similar results Is the market able to adjust itself for differences in grading? Table 5C: Frequency of Grades Given by 3rd Party Certification Services in Field Experiment (Source: Jin et al., 2005, Table 2) 3rd Party Grader PSA BGS SGC Kevin Rick Rodney Total Total No. of Cards 10 212 212 212 212 212 212 11 0 13 0 0 0 24 9.5 0 1 11 1 13 9 8.5 8 7.5 Grade 7 134 40 134 40 57 120 525 124 49 129 92 62 456 66 43 11 37 45 25 227 3 3 4 3 2 15 1 2 2 1 2 0 8 6.5 0 0 0 0 0 6 5.5 5 0 0 0 0 1 2 3 0 0 0 0 0 0 0 0 0 0 0 0 0 4.5 0 0 0 0 0 4 Average Score 0 0 0 0 1 0 1 8.7 8.5 8.9 8.5 8.5 8.7 8.6 Table 5A: Frequency of Grades Given by 3rd Party Certification Services (For a Sample of 1,000 Graded Single Baseball Cards) Grade 3rd Party Grader Market Share % 10 9.5 9.0 8.5 8.0 7.5 7.0 6.0 5.0 < 5 Total Average Score PSA 78.1 69 1 149 0 283 0 129 61 33 56 781 7.5 BGS 3.2 0 11 10 7 1 0 0 1 0 2 32 8.5 BCCG 1.8 9 0 8 0 0 0 0 1 0 0 18 9.3 BVG 1.1 0 0 2 0 0 0 0 3 2 4 11 5.1 GEM 1.1 8 0 0 0 3 0 0 0 0 0 11 9.5 GEM-E 0.3 1 2 0 0 0 0 0 0 0 0 3 9.7 SGC 5.3 2 0 3 2 7 4 6 8 12 9 53 6.1 GAI 4.6 0 1 3 3 12 3 8 5 4 7 46 6.6 PGS 1.5 0 0 6 2 6 0 0 1 0 0 15 8.3 Other 3.0 13 3 4 0 2 2 3 1 1 1 30 8.7 Total 100 102 18 185 14 314 9 146 81 52 79 1000 Table 5B: Frequency of Grades Given by 3rd Party Certification Services (For a Sample of 1,000 Graded Rookie Baseball Cards) Grade 3rd Party Grader Market Share % 10 9.5 9.0 8.5 8.0 7.5 7.0 6.0 5.0 < 5 Total Average Score PSA 36.8 105 0 159 0 80 0 17 4 2 1 368 8.9 BGS 22.7 1 85 73 37 13 14 1 1 1 1 227 8.9 BCCG 10.4 87 0 16 0 0 0 0 1 0 0 104 9.8 BVG 0.9 0 0 0 2 0 1 0 1 2 3 9 5.3 GEM 11.6 116 0 0 0 0 0 0 0 0 0 116 10.0 GEM-E 4.2 42 0 0 0 0 0 0 0 0 0 42 10.0 SGC 2.9 4 0 13 6 4 1 1 0 0 0 29 8.8 GAI 2.3 2 9 8 2 1 0 0 1 0 0 23 9.1 PGS 1.6 16 0 0 0 0 0 0 0 0 0 16 10.0 Other 6.6 33 2 20 8 2 0 0 0 1 0 66 9.4 100% 406 96 289 55 100 16 19 8 6 5 1000 Total Pricing and Value of Baseball Card Certification Services PSA charged a $99 membership fee (dropped recently) Table 3: Price varies ($2-50), extra for faster turnaround For each graded card in the sample, we located an ungraded card matched by player, maker, year of issue, single-rookie) For unrated cards, we gathered price estimates from Beckett Baseball Card Monthly Guide (August 2004, Issue #234 online) For rated cards, we recorded player, year, maker, grader, grade, buyer reputation, seller reputation, number of bids, and selling price Table 6: Gross and net returns on baseball card grading services provided by six major service providers Table 6: Average Returns to Grading of Rookie Cards (by Grader and Decade of Issue) Rater 2000+ Gr.=9 % 1990’s Gr.= 9 % 1980’s Gr.=9 % 1970’s Gr.= 9 % ----- 44 86 176 ----- ----- ----- 55 124 128 ----- 303 ----- 201 (27) (65) (1) ----- (2 ) 25 47 178 (50) (53) (5) 683 112 (14) (15) (11) (3) (68) (72) (25) 20 80.6 (23) 154 (38) (24) ----- ----- ----- ----- 2000+ Gr. =10 % 1990’s Gr.=10 % Number of Cards Gross Return % Net Return % BGS 227 264 46 333 ----- GAI 23 198 92 185 SGC 29 125 4 PSA 368 238 BCCG 104 GEM 116 1980’s Gr.= 10 % Returns to Audit Rater Gross Return Net Return BGS 264% 46% GAI 198% 92% SGC 125% 4% PSA 238% (2%) BCCG 112% (14%) GEM 80.6% (23%) Gross and Net Returns to Certification Gross returns are positive for all six services Net returns: positive for strict graders (BGS, GAI); about zero for medium graders (SGC and PSA); negative for lenient graders (BCCG, GEM) Older cards earn a higher return than more recent cards Grade 10 cards earn a higher return Overall it is better to get a rating of 9 from a strict grader than a rating of 10 from a lenient grader Table 7: Regression Analysis Ri = 1+ 1Yeari + 2Gradei + 3GPSAi + 4GBGSi + 5GBecketti +6GGAIi + 7GGEMi + 8GSGCi + ei Regression (990)df, R2 = 0.1747, p < 0.001. (Adjusted R2 = 0.168) Intercept Year Coefficient Std Error 108.68 12.60 T Statistic 8.63 *** -0.058 0.006 -8.97 *** Grade 0.72 0.06 12.65 *** GPSA 0.37 0.16 2.22** GBGS 1.12 0.18 6.25*** GECKETT -0.18 0.19 -0.98 1.21 0.31 3.87 *** GGEM -0.73 0.18 -3.97*** GSGC 0.48 0.28 1.68* GGAI *** p <0.001 ** p < 0.01 * p < 0.05 Regression Analysis Table 7: Year, grade and identity of grader have a significant effect on net return from the certification service Certification service is more valuable for older cards (selection effect?) The returns are ranked by strictness of grading scales, negative for the lenient graders Easy grade of 10 from GEM does no good for the value of the card No race to the bottom Beckett able to provide different brands of service, and market is able to differentiate among them without getting confused Back to Auditing Regulatory objectives of auditing (quality of certification, independence, etc.) In baseball card market 17/23 offer only certification services but six (including all major) players offer related services (pricing guides, dealers, magazines, shows) as well as handle other collectibles Cross-sellers dominate the market and are able to collect premium price for their service, yielding positive net returns to their clients Independence appears to be neither necessary nor sufficient condition for high quality service in this certification market Why? Skepticism about Deregulation Will a deregulated market for auditing function properly (race to the bottom, Dye and Sunder)? Recent policy changes (PCAOB) have gone in the other direction Lenient graders generate negative returns to their clients Jamal, Maier and Sunder (2003) on market for privacy seals in e-commerce dominated by higher quality Does cross-selling of services promote or inhibit grade inflation? What are the real reasons for poor enforcement of GAAP by audit firms? Concluding Remarks Evidence on ubiquity of certification Certification by government agencies is mostly binary (pass/fail) Private certification has finer scale Field study of baseball card grading market – Dominated by cross sellers (conflict of interest) – Multiple dimensions of competition – No race to the bottom Open Questions Will the tough auditors win or lose? Precision of grading – Significant? Should auditors issue more nuanced reports? Does it pay to get audited (vs. reputation, warranty and disclosure) Will de-regulated audit market lead to quality differentiation?