King of the Mountain: Finding Short-Term Efficacy in the Shiller P/E Ratio Rob Arnott Chairman Oct 19, 2015 King of the Mountain » CAPE in forecasting subsequent returns › Powerful at forecasting long-horizon returns › Less so for short-horizon returns » CAPE and macro regimes › Link with real interest rate and inflation › The 3-D valuation mountain » Application › Potential of forecasting short-term returns 2 CAPE in forecasting subsequent returns 3 Valuation Ratios Mean Revert at Long-Horizon »Low (high) PE implies high (low) future returns CAPE Price/Earnings Ratio (CAPE), United States, 1881.01–2015.09 Source: Prof. Robert Shiller online data. 4 High Stock Valuations Produce Lower Returns CAPE vs. Subsequent 10-year Real Returns: January 1990 – June 2015, United States Source: Research Affiliates, LLC, based on Shiller, Bloomberg, and FactSet. As of June 30 2015. U.S. represented by S&P 500. 5 Same Relationship in EM Equities CAPE vs. Subsequent 10-year Real Returns: January 1990 – June 2015, EM Source: Research Affiliates, LLC, based on Bloomberg and FactSet. As of June 30 2015. U.S. represented by S&P 500. EM represented by MSCI EM Index. 6 EM Risk is Priced Again Relative Valuation Ratios vs Relative Subsequent Returns: January 1990 – June 2015 Source: Research Affiliates, LLC, based on Shiller, Bloomberg, and FactSet. As of June 30 2015. U.S. represented by S&P 500. EM represented by MSCI EM Index. 7 CAPE is Powerful at Forecasting Long-Horizon Returns, Less So for Short-Horizon Returns Real Returns Shiller PE vs. Subsequent 10-Year Real Returns, 1926-2013 Source: Research Affiliates, based on data from Global Financial Database, Robert Shiller online data, and Morningstar EnCorr. 8 CAPE is Powerful at Forecasting Long-Horizon Returns, Less So for Short-Horizon Returns Real Returns Shiller PE vs. Subsequent 5-Year Real Returns, 1926-2013 Source: Research Affiliates, based on data from Global Financial Database, Robert Shiller online data, and Morningstar EnCorr. 9 CAPE is Powerful at Forecasting Long-Horizon Returns, Less So for Short-Horizon Returns Real Returns Shiller PE vs. Subsequent 1-Year Real Returns, 1926-2013 Source: Research Affiliates, based on data from Global Financial Database, Robert Shiller online data, and Morningstar EnCorr. 10 CAPE Varies Across ―Growth Cycle‖ » High valuation may be justified under the right conditions Average CAPE Average CAPE across Business Cycle, 1926–2013 Long-Horizon Average The essence of our research: adding one illustrative measure of economic condition to enhance our understanding of the mean-reversion target for CAPE *Growth / Slowdown: When economy is growing faster / slower than trend. Source: Prof. Robert Shiller online data, Morningstar EnCorr, and St. Louis FRED. From Jan-1926 through Sep-2013. 11 CAPE and Macro Regimes 12 Link Between CAPE and Real Interest Rates » Median CAPE plummets when… › Real interest rate is negative reflecting a desire to aggressively stimulate the economy, or › Real interest rate unusually high reflecting a desire to rein in an overheated economy Median P/E at Different Real Yield* Regimes (United States, 1880.12–2013.12) 19.6 16.4 17.1 17.8 17.2 12.2 10.7 14.8 10.5 *Real yield is defined as 10-year government bond yield less the 3-year inflation. Source: Research Affiliates, based on data from Global Financial Database. See also Leibowitz and Bova (2007). 13 Link Between CAPE and Inflation » Median CAPE also plummets when… › Inflation is unusually high increased money supply without growth › Inflation is zero or negative decreased price leads to lower production Median P/E at Different Inflation* Regimes (United States, 1880.12–2013.12) 18.9 17.6 16.0 16.2 20.3 18.7 15.9 14.7 9.2 Why these results matter? Fed policy intends to depress interest rates for as long as inflation remains low. *Inflation is defined as 1-year trailing inflation, calculated using 12-month changes in the Consumer Price Index (CPI). Source: Research Affiliates, based on data from Global Financial Database. 14 The Macroeconomic Sweet Spot for A Soaring CAPE » Common belief: rock-bottom levels of inflation and real interest rates » 3-D valuation mountain: moderate levels of inflation and real interest rates Median P/E Median P/E at Different Inflation and Real Yield Regimes (United States, 1880.12–2013.12) - Source: Research Affiliates, based on data from Global Financial Database. 15 Median P/E Modeled P/E A Nonlinear Model Based on the Valuation Mountain - Source: Research Affiliates, based on data from Global Financial Database. 16 Use Gaussian Model to Fit a Continuous Function » Polynomials are simple, but are not bounded » Gaussian function is a better alternative » Seven parameters are found by minimizing the weighted* sum of squared errors on a 10 x 10 grid *Weights are proportional to square root of numbers of observations within each region of the grid and inversely proportional to the standard deviation of those observations. 17 Gaussian Model Surface Curves » The ―sweet spot‖ for CAPE in U.S. equity market › Low inflation (~1.5%) › Moderate real interest rate (~3%) Source: Research Affiliates, based on data from Global Financial Database. 18 Gaussian Model—Statistical Fit » Statistical Fit of Various Models Used to Explain P/E (United States, 1880.12–2013.12) Coefficient Model Inflation Real Rate Non-linear Model Linear Model 1 50.7% -2.87 Linear Model 2 Linear Model 3 Adjusted R2 -1.85 10.1% 3.67 9.7% 2.21 12.3% Source: Research Affiliates, based on data from Global Financial Database. 19 Application: Improving Return Forecasts 20 Forecasting Returns with CAPE and Modeled CAPE United States, 1880.12–2013.12 » CAPE is more powerful in forecasting long-horizon returns » Adjusted CAPE enhances short-term return forecast Traditional Forecasting Regression using CAPE only 𝑟𝑡+𝑘 Forecasting Regression using both CAPE and Modeled CAPE 𝑃𝑡 = 𝛼 + 𝛽 ln + 𝜀𝑡+𝑘 𝐸𝑡−3 𝑟𝑡+𝑘 = 𝛼 + 𝛽 ln 𝑃𝑡 − 𝑓 𝑖𝑡 , 𝜋𝑡 𝐸𝑡−3 + 𝜀𝑡+𝑘 Horizon (Months) Coefficient Newey-West t-Stat Non-overlapping t-Stat Adjusted R2 Horizon (Months) Coefficient Newey-West t-Stat Non-overlapping t-Stat Adjusted R2 120 60 36 12 6 1 -0.07 -0.09 -0.09 -0.10 -0.09 -0.08 -6.74 -3.74 -2.97 -3.00 -2.75 -1.80 -2.07 -3.12 -2.18 -2.58 -2.17 -1.79 33.3% 20.3% 13.0% 4.7% 2.0% 0.2% 120 60 36 12 6 1 -0.05 -0.08 -0.11 -0.16 -0.16 -0.18 -2.29 -2.01 -2.33 -3.26 -3.63 -3.22 -0.48 -1.63 -1.72 -2.46 -3.20 -3.22 8.9% 8.7% 9.1% 4.9% 3.0% 0.6% Source: Research Affiliates, based on data from Global Financial Database. 21 Intuition of Observations » CAPE is a strong predictor for long-horizon (510 years) returns › Long-term valuations mean-revert, with high statistical significance » CAPE is a weak predictor for short-horizon (up to 3 years) returns › Current macroeconomic environment affects normal CAPE › Provides better estimate of fair market valuation › Leads to better ability to forecast returns at short-horizon › Improves results shorter than 3 years in U.S. › Current CAPE does not matter, except relative to model CAPE » Transitioning from short to long horizon › Usefulness of ―current‖ macroeconomic variables fades › The macroeconomic environment in 510 years will differ from today 22 Global Markets 23 Link Between CAPE and Real Interest Rate Developed Countries (excluding U.S.) » Remarkable similarity to what we observe in U.S. equity market Median P/E at Different Real Yield* Regimes (Developed Countries, 1972.03–2013.12) 22.0 19.2 19.8 18.0 17.0 14.5 13.3 16.3 14.3 *Real yield is defined as 10-year government bond yield less the 3-year inflation. Pooled data across 22 countries: Australia, Austria, Belgium, Canada, Denmark, Finland, France, Germany, Greece, Hong Kong, Ireland, Italy, Japan, Netherlands, New Zealand, Norway, Portugal, Singapore, Spain, Sweden, Switzerland and the U.K. Source: Research Affiliates, based on data from Global Financial Database. 24 Link Between CAPE and Inflation Developed Countries (excluding U.S.) » Weaker mountain-shaped relationship due to fewer episodes of very low inflation in the international sample Median P/E at Different Inflation* Regimes (Developed Countries, 1972.03–2013.12) 20.7 21.9 22.2 20.2 15.6 17.8 14.6 14.2 9.6 *Inflation is defined as 1-year trailing inflation, calculated using 12-month changes in the Consumer Price Index (CPI). Pooled data across 22 countries: Australia, Austria, Belgium, Canada, Denmark, Finland, France, Germany, Greece, Hong Kong, Ireland, Italy, Japan, Netherlands, New Zealand, Norway, Portugal, Singapore, Spain, Sweden, Switzerland and the U.K. Source: Research Affiliates, based on data from Global Financial Database. 25 Observe Similar Valuation Mountain in Developed Countries (excluding U.S.) » Common belief: rock-bottom levels of inflation and real interest rates » 3-D valuation mountain: moderate levels of inflation and real interest rates Median P/E Median P/E at Different Inflation and Real Yield Regimes (Developed Countries, 1972.03–2013.12) - Source: Research Affiliates, based on data from Global Financial Database. 26 Gaussian Model – Surface Curves Developed Countries (excluding U.S.), 1972.03–2013.12 » Near-identical location of the ―Sweet Spot‖ Developed ex-US PE Curve US PE Curve (from earlier) Source: Research Affiliates, LLC, based on data from Global Financial Database 27 Gaussian Model – Surface Curves Developed Countries (excluding U.S.), 1972.03–2013.12 » Near-identical location of the ―Sweet Spot‖ Developed ex-US PE Curve US PE Curve (from earlier) Source: Research Affiliates, LLC, based on data from Global Financial Database 28 Gaussian Model – Statistical Fit Developed Countries (excluding U.S.), 1972.03–2013.12 » Statistical Fit of Various Models Used to Explain P/E (Developed Countries, 1972.03–2013.12) Coefficient Model Inflation Real Rate Non-linear Model Linear Model 1 29.9% -8.13 Linear Model 2 Linear Model 3 Adjusted R2 -8.46 23.0% -1.95 0.8% -3.31 25.3% Source: Research Affiliates, based on data from Global Financial Database. 29 Forecasting Returns with CAPE and Modeled CAPE Developed Countries, 1972.03–2013.12 » Similar to the U.S sample » Significant increase in forecasting power at short horizons Traditional Forecasting Regression using CAPE Only 𝑟𝑡+𝑘 = 𝛼 + 𝛽 ln Horizon (Months) Coefficient 120 60 36 12 6 1 -0.05 -0.12 -0.12 -0.14 -0.13 -0.10 Forecasting Regression using both CAPE and Modeled CAPE 𝑃𝑡 + 𝜀𝑡+𝑘 𝐸𝑡−3 𝑟𝑡+𝑘 = 𝛼 + 𝛽 ln Newey-West Non-overlapping Adjusted R2 t-Stat t-Stat -20.08 -7.20 -3.97 -2.45 -2.51 -1.94 -2.42 -4.74 -2.72 -2.10 -2.20 -1.94 47.2% 36.2% 22.4% 7.6% 3.6% 0.5% Horizon (Months) Coefficient 120 60 36 12 6 1 -0.07 -0.17 -0.18 -0.24 -0.24 -0.21 𝑃𝑡 − 𝑓 𝑖𝑡 , 𝜋𝑡 𝐸𝑡−3 + 𝜀𝑡+𝑘 Newey-West Non-overlapping Adjusted R2 t-Stat t-Stat -8.02 -8.42 -5.04 -3.92 -4.02 -3.06 -4.14 -6.81 -3.85 -3.00 -3.60 -3.06 42.7% 40.1% 27.4% 12.2% 6.6% 1.1% Source: Research Affiliates, based on data from Global Financial Database. 30 Current Implications » If current real rates were natural, not a consequence of QE » … and, if the resulting real bond yields were natural › The natural valuation levels would be lower than today, by a wide margin. Current Estimate (2015.09.30) Market Shiller PE Inflation Real YieldModel PE Delta US (Large) 24.1 0.20% 1.84% 16.7 -31% Australia 14.4 1.50% 1.11% 16.0 11% Canada 17.3 1.27% 0.16% 14.7 -15% France 14.8 0.05% 0.94% 15.8 7% Germany 16.1 0.19% 0.40% 15.1 -6% Hong Kong 15.5 2.40% -0.87% 13.2 -15% Italy 10.6 0.19% 1.54% 16.5 55% Japan 24.1 0.29% 0.06% 14.6 -39% Spain 11.1 -0.42% 2.31% 16.9 52% Sweden 18.9 -0.17% 0.89% 15.8 -17% Switzerland 21.8 -1.40% 1.26% 16.2 -26% United Kingdom 11.0 0.08% 1.68% 16.6 51% Source: Research Affiliates, LLC, based on data from Global Financial Data, Bloomberg, and central banks of global developed countries 31 Valuation Matters…Even for the Intermediate Term » Understanding CAPE in a more comprehensive way › A 3-D valuation mountain better describe fair value of CAPE » Finding the ―Sweet Spot‖ for Equity Market Valuations › Stock prices can be high under the right condition › Moderate inflation and real interest rate are ideal › Today’s conditions are not in that sweet spot, but QE is arguably a special kind of easy money, perhaps fueling asset bubbles . » Application of the 3-D mountain valuation model › Valuations matter –not just in the long term, but in the short term › Incorporating macroeconomics conditions significantly increases short-term return forecasting power 32 Appendix 33 A 3-D Valuation Mountain CAPE vs. Real Interest Rate and Inflation » What are the ideal macroeconomic conditions for a soaring CAPE? › Common belief: rock-bottom levels of inflation and real interest rates › 3-D valuation mountain: moderate levels of inflation and real interest rates Median P/E at Different Inflation and Real Yield Regimes (United States, 1880.12–2013.12) CAPE Real Rate Inflation Bel ow -1% Bel ow -1% -1% to 1% 1% to 3% 3% to 5% Above 5% Count 11 11 13 14 10 -1% to 1% 7 14 21 16 11 1% to 3% 17 17 19 20 9 3% to 5% 17 18 23 17 17 Above 5% 14 11 10 11 Real Rate Inflation -100% to -1% -1% to 1% 1% to 3% 3% to 5% 5% to 100% Bel ow -1% -1% to 1% 1% to 3% 3% to 5% Above 5% 13 10 34 16 118 4 31 95 89 69 49 106 257 143 80 54 83 118 35 26 78 14 22 53 0 - Source: Research Affiliates, based on data from Global Financial Database. 34 A 3-D Valuation Mountain Developed Countries (excluding U.S.) Median P/E at Different Inflation and Real Yield Regimes (Developed Countries, 1972.03–2013.12) CAPE Real Rate Inflation Bel ow -1% 12 18 16 17 10 Bel ow -1% -1% to 1% 1% to 3% 3% to 5% Above 5% Count -1% to 1% 18 14 14 1% to 3% 23 24 24 17 11 3% to 5% 7 25 22 17 9 Above 5% 16 17 19 15 11 Real Rate Inflation Bel ow -1% -1% to 1% 1% to 3% 3% to 5% Above 5% Bel ow -1% -1% to 1% 1% to 3% 3% to 5% Above 5% 0 0 14 61 136 2 125 365 272 183 29 545 1523 541 342 12 240 1061 369 327 73 120 425 386 380 - Source: Research Affiliates, based on data from Global Financial Database. 35 Important Information By accepting this document you agree to keep its contents confidential. You also agree not to disclose the contents of this document to third parties (including potential co-investors) without the prior permission of Research Affiliates, LLC (Research Affiliates, including its related entities). Research Affiliates, LLC claims compliance with the Global Investment Performance Standards (GIPS®). Current and qualified po tential investors may contact Research Affiliates, LLC at institutional@rallc.com to receive a list of composite descriptions, a GIPS compliant presentation, and general information regarding the firm’s policies for valuing portfolios, calculating performance, and preparing compliant presentations. The material contained in this document is for information purposes only. This material is not intended as an offer or solici tation for the purchase or sale of any security or financial instrument, nor is it advice or a recommendation to enter into any transaction. Any offer to sell or a solicitation of an off er to buy or sell shall be made solely to qualified investors through a private placement memorandum for pooled investment vehicles, or investment management agreement for separately mana ged accounts. This information is intended to supplement information contained in the respective disclosure documents. The information contained herein should not be const rued as financial or investment advice on any subject matter. Research Affiliates, LLC and its related entities do not warrant the accuracy of the information provided herein, eit her expressed or implied, for any particular purpose. The index data published herein are simulated, no allowance has been made for trading costs, management fees, or other costs, are not indicative of any specific investment, are unmanaged and cannot be invested in directly. Past simulated performance is no guarantee of future performance and actual inv estment results may differ. Any information and data pertaining to an index contained in this document relate only to the index itself and not to any asset management product bas ed on the index. With the exception of the data on Research Affiliates Fundamental Index, all other information and data are based on information and data from third party sour ces. Investors should be aware of the risks associated with data sources and quantitative processes used in our investment management process. Errors may exist in data acquired from third party vendors, the construction of model portfolios, and in coding related to the index and portfolio construction proc ess. While Research Affiliates takes steps to identify data and process errors so as to minimize the potential impact of such errors on index and portfolio performance, we cannot guarantee that such errors will not occur. The RAFI® US Equity Long/Short Index, RAFI Fundamental U.S. Style Index Series, and Enhanced RAFI Index Series is calculated by S&P Dow Jones Indices LLC or its affiliates. S&P® is registered trademark of Standard & Poor’s Financial Services LLC (―S&P‖) and Dow Jones® is a registered trademark of Dow Jones Trademark Holdings LLC (―Dow Jones‖) and have been licensed for use by S&P Dow Jones Indices LLC and its affiliates. Investment products based on the RAFI® US Equity Long/Short Index is not sponsored, endorsed, sold or promoted by S&P Dow Jones Indices LLC, Dow Jones, S&P or their respective affiliates and none of S&P Dow Jones Indice s LLC, Dow Jones, S&P or their respective affiliates makes any representation regarding the advisability of investing in such product(s). S&P Dow Jones Indices LLC, its affiliates, sources and distribution agents, and each of their respective officers, directors, employees, agents, representatives and licensors (collectively, the ―Index Calculation Agent‖) shall not be liable to Research Affiliates, LLC, any customer or any third part y for any loss or damage, direct, indirect or consequential, arising from (i) any inaccuracy or incompleteness in, or delays, interruptions, errors or omissions in the delivery of the RAFI® US Equity Long/Short Index or any data related thereto (the ―Index Data‖) or (ii) any decision made or action taken by Research Affiliates, LLC, any customer or third party in reli ance upon the Index Data. The Index Calculation Agent does not make any warranties, express or implied, to Research Affiliates, LLC, any of its customers or anyone else regarding the I ndex Data, including, without limitation, any warranties with respect to the timeliness, sequence, accuracy, completeness, currentness, merchantability, quality or fitness for a particular purpose or any warranties as to the results to be obtained by Research Affiliates, LLC, any of its customers or other person in connection with the use of the Index Data. The Index Calculation Agent shall not be liable to Research Affiliates, LLC, its customers or other third parties for loss of business revenues, lost profits or any indirect, consequent ial, special or similar damages whatsoever, whether in contract, tort or otherwise, even if advised of the possibility of such damages. Russell Investments is the source and owner of the Russell Index data contained or reflected in this material and copyrights related thereto. Russell Investments and Research Affiliates, LLC have entered into a strategic alliance with respect to the Russell Fundamental Indexes. Subject to Research Affiliates, LLC’s intellectual property rights in certain content, Russell Investments is the owner of all copyrights related to the Russell Fundamental Indexes. Russell Investments a nd Research Affiliates, LLC jointly own all trademark and service mark rights in and to the Russell Fundamental Indexes. Research Affiliates, LLC is the owner of the trademarks, servi ce marks, patents and copyrights related to the Fundamental Index and the Fundamental Index methodology. The presentation may contain confidential information and unauthorized use, disclosure, copying, dissemination, or redistribution is strictly prohibited. This is a presentation of Research Affiliates, LLC. Russell Investments is not responsible for the formatting or configuration of this material or for any inaccuracy in Research Affiliates’ presentation thereof. The FTSE Research Affiliates Fundamental Indexes are calculated by FTSE International Limited (―FTSE‖) in conjunction with Re search Affiliates LLC (―RA‖). All rights and interests in the FTSE Research Affiliates Fundamental Indexes vest in FTSE. All rights in and to the RA fundamental weighting methodology used in the calculation of the FTSE Research Affiliates Fundamental Indexes vest in RA. All rights in the FTSE indices and / or FTSE ratings (together the ―FTSE Data‖) ve st in FTSE and/or its licensors. Except to the extent disallowed by applicable law, neither FTSE nor RA nor their licensors shall be liable (including in negligence) for any loss arising out of use of the FTSE Research Affiliates Fundamental Indexes, the FTSE Data or underlying data by any person. ―FTSE™‖ is a trademark of the London Stock Exchange Plc and is used by FTSE under license. ―Research Affiliates‖ and ―Fundamental Index‖ are trademarks of RA. FTSE is not an investment adviser and makes no representation regar ding the advisability of investing in any security. Inclusion of a security in an index is not a recommendation to buy, sell or hold such security. No further distribution of FTSE Data is permitted without FTSE’s express written consent. 37 Important Information The RAFI® Bonds US Investment Grade Index and RAFI® Bonds US High Yield Index is calculated by ALM Research Solutions, LLC. i n conjunction with Research Affiliates, LLC. All rights and interests in the RAFI® Bonds US Investment Grade Index and the RAFI® Bonds US High Yield Index vest in Research Af filiates, LLC. All rights in and to the Research Affiliates, LLC Fundamental Index® concept used in the calculation of the RAFI® Bonds US Investment Grade Index and the RAFI® Bonds US High Yield Index vest in Research Affiliates, LLC. The RAFI® Bonds US Investment Grade Index and the RAFI® Bonds US High Yield Index are not sponsored, endorse d, sold or promoted by ALM Research Solutions, LLC., and ALM Research Solutions, LLC. makes no representation regarding the advisability of investing in such product(s). Ne ither ALM Research Solutions, LLC. nor Research Affiliates, LLC make any warranties, express or implied, to any of their customers nor anyone else regarding the accuracy or completeness of any data related to the RAFI® Bonds US Investment Grade Index or the RAFI® Bonds US High Yield Index. All information is provided for information purposes only. Nei ther ALM Research Solutions, LLC. nor Research Affiliates, LLC accept any liability for any errors or any loss arising from the use of information in this publication. Copyright MSCI. All Rights Reserved. Without prior written permission of MSCI, this information and any other MSCI intellectual property may only be used for your internal use, may not be reproduced or redisseminated in any form and may not be used to create any financial instruments or products or any indices. This information is provided on an ―as is‖ basis, and the user of this information assumes the entire risk of any use made of this information. Neither MSCI nor any third party involved in or related to the computing or compiling of the data makes any express or implied warranties, representations or guarantees concerning the MSCI index-related data, and in no event will MSCI or any third party have any liability for any direct, indirect, special, punitive, consequential or any other damages (including lost profits) relating to any use of t his information. The trademarks Fundamental Index™, RAFI™, Research Affiliates Equity™, RAE™, and the Research Affiliates™ trademark and corpo rate name and all related logos are the exclusive intellectual property of Research Affiliates, LLC and in some cases are registered trademarks in the U.S. and other countries. Various features of the Fundamental Index™ methodology, including an accounting data-based non-capitalization data processing system and method for creating and weighting an index of securities, are protected by various patents, and patent-pending intellectual property of Research Affiliates, LLC. (See all applicable US Patents, Patent Publications, Patent Pending intellectual property and protected trademarks located at http://www.researchaffiliates.com/Pages/legal.aspx#d, which are fully incorporated herein.) Any use of these trademarks, logos, patented or patent pending methodologies without the prior written permission of Research Affiliates, LLC, is expressly prohibited. Research Affiliates, LLC, reserves the right to take any and all necessary action to preserve all of its rights, title, and interest in and to these ma rks, patents or pending patents. Neither Research Affiliates, LLC nor its related entities provide investment advice outside of the United States or offer or sell any securities, commodities or derivative instruments or products. Any such business may only be conducted through registered or licensed entities and individuals permitted to do so wi thin the respective jurisdiction. © Research Affiliates, LLC. All rights reserved. Duplication or dissemination prohibited without prior written permission. 38