The importance of stress testing in assessing the credit risk of bank loan portfolios has grown over time. Currently, these exercises are accepted as the primary means of supporting capital planning, business strategy and portfolio management decision-making (Financial Services Authority 2008). Such analysis gives us insight into the likely magnitude of losses in an extreme but plausible economic environment, conditional on varied drivers of loss. It follows that such activity enables the computation of unexpected losses that can inform regulatory or economic capital according to Basel III guidance (Basel Committee on Banking Supervision 2011).

Most recently in this domain, in satisfaction of the Current Expected Credit Losses (CECL) accounting standards (Financial Accounting Standards Board 2016),¹¹ 1 See Jacobs (2015) for a conceptual framework addressing the measurement of model risk and Jacobs (2020) for the quantification of model risk in the context of a top-down credit model for CECL. or in compliance with the Federal Reserve’s Dodd–Frank Act Stress Test (DFAST) program (Board of Governors of the Federal Reserve System 2016),²² 2 See Jacobs (2019) for an assessment of the accuracy of alternative supervisory methodologies for DFAST stress testing. we observe that the predominant types of models used in the industry differ slightly from those used in the context of the 2007–9 global financial crisis, as their application must meet particular capital adequacy and accounting requirements that were not previously a consideration (Global Credit Data 2019).

In this study, we quantify the degree of model risk due to several forms of model misspecification or violations of model assumptions by utilizing the principle of relative entropy. This methodology studies the distance of an alternative model to a reference model according to some suitable loss metric (see Hansen and Sargent 2007; Glasserman and Xu 2014) and can capture the dimensions of model uncertainty error beyond parameter estimation error. This framework for measuring model risk is applied to a CECL stress testing exercise of default risk for a corporate portfolio. The importance of this application is rooted in the sensitivity of CECL or DFAST results from the perspective of prudential supervision as well as of accounting policy. It is observed that third-party reviewers, such as model validators or regulators, often question the impact of faulty model assumptions on capital and reserve projections. We find that omitted-variable bias due to leaving out a critical risk factor has the most impact, that incorrect specification of the model equation function is of minimal importance and neglected interaction terms among the explanatory variables are of intermediate influence for the measured model risk in terms of forecast bounds.

To elaborate on this finding, the standard manner in which credit models have been adapted for stress testing is through the modification of probability of default (PD) models at the disposal of financial market participants. PD models are designed to accurately measure an obligor’s ability and willingness to meet future debt obligations over some horizon, and are typically associated with a credit score or rating. The majority of PD risk rating methodologies or models currently used in the industry are characterized by a dichotomy of outcomes: point-in-time (PIT) versus through-the-cycle (TTC). In the so-called PIT rating philosophy such PD models should incorporate a complete set of borrower-specific and macroeconomic risk factors that will measure default risk at any point in the economic cycle. In contrast, according to the TTC rating philosophy, the model should abstract from the state of the economy or from cyclical effects and measure default risk over a more extended time horizon that incorporates a variety of macroeconomic states. This TTC orientation implies that ratings derived from the model should show “stability”, wherein material changes in ratings can be ascribed to fundamental, as opposed to transient, factors. PIT PD models are typically deployed in loan pricing and early warning systems, while TTC PD models feature prominently in regulatory capital, credit underwriting and portfolio management applications.

Note that this distinction prevails in particular for wholesale credit asset classes (eg, large corporate or middle market commercial and industrial (C&I) loans), where instead of using PIT PD models directly for stress testing, alternative approaches that are prevalent in the industry are used. One such common methodology involves adding sensitivity to macroeconomic variables in the TTC PD models (Global Credit Data 2019). Such TTC PD models are commonly found in the Basel III framework or for use in credit underwriting, as previously mentioned. The predominant manner in which TTC PD models are used in stress testing is through a rating transition model construct (Gross et al 2020), where the level of modeling is at the rating level and credit ratings are aggregated for different modeling segments across a bank’s portfolio.

These considerations point us to an overarching conceptual question regarding stress testing and how such models are validated. Demonstrating the fitness for purpose of downstream models that are dependent upon upstream credit risk models is a challenge within the industry. Often, model development teams will find it infeasible to test the downstream fitness of upstream models during redevelopment, and it could be the case that they become subject to overarching validation issues that are not resolved until the next redevelopment cycle. This could then involve lags of multiple years and be costly in terms of enhanced model monitoring requirements. In the case of wholesale portfolios, in which it is more common to have this disconnect, this leads to challenges in demonstrating the conceptual soundness of credit risk and stress testing models to model validators and supervisors (Global Credit Data 2019). The type of PD models that we investigate in this paper address this issue, as they are directly applicable to stress testing. To the best of our knowledge, there is no academic or practitioner literature addressing this issue that is particular to the wholesale credit asset class, which is where our research presents its main contribution to the literature.

The position of this research in the academic literature is at the intersection of two streams of inquiry. First, there are a number of empirical studies that focus on the factors that determine corporate default and its forecasting (see, for example, Altman 1968, Jarrow and Turnbull 1995 and Duffie and Singleton 1999). At the other end of the spectrum, there are mainly theoretical studies that focus on modeling frameworks for either understanding corporate default (see, for example, Merton 1974) or perspectives on the TTC versus PIT dichotomy (see, for example, Aguais et al 2008). In this paper, we blend these considerations of theory and empirics while also addressing the previously discussed conceptual validation issue facing practitioners in stress testing for wholesale portfolios.

This research is further distinguished from the existing literature by its utilization of an obligor-level and dynamic modeling framework that considers financial, credit rating and macroeconomic variables that are time varying.³³ 3 See Bellotti and Crook (2013) for an application of this methodology in a retail credit context. We estimate these models over a history that contains several economic cycles and apply them to a CECL stress testing exercise. We implement this exercise through the construction of discrete-time hazard rate models of default, a class of dynamic PD models, by utilizing a data set of corporate ratings and defaults. This methodology has the benefit of accommodating the discrete character of our data (which are quarterly snapshots), and while the use of discrete-time survival models appeared previously in the prediction of corporate defaults, prior studies have not incorporated macroeconomic risk factors or been applied to stress testing (Shumway 2001; Cheng et al 2010). This is, to the best of our knowledge, the first study to do so. Finally, we employ what is, compared with other models in this class, a less complex estimation algorithm. While the computational advantages of our data design approach may not be the primary benefit of our methodology, given the availability of inexpensive computational power, they do allow more resource-constrained model developers to perform rapid prototyping prior to investing in costlier techniques.

We present an innovation to the literature by introducing a structural Merton model style distance-to-default (DTD) measure into a stress testing application. Further, we construct discrete-time hazard rate models of PD, featuring a modeling data design that allows for a computationally efficient estimation technique, which is of particular value to modeling practitioners. Extending the literature along another dimension, the DTD risk factor, derived from equity prices and accounting measures of leverage, admits the design of challenger hybrid structural–reduced-form models, which are compared with the versions of these models containing all the other variables under consideration except for the DTD. It is demonstrated that the challenger models result in improved model performance (ie, measures of discriminatory power and predictive-level PD accuracy) and have a comparable quality of CECL scenario forecasts and that introduction of the structural DTD risk factor does not result in the other variables being rendered statistically insignificant.

The remainder of this study proceeds as follows. Section 2 constitutes a review of the relevant literature, such as studies on hazard rate modeling to predict binary outcomes, either the probability of occurrence or the time to the event, and we specify the discrete-time version within this class of models that we employ in the PD context. Section 3 is an outline of the methodology for our modeling exercise, where we discuss the general framework and its different subclasses, leading to the particular technique employed in this research. In Section 4 we present the empirical results of this study, including descriptive statistics of the modeling data set, estimation results, model performance metrics and the exercise in which we quantify model risk. Finally, in Section 5 we summarize our study and discuss future directions for this line of inquiry.

Rating or scorecard models have historically focused on estimating the PD, as opposed to the severity of losses, in the event of default or loss given default. Default is usually defined as a “failure”, such as bankruptcy, liquidation, failure to pay, deemed unlikely to pay, etc. This construct does not consider downgrades or upgrades in credit ratings, as considered in mark-to-market (MTM) models of credit risk. These default mode (DM) credit risk models project credit losses only due to events of default, unlike MTM models that consider as credit events all credit quality changes. Among such DM models we can identify three broad categories: expert-based systems (eg, artificial neural networks), risk rating methodologies (eg, agency credit ratings from S&P Global Ratings or Moody’s) and credit scoring models (eg, scorecards developed by banks or FICO scores).

The PD scoring model is most prevalent among the credit measurement methodologies used historically. One of the first models in this class was a multiple discriminant analysis (MDA), as illustrated in the classic paper by Altman (1968). These types of models have the advantages of being cost-efficient to deploy and not subject to subjectivity or inconsistency, as observed in expert systems. Altman and Narayanan (1997) documented how these models became prevalent across the industry and academia and concluded that the similarities across applications are more pronounced than the differences. Another class of credit scoring models that is now widespread is the logistic regression model (LRM) (Hosmer et al 2013), a prime example of which is the RiskCalc model from the vendor Moody’s Analytics (Dwyer et al 2004). This is used for commercial credit risk and considered an industry standard.

More advanced studies on the development, application and evaluation of predictive decision support models in the credit industry have been conducted beyond the aforementioned seminal academic and vendor approaches. Thomas (2010) highlights that corporate risk models employ data from balance sheets, financial ratios or macroeconomic indicators, whereas retail models use data from application forms, customer demographics and customer transaction history. He attributes these differences to specific modeling challenges that arise in consumer, as opposed to corporate, credit scoring and that lead many studies to focus on either the corporate or the retail business. García et al (2010) highlight how in PD scorecard development statistical hypothesis testing is often neglected or employed inappropriately, as the assumptions of parametric tests are violated in classifier comparisons, focusing on pairwise comparisons without $p$-value adjustments, which increase the actual probability of type I errors. Hofer (2015) addresses the research question of how to update PD scorecards in the face of new information, which is increasingly relevant to the industry as modeling data becomes more granular, especially in the retail sector but also for wholesale portfolios as there has been a movement from annual to quarterly observations. Hao et al (2011) propose a novel algorithm with several classifiers, paired algorithms and ensemble strategies in a factorial design, which focuses on preselected methods and omits a systematic comparison of several state-of-the-art classifiers, while Abellán and Mantas (2014) feature fewer classifiers and propose a novel algorithm that is compared with various reference methods. Finally, the benchmarking study by Lessmann et al (2015) compares several novel classification algorithms in state-of-the-art PD credit scoring models, in which they examine the extent to which the assessment of alternative scorecards differs across established and novel indicators of predictive accuracy.

In his seminal paper on the structural approach to credit risk, Merton (1974) models equity in a firm with leverage as a call option on assets where the strike price coincides with the face value of the debt. The PD is derived by solving for the option value numerically with the unobserved asset value and its volatility given the quantity of debt and a valuation horizon. The product of this process is a firm’s DTD, which represents the number of standard deviations separating asset and debt repayment values and is inversely related to the PD. The CreditEdge public firm model developed by Moody’s Analytics is a well-established implementation of this framework that also uses historical default rates to empirically calibrate the output and produces the expected default frequency (EDF). Since the EDFs are ultimately based upon equity prices, there is consequently a heightened sensitivity to the changing financial state of the obligor, in contrast to agency credit ratings, which are more reliant on static data available at the time of underwriting or periodic reassessment of the borrower.

Another class of commonly used credit risk models arises from the previously discussed structural Merton approach and an alternative reduced-form framework originated by Jarrow and Turnbull (1995) and Duffie and Singleton (1999) that uses intensity-based models to estimate stochastic hazard rates. This school of thought differs in the methodology employed in estimating PDs. While the structural Merton approach considers an economic process that produces defaults, the reduced-form approach extracts a random intensity process that generates defaults from the prices of defaultable debt. A prominent example of a model in this class is the proprietary Kamakura Risk Manager, which incorporates an econometric methodology based on Chava and Jarrow (2004). This so-called Jarrow–Chava model (JCM) is sometimes called a hybrid approach, in that it combines the direct modeling of default, as in the LRM, with the use of either equity or debt market data, and in the case of traded debt instruments this construct has the potential to control for the distorting effect of illiquidity on the measurement of default risk. Note that a critique of the JCM is that the presence of anomalies, such as embedded options in the debt markets, can adversely impact the accuracy of these models. In this study, we circumvent this limitation by combining the use of fundamental factors, as in the LRM, with equity market information, as in the structural Merton model, in our version of a hybrid hazard rate model, as will be detailed in Section 3.

Beyond these seminal studies and vendor applications, there are several more recent and state-of-the-art studies that apply survival analysis in credit risk modeling. Baesens et al (2005) discuss and contrast statistical and neural network approaches for survival analysis. Several neural network survival analysis models are discussed and evaluated according to how they deal with censored observations, time-varying inputs and the monotonicity and scalability of the generated survival curves. Baesens et al compare the performance of a neural network survival analysis model with that of the proportional hazards model for predicting both loan default and early repayment, using data from a UK financial institution. Dirick et al (2015) study a special type of survival model called the mixture cure model, which facilitates the prediction of multiple events of interest, such as default and early repayment.

Stress testing based upon hypothetical scenarios, usually a blend of macroeconomic projections and the application of judgmental elements, has become a prevalent tool in the supervision of financial institutions (Financial Services Authority 2008). The qualitative aspect of the stress testing process is considered by some to be deficient, as illustrated by the critique of the exercise conducted by US supervisors during the 2007–9 global financial crisis, where the projection of unemployment in the 2009 adverse scenario fell short of the realization of this factor in under a year and was therefore deemed to be insufficiently severe (Board of Governors of the Federal Reserve System 2009). In his analysis of this incident, Baker (2009) claims that the supervisors may have underpredicted loan losses on the order of USD120 billion, placing him among those who conclude that this surprise on the part of the regulators is evidence that these tests are failures. In the retail credit risk context, Haldane (2009) attributes this weakness to either the omission of, or the underprediction of the impact of, risk factors in the dynamic macroeconomic model that was used (the so-called disaster myopia). He also points to the phenomenon termed “misaligned incentives”, which means that institutions had no intention of designing realistic stress tests. Jacobs (2019) shows that, in addition to these downward biases, the prevalent econometric methodologies used by many supervisors and banks may be subject to heightened inaccuracies, which are attributed to a misspecified dependence structure between risk factors. He also highlights the challenges resulting from the paucity of default data in applying high-dimensional approaches to the stress testing of credit risk, as in the common approach investigated and improved upon, which will necessarily feature misspecified correlations that will give rise to inaccuracies.

There is a well-established literature on the applications of the survival approach to default prediction, as applied to stress testing applications, that follows the period of the pre-2010 downturn, an essentially profound paradigm shift to a stream of research presenting a variety of novel approaches, attributable to the increased importance of stress testing in the Basel III regulation and in daily practice by regulators. Bellotti and Crook (2013) estimate discrete-time survival models of borrower default for credit cards that include behavioral and macroeconomic factors across the life of the loan. They find that the dynamic models that include these variables provide statistically significant improvements in model fit and improved forecasts of default at both account and portfolio levels when applied to an out-of-sample data set. They also simulate extreme economic conditions and show how these models can be used to stress test credit card portfolios. Bellotti and Crook (2014) offer improved methodologies for scenario generation to predict default rates for retail portfolios implemented using discrete survival analysis, enabling macroeconomic conditions to be included as time-varying covariates. They depart from traditional models by employing Monte Carlo simulation to generate a distribution of estimated default rates from which value-at-risk (VaR) and expected shortfall are computed as a means of stress testing. Breeden and Crook (2022) propose the use of survival models suitable for CECL stress testing that include lagged delinquency as a covariate and using a large sample of 30-year mortgages. They show that the proposed method is more accurate than any of the other methodologies they considered (roll rate, state transition and vintage models) for both short-term and long-term predictions of the default.

Finally, we review some of the foundational studies in the quantification of model risk according to the principle of relative entropy. Hansen and Sargent (2007) propose an alternative paradigm to the standard theory of decision-making under uncertainty based on a statistical model that informs an optimal distribution of outcomes. They adapt robust control techniques through developing a theory of model risk measurement that acknowledges misspecification in economic modeling and they apply this framework to a variety of problems in dynamic macroeconomics. Glasserman and Xu (2014) apply this framework to a financial risk measurement that relies on models of prices and other market variables that inevitably rely on imperfect assumptions that give rise to model risk. They develop a framework for quantifying the impact of model error through measuring and minimizing risk in a way that is robust to model error. Their robust approach starts from a baseline model and finds the worst-case error in risk measurement that would be incurred through a deviation from a baseline model given a precise constraint on the plausibility of the deviation. Using relative entropy to constrain model distance leads to an explicit characterization of worst-case model errors that lends itself to Monte Carlo simulation, allowing the straightforward calculation of bounds on the model error with very little computational effort beyond that required to evaluate performance under the baseline nominal model. They apply this technique to a variety of applications in finance, such as problems of portfolio risk measurement, credit risk, delta hedging and counterparty risk measured through credit valuation adjustment. Skoglund (2019) applies the principle of relative entropy to quantify the model risk inherent in loss-projection models used in macroeconomic stress testing and impairment estimation in an application to a retail portfolio and a delinquency transition model. He argues that this technique can complement traditional model risk quantification techniques, where a specific direction or range of model misspecification reasons, such as model sensitivity analysis, model parameter uncertainty analysis, competing models and conservative model assumptions, is usually considered.

In this section, we outline the development of the econometric framework that we employ. We proceed by first describing the most general model and defining a set of common terms. We then describe various special cases, starting with the hazard rate survival model in continuous time, followed by the discrete-time version of this model. Finally, we describe the implementation of the landmark (or snapshot)⁴⁴ 4 In the snapshot data sample design, for variables that have a different frequency such as quarterly macroeconomic variables but annual or semi-annual financial variables, for each instance of the former we create a time series that evolves while the latter is frozen (Houwelingen and Putter 2008). data sample design of the latter version of the hazard rate model.

Let us denote by $t$ the calendar time, which we decompose as $t=a_i+{\tau }_i$. The time of origination is $a_i$ (ie, the date on which the snapshot is measured) and the time of duration is ${\tau }_i$ (ie, the time from the measurement of the snapshot to the end of the forecast horizon). Units of observation or obligors are subscripted by $i$ for $i=1,\mathrm{\dots },N$ obligors. We may record time at various granularities, in the case of C&I borrowers either quarterly or annually, and while spacing may be irregular due to the exact timing of when financial statements are spread, in general these will be in multiples of one quarter, so that in reality we are dealing with discrete sampling. Variables subscripted by $i$ differ among borrowers but not temporally (like a segmentation characteristic of a borrower, such as a PD scorecard or an industry group). Variables subscripted by $t$ differ through calendar time, but are common among obligors (as with a macroeconomic variable), and a term subscripted by $it$ may be permitted to vary both temporally and cross-sectionally (as with a financial ratio). The corresponding risk factors or covariates are given by ${𝒘}_i$, ${𝒛}_t$ and ${𝒙}_{it}$, while the respective parameter vectors are given by ${𝜷}_1$, ${𝜷}_2$ and ${𝜷}_3$. The terms ${𝜸}_{12}$, ${𝜸}_{13}$ and ${𝜸}_{23}$ denote matrixes of interaction term parameters between these three sets of parameters to be estimated.

The following describes a rather general and stylized econometric model of PD with respect to obligor $i$ in a discrete-time period $t$. We will later impose restrictions upon this framework to arrive at a representation of the credit risk models used in practice. Let us denote by $d_{it}^{*}$ a continuous latent variable representing the utility⁵⁵ 5 Since regression-based approaches to PD modeling are inherently empirical (in contrast to structural Merton model approaches), there is, strictly speaking, no theoretical model to describe. However, we can link the logistic regression approach to the economic theory of consumer or producer behavior, where the state of default can be viewed as a decision made by either a borrower or lender who is optimizing a utility function (Small and Rosen 1981). gained by the default of borrower $i$ in period $t$. We define the event of default as $d_{it}=1$ if $d_{it}^{*}>0$, and the event of non-default as $d_{it}=0$ if $d_{it}^{*}\le 0$. Suppose that this latent variable is a function linear in the risk factors and their interaction terms plus a residual term ${\eta }_{it}$ and borrower-specific intercept term ${\beta }_{0i}$:

and then define the conditional PD as

which implies that

where the distribution function of ${\eta }_{it}$ is given by $F(\cdot )$. The variables subscripted by time may include lags of variable lengths.

Through the imposition of restrictions or assumptions governing various modeling aspects it may be demonstrated how this construct subsumes many PD models currently employed by practitioners. A canonical case is obtained by restricting all of the interaction terms to zero, which gives rise to the typical PIT PD model used in early warning or credit portfolio management:

Usually, in this setting, $t$ is a horizon spanning from one month to one year, and the function $F(\cdot )$ is typically the logistic link function of the LRM, $1/(1+\exp (-x))$ (Hosmer et al 2013). It is common among practitioners to employ a linear transformation to the left-hand side term in (3.4), deriving a quantity interpreted as a “score”, where an alternative means of deriving this is through scaling the logit estimate.

An important observation is that the model in (3.3) and (3.4) may not be suitable for an unbalanced panel data set,⁶⁶ 6 Panel data can also be characterized as unbalanced panel data or balanced panel data. Balanced panel data sets have the same number of observations for all groups. Unbalanced panel data sets have missing values at some time observations for some of the groups. The main concern with unbalanced panel data is the question of why the data are unbalanced. If observations are missing at random, then this is not a problem. As an example, if the attrition of firms in your data over time is not random, ie, it is related to the idiosyncratic errors, then this sample selection may bias the estimates. which is exactly the format of data typical of the C&I asset classes. What this data design means is that there will be defaulted obligors over the prediction horizon, where subsequent performance is unobservable or otherwise recorded at alternative noncontiguous calendar times in the modeling data set. The consequence of this is potential bias in parameter estimates. Among the various means of accommodating this phenomenon is the survival analysis estimation technique (Kalbfleisch and Prentice 2002; Cox and Oakes 1984). This framework is prevalent in retail credit risk modeling, where the approximation of discrete time as continuous is more likely to be realistic, as opposed to the wholesale setting, where this assumption is more likely to be tenuous.

We first review the case of the continuous-time survival model, where the target quantity is the instantaneous probability of transitioning from one state (eg, an obligor with performing loans) to another (eg, default). Denoting by $T_i$ the duration time to obligor default implies that the conditional PD in the next instant, given that the obligor is currently not in default, may be represented by the hazard function having the duration time $\tau$:

It follows that the survival probability (ie, the probability of not being in default over some time interval) may be expressed as a function that can be written as the following integral representation of the hazard function:

A popular approach in both the academic literature and consumer credit practice is the use of the Cox proportional hazard model (CPHM) (Cox 1972) to estimate the hazard function as well as the related survival probabilities. The CPHM model further admits inclusion of dynamic covariates and can be expressed as

where the risk factors ${𝒙}_i(\tau )$ are obligor specific and dynamic across the time of duration (eg, financial variables), $𝒛(a_i+{\tau }_i)$ are risk factors varying over absolute time but constant over the cross section (eg, macroeconomic factors), and ${\lambda }_0(\tau )$ is a baseline hazard function of time which models the evolution of default risk independently of the other risk factors (ie, the idea being that this is a time-dependent residual of sorts). In this construct, forecasts of obligor financial or macroeconomic conditions after the beginning of a forecast period propagate through all subsequent time periods and influence the hazard function and survival probabilities over the entire forecast horizon.

There are various ways in which standard LRMs are inferior to survival models. First, survival models admit PD prediction over arbitrary forecast horizons apart from the window of the default flag used in developing the LRM, such as a one-year horizon of most PIT PD models, and thus are inherently suitable for applications such as CECL or DFAST loss forecasting. In addition, unlike in LRMs the PD predictions in survival models are conditional on not previously having been in default, and the data would have to be modified (as discussed below) in order to accommodate conditional defaults. Finally, as survival probabilities are available across the entire projection horizon there is the potential for an application to profitability forecasting, or for use as a challenger to an economic capital construct.

In (3.3) we represent a discrete-time panel model of binary choice, which is an LRM built upon a panel data design and is equivalent to a discrete-time survival model. We previously alluded to the fact that it is most common for financial institutions to use panel data sample designs that could be used to model dynamic risk factors. Essentially, this involves estimating LRMs for a set of fixed horizons but with a different model for each forecast window (for example for quarters 1 through $3\times 4=12$ for CECL/DFAST applications). In fact, this is the approach taken by the vendor Kamakura in its corporate PD model, with the obvious disadvantage that it is extremely resource intensive and requires a specialized software infrastructure. Another issue with this approach is the degradation of performance as the default horizon lengthens. This may be undesirable in an application such as CECL where there is a premium on accuracy, in the sense of predicting the level of default rates, as well as in distinguishing between defaults and non-defaults.

An alternative to estimating an LRM for each forecast horizon (as just described), which gives identical results in the academic literature and, in some cases, in retail credit risk modeling, and can be implemented in some standard software packages (eg, Python scikit-survival), is as follows. As we have a discrete-time panel data sample design, then, given a modeling data set design matrix of appropriate form, a discrete survival model may be estimated with a hazard function of the following form:

where we denote by $h_i^{\mathrm{d}}$ the discrete hazard rate for the $i$th obligor and by $S(\cdot )$ the associated survival probability. Cox (1972) has proposed the following specification of this relationship:

where the logit function (or log-odds ratio function) is the inverse of the logistic link function $F^{-1}(x)=\log (x/(1-x))$ and the discrete baseline hazard function is given by $h_i^{\mathrm{d}}({\tau }_i)$.

Jenkins (1995) proposes estimating the model (3.9) by specifying indicator variables corresponding to each time interval. Alternatively, Singer and Willett (1993) suggest using functions of the duration time index to capture this effect as a legitimate approach. In either case, we define a default indicator that is zero in all intervals where default is not observed and unity during the period where a default occurs. Subsequent to default the obligor does not appear in the data set so long as the default is not cured, in which case the obligor reappears in the data set as a performing entity and the indicator is reset to zero.

The specifications discussed in this section prior to the models in (3.8) and (3.9) have all assumed that time is continuous, but as we have pointed out this is not the case in reality, and, moreover, in the C&I asset class with quarterly observations this is likely to not be a realistic setup. Stepanova and Thomas (2002) find in spite of this argument that estimation results assuming continuous or discrete time show little difference. This may not be surprising, as Kalbfleisch and Prentice (2002) show that as the observation intervals tend to zero the discrete- and continuous-time models do indeed converge. Nevertheless, we believe the observation that the continuous-time model is an adequate approximation could be highly dependent on the particular data set, and in our case we do not find the differences to be immaterial in terms of either the coefficient estimates or the measures of model performance, which leads us to favor employing a discrete-time model.

Finally, we come to the approach used to estimate the hazard models in this research, which is based upon a paper by Houwelingen and Putter (2008) in the biostatistics literature. They model survival probabilities at a five-year horizon for acute lymphocytic leukemia patients after transplantation of bone marrow. This research proposes a landmark methodology and compares it with an established multistate modeling methodology in biostatistics. Houwelingen and Putter show that this technique greatly simplifies the modeling methodology, as it reduces to LRM estimation on the so-called snapshotted data set and leads to easy-to-interpret prediction rules.

At each landmark (snapshot in our terminology) point a simple Cox constant baseline hazard model is fitted on the interval, which is mathematically equivalent to estimating an LRM model on the restructured data set. This is in line with the methodologies used in the industry for PD scorecard development and does not assume continuous time, is less computationally intensive than the panel logistic approach of Kamakura and allows for simple implementation within standard software such as Python. Our approach can be expressed mathematically as a modification of the LRM equation (3.4):

where the snapshotted data set is given by

$t_i^s$ is the time since snapshot, $f(\cdot )$ is an appropriate transformation of the latter, $t_i^s{𝒙}_{it}^S$ is a set of time-interacting terms on the obligor-specific variables that vary over time (which can include the PD rating as well as financial ratios), and $g(\cdot )$ is some appropriate transformation of the latter. The time interactions capture the decay effects over a forecasting horizon with respect to obligor specific risk factors. The term $f(t_i^s)$ is analogous to the baseline hazard function in the Cox proportional hazard specification of (3.10).⁷⁷ 7 However, a downside of this approach is that the model fit obtained that is available in standard software cannot be used to test the statistical significance of the parameter estimates. The correct standard errors can be obtained by taking into account the clustering of the data (ie, each snapshot is effectively a separate case, several such cases exist for each obligor and there is correlation over time in the former and cross-sectionally in the latter) using the so-called sandwich estimators of Lin and Wei (1989). This approach is incorporated in software packages such as SAS (the GENMOD or SURVEYSELECT procedures) or Python (the generalized estimation equations in the logistic regression function of the statsmodels library). However, such approaches are very computationally intensive, and exhibit stability issues in the case where we have highly unbalanced panels, where the latter means nothing more than defaults being relatively rare and concentrated over time. However, in this research we obtain standard errors through bootstrapping, where we impose the proper stratification to preserve the correlation structure of the data, which is straightforward to implement in Python due to vectorized operations.

In this study we measured the model risk attributable to various model assumptions in dynamic econometric models of large corporate default according to the principle of relative entropy. This methodology studies the distance of an alternative model to a reference model according to some suitable loss metric (see Hansen and Sargent 2007; Glasserman and Xu 2014) and can capture dimensions of model uncertainty error beyond parameter estimation error. This framework for measuring model risk was applied to a CECL stress testing exercise of PD for a corporate portfolio. It was observed that omitted-variable bias (with respect to the Merton DTD risk factor) has the greatest impact upon measured model risk in terms of bounds on PD forecasts, the incorrect specification of the link function has the least impact and the neglect of interaction effects between risk factors has an intermediate impact. The importance of this application is rooted in the sensitivity of the CECL or DFAST results from the perspective of prudential supervision as well as accounting policy, as model validators or regulators often question the impact of faulty model assumptions on capital and reserve projections. We addressed this issue through this research.

This quantification of model risk was accomplished through the consideration of an obligor-level hazard rate methodology for corporate PD modeling that features macroeconomic, financial, equity market, duration and credit rating variables. This methodology was applied to stress testing for CECL, where we have departed from the common practice for wholesale portfolios of adapting rating transition models where the ratings are stressed for this purpose, and to our knowledge this is one of the first studies in the literature to have done so. We have further innovated by developing explicitly discrete-time hazard models with a specialized data sample design (the landmark methodology) that is particularly tractable in computation. This allows for the inclusion of a large number of variables and the utilization of a long historical data set.

The models that we developed are particularly suitable to this experiment in measuring model risk, as they feature rich risk modeling data and variable structures that allow for the investigation of varied modeling assumptions. Our base data were a lengthy borrower-level history of corporate ratings and defaults sourced from Moody’s in the period 1990–2015. The data were enhanced by attaching an extensive set of financial, macroeconomic and equity market variables to form the basis of candidate explanatory variables. The obligor-level hazard rate models developed had a one-quarter default horizon and, further, featured time decay and duration effects. Based upon the relevant literature, we also considered an alternative structural risk factor, which is: the structural modeling DTD measure constructed from the market value of equity and accounting leverage measures. We then compared these hybrid structural–reduced-form models with the financial ratio and macroeconomic variable only models. It has been shown that adding the DTD measures to our leading models did not invalidate the other variables chosen but significantly augmented model performance and resulted in comparable scenario forecasts.

We have addressed an overarching conceptual validation question regarding stress testing in demonstrating the model’s fitness-for-purpose for use as inputs to downstream models. This is a particular concern in the case of wholesale portfolios, since in this setting it is the more predominant practice to have this disconnect, which leads to challenges in demonstrating the conceptual soundness of both credit risk and stress testing models to model validators and supervisors (Global Credit Data 2019). The type of PD models we investigated in this paper addressed this issue as they are directly applicable to stress testing. There is no known academic or practitioner literature addressing this issue that is particular to the wholesale credit asset class, which is where our research presents its main contribution.

Our conclusion is that validation methods chosen in the stress testing context should be capable of testing model assumptions, given the sensitive regulatory uses of these models and concerns raised in the industry about the effect of model misspecification on capital and reserves. Our research adds to the literature by offering state-of-the-art techniques as viable options in the arsenal of model validators, developers and supervisors seeking to manage model risk.

Given the wide relevance and scope of the topics addressed in this study, there is no shortage of fruitful avenues along which we could extend this research. Some proposals include, but are not limited to:

• •

  alternative econometric techniques, such as various classes of machine learning models, including nonparametric alternatives;

• •

  asset classes beyond the large corporate segments, such as small business, real estate or even retail;

• •

  the consideration of industry specificity in model specification; and

• •

  data sets in jurisdictions apart from the United States, else pooled data encompassing different countries with a consideration of geographical effects.

The author reports no conflicts of interest. The author alone is responsible for the content and writing of the paper.