Adapting to data limited uncertainty - an alternative approach to Mortgage and Consumer Lending credit loss and performance forecasting
There are times when historical data is not sufficiently predictive. Certainly, the pandemic is an example of a sudden change causing historic credit data to lack predictability. Also, future trends like Environmental, Social, and Governance ("ESG") are expected to have significant impact on lending volumes, credit risk, and profitability. Yet no one knows exactly how and when ESG will impact Mortgage and Consumer Lending businesses. Changes like these are reverberating through our economy with many unknown effects. Change is only accelerating. This uncertainty tests those responsible for forecasting the future. The uncertainty environment challenges those responsible for credit loss estimation and providing accurate inputs to the Allowance for Credit Loss (ACL), loan pricing, loan decisions, and a wide variety of financial services analytical needs.
This is Part 3 of a 3 part series:
(Please click the boxes for the other articles.)
We use a simulation framework based on the time-tested "Five Cs Of Credit.” In the past, banks relied on subjective judgment to assess the credit risk of a borrower. Today, much of that judgment has been codified in statistical and related decision models. Essentially, underwriters, credit analysts, and the decision models rely on the following borrower characteristic information in deciding whether to make a given loan. These characteristics are commonly referred to as the Five Cs of Credit (10), a description of which is found in Table 1.
Table 1: The 5 Cs of Credit
Characteristic type is important as it describes the degree to which a characteristic is in direct control by the borrower.
Endogenous characteristics are unique to each borrower. The borrower has the direct ability to impact his or her endogenous characteristics.
Exogenous characteristics are generally outside of the borrower’s direct control, though it may impact the borrower’s probability of default (“PD”) or the loss given default (“LGD”). Exogenous characteristics generally refer to the macroeconomy.
To be fair, these characteristic types are not always strictly defined. For example, the collateral value will certainly have some exogenous impact from the macro environment. The point is, it was the borrower that ultimately made the endogenous choice of collateral and has control over its disposition.
Developing judgmental credit loss and related simulations are, by their very nature, a team sport. As such, the methodology must be collaborative, promote teamwork, and build consensus. It uses seven steps, as shown in the next graphic. Steps 1 and 2 are foundational. Once they are completed, the governance system and environmental data should be reassessed for continuous improvement on a periodic basis and in alignment with the operational steps. These foundational steps set the stage for the operational cycle of steps 3 through 7, which continue in an evolutionary fashion.
Next, we provide the Monte Carlo scenario simulation segment and product build process in more detail:
Step 1: Establish the Governance Process
The governance process will be established. This may take the form of a Credit Loss Review Board (CLRB). The CLRB responsibility includes selecting products and associated resources to participate. The CLRB will approve the credentials of all those asked to participate. The Five Cs framework will be socialized and confirmed as the simulation framework for scenario planning. For each product, there are 4 different scenario segments based on 1) the individual borrower characteristics as captured in the endogenous factors, and 2) the likely macroeconomic changes captured in the exogenous market environment. (See the appendix for a scenario planning segmentation example.) The framework and process will be communicated to participants.
Step 2: Collect key environmental data
Data for each scenario segment and product will be collected using available information based on the organization's borrower database information and market information from public and company licensed private sources. While participant independence is required for scenario modeling, base information should be shared across participants. As a valuable participant, those members providing the data should also participate in the scenario planning exercise. Data that does not exist but is likely correlated to credit loss in the context of the uncertainty condition should be noted. This is a critical step as this is the information field from which all participants will create.
Step 3: Select key predictive credit drivers
The first step in the build stage is for each participant or team to familiarize them selves with the predictive credit drivers. At this point, participants will:
Independently select a fixed number of credit drivers they believe to be correlated to credit risk and in the context of the uncertainty condition.
Participate in a challenge session to consider additional data and to advocate for the participants data selection.
Independently update the driver selection based on veracity of the collective input.
At a minimum, one participant or team per segment and product is necessary. To increase the accuracy of the simulation, it is recommended to have multiple participants per segment.
Step 4: Weigh power and interactions of credit drivers
Based on step 3, the participants will rate the relative importance and interactions of the credit loss drivers. This step is crucial to building an effective scoring model. The weighting factors must be accurate and enjoy the consensus of the group. The Analytic Hierarchy Process (AHP) is the gold standard for performing this critical step. (11) This step collects the judgment of the credit risk experts in an objective manner.
Step 5: Analyze the results
The facilitator-led credit loss scoring sessions generate a significant amount of data to analyze, requiring the generation of several charts and tables to support the analysis and facilitate the CLRB’s review and decision. The analysis covers strategic business value and profitability, credit loss and other performance driver related costs, and strategic risks. The analysis approach enables the separation and clarification of objectives, facts, and the judgments of the team members.
Step 6: Optimize forecast based on scenario outcomes
The performance outcome simulates future losses and key performance measures 1 to 5 years forward. Additional estimates may be rendered via the simulation process, such as volume, attrition, and profitability. Note: the endogenous scenarios found in the appendix (1&2 or 3&4) are additive to provide a total loss estimate for the associated Exogenous Market Environment. Upon completion of the scenario segments, outcomes will be compared and utilized to drive the Allowance for Credit Loss or related performance estimating needs.
Step 7: Communicate and adapt business to scenario outcomes
Prior to making business decisions, a sensitivity analysis is conducted to determine the degree to which the recommendations are sensitive to slight variations in the weighting of the credit loss drivers. The sensitivity analysis will increase the CLRB’s confidence in their decisions. The simulation is intended to be an ongoing process. Regularly, (at least annually) the simulation will be re-run. Ideally, some similar participants will be included, as well as, "new blood" to bring fresh perspective. The step 2 environmental data will be updated and previously indicated "does not exist" data will be researched and provided as available.
The key benefits
It is important these sims are managed in a professional environment. This simulation decision-making environment consistently delivers the following benefits:
Facilitating a professional sim process.
Maintaining participant independence.
Improved stakeholder engagement, including greater consensus and buy-in.
Housing and presenting consistent background information to initiate the participant's environmental understanding.
Providing a straight forward, cloud-based workflow to capture the participant's credit weighted scoring and other scenario planning factors.
An optimization engine to evaluate simulated credit scenario outcomes.
Reporting engine to summarize and assist in the communicating of results.
An ongoing, auditable, record to help maintain process governance and decision transparency.
Enabling efficient scenario updating as new information is learned.
Higher velocity decisions and better return on investment
Conclusion
ESG, the pandemic, and other events create significant credit uncertainty. The chaotic nature of a new uncertainty world creates significant differences in terms of how people respond and the changes related to their credit behavior. The article describes why the new phase largely invalidates credit loss point estimates of the recent past and suggests an alternative credit loss estimation approach using Monte Carlo simulation modeling techniques.
Using a judgmental simulation model is appropriate given the lack of data available to drive desired decision support and as grounded in traditional credit decisioning. Ultimately, the new uncertainty world will give way to a more data-rich risk world. As this occurs, data will stabilize and grow, statistical confidence will increase, inertia will decline, and we will return to the comfortable world of risk modeling.
Until then, the Sim is our Friend.
The Decision Support Solution Methodology was created by Definitive Business Solutions, Inc. This methodology is enabled by Definitive Pro®, a cloud-based, group decision support service that provides a collaborative process to build consensus and make complex, multi-criteria decisions in a wide range of scenarios. It uses the leading theory in multi-criterion decision making (the Analytic Hierarchy Process), which provides the ability to synthesize quantitative and qualitative factors and set priorities. It also employs a state-of-the-art mathematical programming solver to find the most favorable solution and optimize the allocation of resources. (12)
For more information, please contact Definitive Business Solutions, Inc.:
John Sammarco, President | JSammarco@definitiveinc.com
Jeff Hulett, Executive Vice President | JHulett@definitiveinc.com
Appendix
This scenario planning example is for the Pandemic. It may be generalized for other uncertainty-based scenarios, like ESG.
Notes:
(1) Roberta Wohlstetter, Pearl Harbor: Warning and Decision, June, 1962
(2) The 9/11 Commission Report at 9-11commission.gov, July 2004
(3) Wall Street and the Financial Crisis: Anatomy of a Financial Collapse, United States Senate Permanent Subcommittee on Investigations, April 2011
(4) Stephen Kinzer, The coronavirus pandemic is a failure of imagination, The Boston Globe, March 2020
(5) Bill Gates, Who Has Warned About Pandemics For Years, On The U.S. Response So Far, NPR, Heidi Glenn, April 2020.
(6) In 2005, Dr. Rajan said the financial system was at risk “of a catastrophic meltdown.”
(7) The subtle difference between risk and uncertainty is manifest when considering OTAs. Generally, making an OTA relates to potential qualitative events that may impact the risk-based expected loss, specific to its standard deviation. Uncertainty, on the other hand, is focused on tail risk, or kurtosis. Thicker tails may cause wildly different loss results, as per the Financial Crisis. Please see our article The numbers don't lie - anticipating risk, managing uncertainty, and making quality decisions for more information.
(8) In general, using independent participants grounded approach to creating simulations is known as the "common task method." Independent participants, utilizing common data sources compete to develop accurate outcomes. While this is not as common in credit loss estimation, it is common in computer science in the development of machine learning algorithms for standard tests or photograph interpretations. Other examples include (a) In 1980, Robert Axelrod, professor of political science at the University of Michigan, held a tournament of various strategies for the prisoner's dilemma. He invited a number of well-known game theorists to submit strategies to be run by computers. (spoiler alert: a very simple "Tit For Tat" strategy won). b) In 2020, after a long-term, longitudinal study about "Fragile Families" was completed, a competition of over 160 teams was held to find the most predictive algorithms to answer key questions concerning teenage children at risk. Interestingly enough, consistent with the game theory example, the simple models did quite well, almost as good as the more complex Machine Learning models. Given the danger of unrevealed overfit, one can argue simple is always better, especially when different algorithms have similar explanatory power. (To wit: Occam’s Razor)
(9) This quote is from the book Noise: A Flaw in Human Judgment by Daniel Kahneman, Olivier Sibony, Cass R. Sunstein.
(10) Troy Segal, The Five C's Of Credit, March, 2020
(11) Thomas L. Saaty (1982) Decision Making for Leaders: The Analytic Hierarchy Process for Decisions in a Complex World
(12) Definitive Business Solutions, Inc. designed and integrated a bespoke solver engine, utilizing a number of solver programing code sets and data transmission capabilities, uniquely needed for the AHP and cloud environment.