Nate Silver, The Signal and the Noise

“Statistical techniques, however, are subject to diminishing returns. Hurricanes are not exactly

rare, but severe storms hit the United States

perhaps once every year on average. Whenever

you have a large number of candidate variables

applied to a rarely occurring phenomenon,

there is the risk of overfitting your model and

mistaking the noise in the past data for a signal.

There is an alternative, however, when you

have some knowledge of the structure behind

the system. This second type of model essentially creates a simulation of the physical

mechanics of some portion of the universe. It

takes much more work to build than a purely

statistical method and requires a more solid

understanding of the root causes of the phenomenon.”

Jeff’s thinking….

Thinking on statistical models v simulation.

This is the kind of model I built for forecasting the appropriate risk premia used in loan acceptance and pricing decisions.

The model simulated the loan business, with statistical based inputs for consumer behavior. The traditional statistical approach was still used, but used for simulation inputs.

This model was harder and more complex, but also simple in that it was easy to understand the real world context of the model and the dynamics between the model components. Also, I'm sure there were plenty of SIFs assumptions that hopefully ere irrelevant or minimally irrelevant.

My biggest concern about simulation is, 1) how do you know you got the model right and 2) how do you account for the uncertainty that can require a new simulation because the world has changed? Especially in the case of economics.

1) can be handled through challenge but 2) is different. It may suffer from the same "inertia on inertia paradox" as do statistical models.

At least in the physical sciences, there are physical laws that can be counted on not to change. The last financial crisis demonstrated that economics does not have the same sort of stable laws. Especially in uncertain (wild) environments defined by high inertia, high entropy ...where independence is harder to achieve and kurtosis is >3.

## Comments