I once read that meteorologists are better calibrated forecasters than stock pickers. By calibrated, I do not mean they are better or worse at forecasting, but that their level of confidence matches their accuracy. When they believe they are 80 % likely to be right, they are right about 80 % of the time. When they are uncertain, their forecasts are indeed less accurate. Stock pickers, and I suspect some economists, tend to be less well calibrated. They are not necessarily poor forecasters, but they are often overconfident, and that is a different problem altogether.

The point came from Superforecasting by Philip Tetlock and Dan Gardner, which explored how some individuals become remarkably well-calibrated forecasters through constant feedback and disciplined self-assessment. One reason meteorologists perform better is feedback. Stock pickers and economists can always point to exceptional events that explain why forecasts went wrong. Who knew there would be a drought, or that Russia would invade Ukraine, or that COVID-19 would shut down economies? There is always a convenient exogenous shock. Weather forecasters do not have that luxury. They live by feedback. They issue forecasts every day and are judged immediately by outcomes they cannot explain away. Their models evolve constantly because their results are tested constantly.

That contrast came to mind when I first encountered Computable General Equilibrium (CGE) models about a decade ago. They were intriguing. A little too clever, perhaps, and impressive in their scale. Thousands of equations, countless parameters, and from all of that emerges the predicted effect of an intervention on GDP, employment, or wages—sometimes quoted to two decimal places. At the time, it seemed remarkable, but I could not see the transparency I wanted. There was little evidence of how these models were calibrated, how reliable they were, or how confident one should be in their results.

I came across them again recently. I am not saying CGE models are unhelpful or that their results are wrong. They can be useful for exploring direction and mechanism – how substitution between sectors or between labour and capital might unfold. They are elegant frameworks for thinking through equilibrium interactions. What concerns me is the way they are sometimes used. There is often little effort to quantify uncertainty, measure accuracy, or report diagnostics that would tell us how well the model is performing. That makes me uneasy.

I would like to see genuinely robust studies that take a CGE model, generate predictions, and check them against what actually happens. Yes, there will always be surprises, but if the models barely outperform a simple trend line, the enormous complexity and opacity seem hard to justify.

Even before that, I would want to see what the model predicts with no intervention at all. CGE models are built to find a general equilibrium, but there is no guarantee the economy we feed in is itself in equilibrium. The first step should surely be to see what the model does on its own. If its natural equilibrium is very different from the starting point, that tells us something. Either the real economy is far from equilibrium, or the model’s structure is not reflecting it well.

I am not suggesting this problem occurs frequently, but the diagnostic results are rarely reported, and that makes me wonder whether it sometimes does. If the model is not allowed to reach its internal equilibrium before the intervention is applied, the change in GDP or other results could be partly driven by the model’s own adjustment toward equilibrium. It would be useful to know.

I would also like to see how stable the results are when the model is run with successive years’ data. If we input the economy as it stood each year, do the parameters and balances remain consistent? If not, that raises questions about the model’s stability. For example, can it predict the next year’s social accounting matrix (the SAM) with any meaningful accuracy? If it cannot, then I am not sure how much confidence we can place in the counterfactual results.

As I understand it, CGE models are calibrated rather than statistically estimated. They rely on factual data inputs, some parameters solved to achieve equilibrium, and elasticities imported from previous studies. Those elasticities differ across the literature, and the uncertainty around them is often considerable. How confident are we in those numbers? What happens if we vary them within plausible ranges? If small changes produce very different equilibria even before any intervention, then those elasticities are critical. If the results remain fairly stable, that is reassuring.

We could also test how much the estimated impact of a policy or intervention depends on these elasticity choices. If the results swing widely, confidence in the model’s precision should be low. And if the model’s internal uncertainty interval for GDP, wages, or productivity is wider than the intervention effect, that should be stated clearly.

In actuarial work, we routinely perform analysis of surplus to understand why results differ from expectations. We ask whether deviations stem from data, assumptions, or the model itself. That discipline drives improvement and professional scepticism.

Actuaries are not perfect. There are areas of our own work where experience analysis or model validation is weak. Some solvency projections (ORSA AvE notably, but there are others) , for example, are never properly reconciled to outcomes. That is disappointing, but at least the mindset of testing and validation is embedded in the profession. The actuarial control cycle is still taught.

We are also learning from “new” data science, which takes validation seriously. Models are trained and tested on different data, evaluated out of sample and out of time, and using techniques such as Monte Carlo Cross Validation (a new discovery for me) or k-fold cross-validation. These approaches help ensure that models perform reliably, not just on the data that built them but on data they have never seen.

I would like to see economists apply similar rigour to CGE modelling. Calibration is not validation. Without testing, backtesting, and clear communication of uncertainty, the apparent precision of these models can be misleading.

CGE models can be valuable tools, but their uncertainties and dependencies are often poorly understood, poorly expressed, and underappreciated.

Perhaps I am missing something. Perhaps these diagnostic checks are done quietly somewhere. I would be glad to be corrected by economists who understand these models better than I do. I am genuinely open to learning more.

Because if these models can withstand that kind of scrutiny, they deserve confidence. If they cannot, they still have value, but we should be honest about what they are: structured thought experiments with useful stories to understand. Not forecasts – and those second decimals places should never be shown.

Pick n Pay had to spend a fortune on the Smart Shopper system and has ongoing expenses in terms of rewards and analysis. The curious thing for me is how many loyalty cards incur the system and reward costs for retailers, but without gaining the full benefit of analysis and thus insight into customers.

I don’t get tailored book suggestions from Exclusive Books. They also haven’t tried to entice me back to their stores since I started buying first from Bookfinder.com and then almost exclusively ebooks from Amazon. They’ve basically lost a customer and haven’t done anything about it.

Even my friend’s St Elmos offers sweet deals to customers who haven’t ordered in a while to entice them back. Pick n Pay turned sub R100 pm customers into R350 pm customers (at least while the special was one) by specifically targeting customers that are familiar with Pick n Pay but need a push to become regular, high-spending customers.

I haven’t had a movie card with Ster Kinekor in a while, but I always use the same email address and credit when I purchase tickets online (which I do almost universally). There have been periods of several months where I haven’t gone to the movies, but no attempt from Ster Kinekor to woo me back with free popcorn or a careful movie recommendation.

Retailers are missing a trick to get an edge over their competitors.

So what assumptions are pension funds making? The WSJ has an interesting article showing that the average US pension fund is assuming future returns of approximately 8%. To put that in perspective, yields on 30 year T-bonds in the US are about 3.9%, 10-year yields are below 3% and inflation is currently about nothing. This is a huge real return and suggests that many of these pension funds may be underfunded.

It’s also interesting to work out what Equity Risk Premiums these valuation assumptions imply. FinanceClippings makes  some educated guesses at likely portfolio construction, and estimates assumed ERPs of nearly 8%. For reasons I’ve described before, an 8% ERP is madness.

My own calculations

FinanceClippings assumes a simple portfolio mix of 50% equities and 50% government bonds in this calculation, and assumes the average yield will be consistent with 30-year assumptions. I would differ slightly here. If we are looking at an overall portfolio, I would expect some investment grade corporate bonds and property in the mix too. These assets could be expected to earn 1% to 2% over risk-free over time (after adjusting for expected default loss on the corporate bonds). These return assumptions may seem low to some, but this is another area where it’s easy to overestimate the possible returns based on inappropriate periods of data.

On the other hand, it might be a little optimistic to assume 30-year yields as the average yield for the portfolio, depending on the remaining term of these pension funds. If we use a shorter duration, the risk-free yields approach 3% quite quickly.

Using a range of assumptions that make more sense to me, I see the pension funds assuming ERPs of between 6% and 6.5%.  Lower than the 8% to be sure, but optimistic all the same. (The sharp-eyed reader may see in my calcs that I am assuming an additive ERP rather than a compound ERP. Small differences, doesn’t change the conclusions at all.)

Check the calcs and assumptions and play around with them yourself –> Pension ERP implied assumption calculator tool [XLS]

Irresponsible assumptions

Whichever methods one applies here, it’s clear that too-high ERPs are being assumed. It’s irresponsible and it’s setting up tomorrow’s employees, shareholders, customers and citizens for problems that companies and valuators simply didn’t have the courage to deal with today.

It’s done none of these because the logic underlying it was flawed. Analysts focused on interconnect as an expense, happily ignoring the revenue side (since it was a fee paid to another company within the industry). Never has a telecoms issue been so badly hijacked by lack of understanding.

Now, in a press release that is a little vague, BMI TechKnowledge reflect concerns that telecoms growth rates may be lower as a result of falling mobile termination rates.

The economics of a fund manager depends entirely on growing funds under management so that revenues (based on assets under management) grow to be larger than costs (significantly fixed and at most semi-variable). Details of performance fees and the second order impact of investment performance aside, a successful fund manager must attract positive net client cashflow, and lots of it.

Half the 960 available unit trusts have less than R100m in AUM. Some of these may be rapidly growing new funds, but many have been stagnant with slow growth for several years.

The FSB’s attention presents opportunities for consolidation between funds and should place larger funds in a stronger position competitively. Total Expense Ratios (TER) for these funds with significant scale should already be lower than smaller funds. Maybe it’s time the larger funds made more if their size and cost efficiencies. If they are going to take the heat for being too large to be nimble, they might as well reap the benefits too.

It will be interesting to see what this means for white labelled funds and whether the economics of these convince the regulator that they should survive.

Posted with WordPress for BlackBerry.

Their housing price index is more robust (and more independent) than those of the banks based off their own data and target markets. Rather than consider only the average price of houses sold in that particular month (which is a function of house price growth / decline but also how the type, condition, size and location of the houses sold that month differ from the prior month and year) they consider repeat sales where the same property has been bought and sold more than once.

This data is combined or “chain-linked” to provide a continuous measure of house price inflation over time.

The result of all of this data, best-in-class methodology and analysis? When Lightstone says “opportunities abound in local market” I actually listen. Since their business model is to sell information, I’m more likely to trust what they say.

1 The Competition Commission is necessary

Monopolies serve to increase prices for consumers. In Monopoly, the “rents” charged are instantly higher as soon as a player has a monopoly on property in a certain area.

Worse than the increase in prices and decrease in supply, the additional profit for suppliers is not equal to the cost to consumers from higher prices, resulting in an overall “dead weight loss of monopoly” or an overall cost to society.

To date, the Competition Commission has focussed on cartels and price collusion. We can only hope soon they will consider issues of local loop unbundling to provide real competition to the effective monopoly Telkom has on ADSL lines.

2 The easy availability of credit is a useful and dangerous tool

An important consideration in Monopoly is to prevent competitors from owning property (so that you don’t have to pay rent on as many properties as possible) and definitely don’t get a monopoly (so they can increase rents and improve the properties, increasing rents even further).

A key way of doing this is by buying as many properties as possible, even if you mortgage many of them with a fresh injection of cash from the bank and very reasonable repayment terms. Leverage, or borrowing, can provide cost effective (especially after tax considerations) financing for profitable projects. It can be cheaper to raise than equity and increases returns to shareholders.

Leverage also creates significant risks, both in Monopoly and real life. Overextending can make you more likely to end up bankrupt. Also, by artificially increase the supply of funds for purchasing property while keeping the supply constant, the price of property will increase dramatically with distorting impacts on the allocation of resources within an economy.

3 Trading is good for those who trade

Every time two people (or organisations, or countries) trade, both parties are better off than they were before. (A hint, otherwise they wouldn’t have traded.) The trading parties are each better off, and the non-trading competitors are worse off, because a potential trade is now off the table and they are not in a better position otherwise.

There are some plausible arguments for trade restrictions in very particular circumstances. For example, I think I buy the idea of nurturing infant industries, but only if there is a reasonable chance that the industries won’t only grow as far as pimply, sulky adolescents with no drive or ambition.

Other than that, trade is good as it increases prosperity on average, which is one important way to lift more families incomes above the poverty line.

4 Liquidity risk can be the end of you

While buying as much property as possible and mortgaging some to provide additional funds will generally work in your favour and allow you greater control over the board, it also introduces liquidity risk. If you don’t have the funds to pay rent or taxes or fines when they unexpectedly fall due, you will be forced into a fire-sale of houses and hotels at way below replacement cost. You may need to mortgage even more properties, reducing your income generating ability and possibly sending you into a debt trap where your income isn’t sufficient to meet your expenses.

Always watch your liquidity. It has almost become a truism that more businesses fail because of liquidity than solvency or profitability.

5 The world is full of randomness, but it always helps to calculate the numbers

In Monopoly, the player “roles the dice and moves the mice” so to speak. There is plenty of luck around and skill can seem to disappear in an avalanche of random numbers.

Nassim Taleb (of Fooled By Randomness fame) would suggest that in everyday life we underestimate risk and overestimate skill. I tend to agree with this overall comment. It doesn’t mean that we should try to use our skill as much as possible.

Did you know that certain properties are relatively more valuable than others? The properties just after Jail are more likely to be hit than others, since several cards and one square on the board send the player to jail. Similarly, several chance cards require the player to advance a certain way, making the squares immediately after a chance square less likely to be hit. The cost to benefit of houses on certain squares are better than others, and the best bang for buck is somewhere between 2 and 3 houses per property.

It’s important to be aware that risk and uncertainty exists, but it’s more important to understand risk in a way that allows informed, risk-aware decisions to be made.

Identify, measure, mitigate and monitor risk!

Are all the lessons good?

Slightly tongue in cheek, there are a few other lessons of dubious validity:

1. Stay in jail and make more money
2. You get to keep the money if the bank makes an error in your favour
3. You can take as long as you want to repay your mortgage and only have to pay 10% interest
4. If the bank runs out of money, it can always photocopy some more
5. When the game is over, you can forget about the debt you have and walk away

Zurich Financial Services has just been fined £2.3m for a data loss event incurred in 2008 in South Africa.

Zurich joins HSBC, Nationwide and Norwich Union in the club of companies fined by the FSA now.

In fairness, the fine wasn’t so much for losing the data, but rather for:

• losing
• unencrypted data
• and not having monitoring and controls in place
• so that it was only discovered and reported to regulators a year later

The South African perspective

The FSA’s seriousness about these issues is mirrored in our looming Protection of Personal Information Bill. This is not the same as the disturbing proposals for a Protection of Information Bill which covers public or government information.

The Protection of Personal Information (POPI) Bill seeks to effect provisions in our constitution for rights to privacy. As more and more private and confidential information about each of us is stored, processed, transmitted and mined by institutions, there is a clear need for controls around what can done with this information and what controls and safeguards are required.

Operational or legal risk?

The fines and penalties associated with data and privacy laws create additional risks for any enterprise with customer data on file.  Operational risk is often (although not uniquely) defined as the failure of people, processes or systems giving rise to a loss. Legal or compliance risk is the risk of falling afoul of the law through non-compliance with laws and regulations.

Yes, this risk should be covered by your risk management system

It’s not particularly important how your organisation classifies the risk, but it is critical to identify, measure, manage and monitor the risks as part of an enterprise-wide risk management system.

As with all risks, it’s often the allocation of specific responsibility for risks, the listing of risks in a risk register and the regular reporting on these risks that slowly changes and organisations attitude to it more than anything else. Whether or not you model the risk in detail or attempt some sort of quantitative analysis is decidedly secondary.