Category Archives: modelling

How not to calibrate a model

Posted on by
Reply

Any model is a simplification of reality. If it isn’t, then it isn’t a model as rather is the reality.

A MODEL ISN’T REALITY

Any simplified model I can imagine will also therefore not match reality exactly. The closer the model gets to the real world in more scenarios, the better it is.

Not all model parameters are created equal

Part of the approach to getting a model to match reality as closely as possible is calibration. Models will typically have a range of parameters. Some will be well-established and can be set confidently without much debate. Others will have a range of reasonable or possible values based on empirical research or theory. Yet others will be relatively arbitrary or unobservable.

We don’t have to guess these values, even for the unobservable parameters. Through the process of calibration, the outputs of our model can be matched as closely as possible to actual historical values by changing the input parameters. The more certain we are of the parameters a priori the less we vary the parameters to calibrate the model. The parameters with most uncertainty are free to move as much as possible to fit the desired outputs.

During this process, the more structure or relationships that can be specified the better. The danger is that with relatively few data points (typically) and relatively many parameters (again typically) there will be multiple parameter sets that fit the data with possibly only very limited difference in “goodness of fit” for the results. The more information we add to the calibration process (additional raw data, more narrowly constrained parameters based on other research, tighter relationships between parameters) the more likely we are to derive a useful, sensible model that not only fits out calibration data well but also will be useful for predictions of the future or different decisions.

How not to calibrate a model

Scientific American has a naive article outlining “why economic models are always wrong”. I have two major problems with the story: Continue reading →

Multi-tasking lowers productivity

Posted on by

I have to multi-task. I am the bottle-neck for too many problems already and my team needs input from me before they can continue in many areas.

But I know it doesn’t make me efficient. Switching between tasks takes time. You forget important details, and struggle to get the depth of understanding and focus required for complex issues. The cross-pollination of ideas and solutions doesn’t come close to making up for these drawbacks.

From the research:

Descriptive evidence suggests that judges who keep fewer trials active and wait to close the open ones before starting new ones, dispose more rapidly of a larger number of cases per unit of time. In this way, their backlog remains low even though they receive the same workload as other judges who juggle more trials at any given time.

Did you read those magic words? “…backlog remains low…” I don’t know anyone who doesn’t wish for the luxury of a shorter backlog of work.

The paper itself is fairly complex, analysing theoretical models of human task scheduling.  You should probably add it to your pile of things to read in the middle of other work.

Property investment – the value of data over opinions

Posted on by

Lightstone have a trick up their sleeves. Their raison d’être is collecting, analysing, understanding and packaging data for themselves and others to use to understand past, current and future property valuations.

Their housing price index is more robust (and more independent) than those of the banks based off their own data and target markets. Rather than consider only the average price of houses sold in that particular month (which is a function of house price growth / decline but also how the type, condition, size and location of the houses sold that month differ from the prior month and year) they consider repeat sales where the same property has been bought and sold more than once.

This data is combined or “chain-linked” to provide a continuous measure of house price inflation over time.

House Price Inflation 2010

House Price Inflation 2010 source: lightstone.co.za

The result of all of this data, best-in-class methodology and analysis? When Lightstone says “opportunities abound in local market” I actually listen. Since their business model is to sell information, I’m more likely to trust what they say.

Most decisions are made without all the information

Posted on by

Tyler Reed blogs about entrepreneurs having to make decisions with limited information.

It’s almost all unknown

I don’t disagree.  It’s just that almost every meaningful decision ever made is made without all the information.

Unknowns can be categorised a hundred different ways. One way is to think about:

  1. Unknown past information
  2. Uncertainty around the current situation or position
  3. Unknown future outcomes

Even a game like chess, where the past history of the game is easily known by good players, the current position is clearly visible and all the possible moves are knowable, it is not possible have all the information about how your opponent will react to your move.

How to deal with decision making under uncertainty – part 1

Tyler suggests that gut-based decision making can be effective much of the time – and it can. It there genuinely is no time for anything more than an instinctive reaction, you probably are best going with your gut.

Even if you have plenty of time, listening to your guy to formulate an idea is a great idea. Insight comes partly from experience and the reinforced neural pathways of our learning brain. If you stop with the gut though, you are missing out. There is a tremendous amount of research showing how ridiculously badly our instincts perform in many areas, particularly those relating to uncertainty and complexity! Continue reading →

5 Things to Learn from Monopoly

Posted on by

I haven’t played Monopoly in a while (preferring Settlers of Catan, Carcasonne, Tigris and Euphrates and even Cranium), but after a recent conversation I started thinking about the game dynamics. There is surprisingly much that is relevant to the current story of our economy.

1 The Competition Commission is necessary

Monopolies serve to increase prices for consumers. In Monopoly, the “rents” charged are instantly higher as soon as a player has a monopoly on property in a certain area.

Worse than the increase in prices and decrease in supply, the additional profit for suppliers is not equal to the cost to consumers from higher prices, resulting in an overall “dead weight loss of monopoly” or an overall cost to society. Continue reading →

What other people want

Posted on by

Apparently, car thieves don’t want your pink car. It’s not because they don’t like the colour (although they probably don’t). It’s also not only that it’s too distinctive and will be easily spotted (see the discussion later about red cars).

It’s that nobody else wants it. The resale value is much lower than other vehicles, and the risks and costs of stealing it are no lower.

Unlike the conclusion around high risk vehicles in my post on hijacking, this actually means you should be safer in a pink car. Just not safer from ridicule.

Dutch professor, Ben Vollaard, studied theft rates for vehicles as part of his research area of the economics of crimes. The data covered 109 vehicles from 2004 to 2008. Not the largest sample size, but enough to start thinking. Continue reading →

Return to mumbling

Posted on by

In a previous post, Mumbling in the Dark, I argued that too much attention was being paid to Eskom’s proposed increased charges and too little at 5 separate points, the first being their actual cost of production.

It seems that NERSA has been looking at this and has discovered some areas where Eskom was incorrectly estimating their future costs. The numbers are large, but not so large in the greater budget of Eskom. This is more like a snowflake on an iceberg than anything meaningful, but it does show the value of looking at the real issues.

Interconnecting confusion

Posted on by

Interconnect fees and the reasons for their reduction are possibly the most misunderstood “big” news story over the last twelve months.

The hype and hoopla around this topic is fueled by our feelings as consumers of being charged too much big big monopoly companies. So I should start by saying that I’m not saying that we are paying too much. I’m not saying that because I don’t know enough about the costs of providing cellular services in South Africa. Maybe we are, maybe we’re not. Also, I’m not saying there aren’t monopolistic practices in the market – again I simply don’t know. Given the other stories torn from inside companies by the sharp teeth and salivating jaws of the Competition Commission, it’s understandable that many suspect consumer-unfriendly play by most large South African companies, particularly those in industries with a small number of players.

What I am saying is that most of what you read in the news about interconnect is horribly misguided.

The biggest misconception is that interconnect fees are an expense for cellular providers, and that the removal of this expense would allow them to reduce tariffs to consumers. Well, it is an expense, but it is also a source of revenue. Every time one company pays an interconnect fee, another company is receiving it.

Interconnect does not change the total amount of profit within the cellular industry. It may redistribute it a little, and there may be negative medium term competitive implications arising from interconnect, but lower interconnect won’t automatically increase profits that could allow competitive price lowering for the benefit of consumers.

TechCentral has an interesting article: Bain warns consumers not to expect cellular price cuts.  Of course, it also include some done-to-death flawed statements (whether from Bain or inserted by the zealous staff writer) such as:

Because new players have few customers at first, most calls on their networks will be to networks of other operators. High interconnection fees make it difficult for them to enter the market.

It’s not that this statement is incorrect (it is in fact correct) it’s just that it is horribly misleading because it only presents one side of the story. I’ve reworded it to provide the stunning insight: Continue reading →