The sentiment is understandable. The existing model is slow, poorly documented, and full of code that nobody fully understands. There is an opaque subroutine that handles a tranche of conventional with-profits business written in the 1980s, and the only documentation is a comment that reads: “Whatever you do, do not remove this.” The codebase is not so much software as it is sedimentary rock, and each layer contains its own fossils.

So you start fresh. Clean architecture. Modern platform. This time, you tell yourself, we will do it properly.

Joel Spolsky wrote about this pattern over two decades ago, calling it the single worst strategic mistake a software company can make. Netscape rewrote their browser from scratch and nearly died. More recently, Sonos rewrote their mobile app from the ground up in 2024, promising “an unprecedented streaming experience.” The result was a catastrophe: core features were missing, existing systems broke, the company’s share price dropped by 25%, and the CEO lost his job. The rewrite had taken two years. The team had done extensive user testing. The prototypes looked good. The problem was not the vision. It was the thousand things the old app had quietly been handling that nobody had thought to write down.

The complexity is in the problem, not the code

When a developer looks at a legacy actuarial model, they see spaghetti and hacks and massively nested if statements and think “bad code.” Sometimes they are right. But more often, the cludge is an accurate representation of complicated business logic. That weird branch that treats one closed book differently from every other? That is a tranche of policies migrated in from another licence in some distant corporate past, policies that stubbornly crawl off the books at their own pace rather than running off as the assumptions suggest. The hardcoded matrix of flags in the decrement logic? That is handling multiple causal surrender and lapse events, where the interaction effects were agreed through industry negotiation rather than derived from first principles.

Consider the products themselves. Conventional with-profits policies from the 1970s and 1980s, now-defunct retail discretionary smoothed bonus products, universal life variants with guarantee structures that differ by tranche because of policy wording glitches that affected business written during a narrow window. These are not edge cases. In a mature life insurer’s book, they are the book. Every product variant that was ever sold, however briefly, however accidentally, lives in the model forever.

You can refactor the syntax. You can rename the variables. You can restructure the control flow. You can attempt to enforce coding standards. The essential complexity survives every rewrite, because it was never a function of the code to begin with. It was in the contracts, the policy wordings, the historical decisions, and the accumulated institutional memory of the actuarial team.

This is why model rewrites so often converge back towards the same complexity as the original. The optimistic timeline assumes the new model will be simpler. It will not be simpler. It will be the same complexity with less battle-testing.

The rabbit hole goes deep

I recently fell down a rabbit hole reading about why COBOL systems persist in banking. The reason turned out to be more interesting than I expected, and it illustrates something important about legacy systems in general.

COBOL performs arithmetic in decimal. Not binary floating-point, which is what R, Python, C, and virtually every modern language use by default, but actual base-10 arithmetic. This matters because in binary floating-point, 0.1 cannot be represented exactly. It is not a rounding issue or an implementation bug. One-tenth in base 2 is a repeating fraction, the same way one-third is repeating in base 10. The practical consequence is that 0.1 + 0.2 does not equal 0.3 in any language that uses IEEE 754 doubles. Try it in R. It returns FALSE. Excel, incidentally, will tell you it returns TRUE, because Microsoft decided decades ago to silently paper over the issue with heuristic rounding at the display level. It is a load-bearing lie: technically dishonest, pragmatically essential, and the foundation upon which the entire global financial system’s spreadsheets rest.

For actuarial modelling this rarely matters in practice. You are working with estimates and probabilities, and 15 significant digits of precision is more than enough. But the point is not about COBOL or Excel specifically. It is about how deep the assumptions in any system go. The banking world discovered, when they tried to move off COBOL, that the decimal arithmetic was not just a feature of the language but a load-bearing property of the entire ecosystem. Rounding behaviour, reconciliation logic, audit trails: all of it assumed exact decimal representation.

Actuarial models have their own version of this. Assumptions about evaluation order, about when intermediate rounding occurs, about how decrements interact, about the precise sequence of operations in a monthly projection step. These assumptions are rarely documented because the people who wrote the model did not think of them as assumptions. They were just “how it works.” Until someone tries to replicate “how it works” in a new environment and discovers that a hundred small implicit choices produce a hundred small differences, each individually immaterial, collectively significant, and individually painful to diagnose.

The documentation mirage

The standard diagnosis at this point is: “Well, we should have documented it properly.” And yes, in an ideal world, every modelling decision would be recorded, every assumption justified, every edge case explained. But documentation has a half-life. The moment you write it, it starts decaying. And the effort to keep it current competes with the effort to actually do the work it documents.

Models most in need of documentation are the complex ones that change frequently, which are precisely the ones where documentation goes stale fastest. The simple stable model that has not changed in five years has beautiful documentation. The critical model that three people are iterating on weekly has a README from 2019 and some optimistic comments about “come back to this” in the code.

Good documentation requires a different skill from good modelling. Explaining why you made a choice is harder than making the choice. Most documentation ends up describing what the model does, which anyone with the code can see, rather than why it does it that way, which is the thing that is actually lost when someone leaves.

So when someone says “we will document the old model thoroughly before we migrate,” treat that with the scepticism it deserves. It is not that documentation is worthless. It is that comprehensive documentation of a complex actuarial model is closer to a research project than a task on a Gantt chart, and it will never be truly complete. The knowledge that matters most, the kind that explains why line 437 exists, often lives only in the model itself and in the memories of the people who built it. Sometimes the most valuable documentation in the entire codebase is a comment that says “do not remove this.” It tells you nothing about what the code does. It tells you the one thing that matters: someone before you tried removing it, and something terrible happened.

The testing problem is nearly intractable

The textbook software development answer to safe migration/refactoring is: write comprehensive tests against the old system, then verify the new system produces the same results. This sounds clean. It falls apart quickly in practice.

An actuarial model is not best thought of as a simple function that takes an input and returns an output. It is a function of data, assumptions, methodology, regulatory basis, reporting date, and a constellation of configuration choices. To truly test it, you would need to verify results across combinations of all of these dimensions: base and stressed assumptions, multiple products, different data vintages, BEL, SCR, RM, RA, EV, ORSA projections and more. The required test suite is, for practical purposes, infinite.

But even if you could write enough tests, testing locks in current behaviour, not correct behaviour. If the existing model has a subtle bug that slightly misstates the impact of a particular stress scenario, your tests will faithfully preserve that bug. You now have a comprehensive test suite that gives you high confidence in reproducing the wrong answer. Which is arguably worse than no tests, because it creates false certainty.

Conversely, tests that are too simple create their own risks. A test that says “the total reserve is within 1% of the old model” might pass while masking a situation where two large errors in opposite directions cancel out. Over time, as the portfolio changes and one of those errors stops being offset, the discrepancy surfaces, and by then nobody remembers that the migration was the cause.

The practical approach is something looser than formal test suites: attribution analyses, movement analyses, and sense checks that tell you why results changed, not just whether they changed. An unexplained difference is not a rounding tolerance to be waved through. It is an unknown.

The replication question

Here is a question that derails more migration projects than any technical challenge: during the rebuild, the team discovers that the old model has a bug. Maybe the lapse rates for a particular cohort are being applied incorrectly. Maybe a decrement interaction is not handling concurrent events properly. The old model has been producing results with this error for years, and those results have been reported, audited, and relied upon.

Do you replicate the bug in the new system to maintain consistency? Or do you fix it?

There is no clean answer. Replicating the bug feels deeply unsatisfying. Every professional instinct says to fix it now that you have found it. But fixing it means your “like-for-like” migration is no longer like-for-like, and every future reconciliation has to account for the correction. The reconciliation work, which is already the hardest part of any migration, becomes significantly harder when you are trying to separate platform differences from methodology changes.

My view is that, in general, replication first, correction later, is the lesser evil. Get the new system producing the same numbers as the old one, including the known (or newly identified) bugs, and then fix the bugs as a subsequent, documented change with its own impact assessment. It is slower. It is unsatisfying. But it is the approach that keeps the audit trail clean and the actuarial team’s confidence intact.

This only works if errors are logged rigorously at the point of discovery. Each one needs a description of the issue, the products and tranches affected, an estimate of the financial impact, and enough technical detail that a reader coming to the register months later can find commonality across findings and know where to start when the correction work begins. A vague note that says “lapse rates may be slightly off for product X” is not useful. A note that says “the lapse decrement for product X, tranches written between 2003 and 2007, is applied before the surrender decrement rather than simultaneously, resulting in an estimated overstatement of reserves of approximately R2m” gives the next person something to work with.

One big model or many small ones?

A migration forces a structural question that is easy to defer and hard to answer well: should the new platform use a single large model with flags and indicators to handle product variation, or multiple smaller models that are individually simpler but collectively harder to manage?

The single-model approach is appealing because it eliminates duplication. Core logic like decrements, lapse rules, and economic scenario generation lives in one place. Change it once, and every product picks up the change. But the model becomes increasingly complex, the flag-and-indicator logic becomes its own source of bugs, and a change intended for one product can have unintended consequences for others. Testing a single change requires running everything, which in a large model can mean hours of runtime.

The multi-model approach is appealing because each model is smaller, easier to understand, and faster to run. But it introduces duplication, and duplicated logic mutates over time the way DNA does: small transcription errors accumulate with each copy, and eventually two models that should be applying the same lapse basis are not. One gets updated, the other does not. A fix is applied in three of the five copies but missed in the other two. Maintaining consistency across a library of models requires discipline and tooling that is easy to underestimate.

There is no universally right answer. But the choice should be made deliberately at the start of a migration, not allowed to emerge organically, because reversing it later is extremely expensive.

Living with two systems

A related pragmatic question: does every product need to migrate?

For a mature life insurer, the long tail of the book often includes products that are tiny, declining, and odd. They might represent a few hundred policies with unique features that would take weeks to replicate in the new platform. The cost of migration far exceeds the operational benefit, especially if the product will run off within a few years anyway.

The temptation is to leave these in the old system and migrate everything else. This can work, but it introduces its own pain. Running two modelling environments means maintaining two sets of assumptions, two data feeds, two run processes. If the old system is not integrated into the new orchestration and workflow environment, the residual products become a manual process that someone has to remember to kick off, reconcile, and consolidate. Automated kick-off of external models outside the main controller environment is possible in principle, but brittle in practice: it adds integration points, error handling, and monitoring requirements that erode the simplicity gains of leaving the products behind.

Over time, the old system becomes an unloved orphan: patched reluctantly, understood by fewer and fewer people, and increasingly fragile.

The decision should be explicit and time-bound. If the plan is to leave products in the old system, there should be a clear runoff date after which those products will either be migrated or, if they are small enough, approximated in the new system. An open-ended commitment to maintain two platforms in parallel is a commitment that tends to last much longer than anyone intended.

The case for migrating anyway

Given all of this, why would anyone migrate? Because the reasons to move are real, even if the process is painful.

The strongest argument is rarely about the actuarial code itself. The core projection logic, the bit that actually calculates reserves and capital, is broadly the same complexity regardless of the platform. The arithmetic is not all that special. The case for migration is about everything around that core: better long-term vendor support, modern workflow and automation capabilities, improved system architecture that enables genuine runtime improvements, proper change control and audit trails, user management, cloud processing, and the ability to integrate with modern data pipelines.

A legacy model might produce perfectly good numbers, but if the assumption update process involves manually editing text files, if change control is a folder of dated ZIP archives, if running a stress scenario means waiting overnight for a batch job, then the platform is constraining the business even if the arithmetic is fine.

Migration also presents an opportunity, if managed carefully, to clean up decades of accumulated inconsistencies. Not the load-bearing hacks, which exist for good reason, but the other kind: the copy-pasted subroutine that was modified slightly for each product and now exists in seven inconsistent versions. The assumption tables that use three different date conventions because they were built by three different people over fifteen years. The configuration settings that nobody is sure are still active. A migration done with eyes open can rationalise these, as long as the team resists the temptation to rationalise everything at once.

What not to do

One recurring temptation is to skip the vendor platforms entirely and build a bespoke system from the ground up in Python, or Julia, or whatever the language of the moment happens to be. The argument is appealing: we know our business better than any vendor, modern languages are fast enough, and we will have complete control.

I am sympathetic to this, because I have done it. In the early 2000s, I built a proof-of-concept actuarial system on an open-source stack: PHP and C++ on the back end, HTML and JavaScript on the front end, with memoized calculation so that values were computed on demand and cached for reuse. It was designed from the ground up to be standards-compliant so the server could run on a laptop, on premises, or in the nascent cloud. It worked – I’m still impressed with young me there. The core system logic was not the hard part.

The hard part was everything else. The assumptions manager. The data input validation and transformation. Change control and version history. User management and access permissions. Audit logging. Batch processing and job scheduling. Error handling and recovery. Reporting and output formatting. The ability to offload computation to remote infrastructure. That was where I bailed on the project.

Building a basic actuarial projection engine is a manageable problem. A competent developer can get a working prototype running in weeks (it took me a little longer…). But a projection engine is perhaps 20% of what an actuarial modelling platform needs to be. The other 80% is the engineering infrastructure that makes it usable, auditable, and safe in a production environment with multiple users, regulatory oversight, and real money at stake. That 80% is what the established vendor platforms have spent decades building, and it is what a bespoke system will spend years rediscovering.

The comparison is often made to a spreadsheet. Excel’s calculation engine is straightforward. What makes Excel a product is everything around it: the interface, the formatting, the collaboration features, the ecosystem. Building “Excel but for our specific needs” sounds efficient until you realise how much of the value is in the parts you thought were trivial.

A pragmatic framework

If you are considering a migration, here is what I would suggest.

1. Be honest about the motivation. If the primary driver is “the code is complex,” that is not a good enough reason. The new code will be complex too, eventually, because the problem domain is complex. If the driver is “the vendor’s support is no longer working” ,” or “our current architecture cannot support the automation and workflow improvements the business needs,” those are real reasons that justify real investment.
2. Plan for the migration to take longer than you expect. (Hofstadter’s Law applies with full force here: it always takes longer than you expect, even when you take into account Hofstadter’s Law). Every model migration I’ve seen that stayed on its original timeline did so by cutting scope, not by being faster than expected. Budget for the edge cases, the discovered bugs, the reconciliation work, and the institutional knowledge that only surfaces when someone tries to replicate it. If your initial estimate is eighteen months, plan for three years and you might finish in two and a half.
3. Decide your replication and error-handling policy upfront. I have argued for replicating first and correcting later, with a rigorous error log that gives subsequent teams enough detail to act on. Either way, what matters is that the policy is deliberate and that discoveries are recorded with care. Not having a policy is not an option.
4. Invest in attribution analysis rather than pass/fail testing. You need to understand why results differ, not just detect that they differ. A 0.1% difference with a clear explanation traced to a specific implementation choice is better than a 0.0% difference that you achieved by accident.
5. Make deliberate structural choices early. One model or many? Which products migrate and which stay behind, and for how long? These decisions shape the entire project and are expensive to reverse.
6. Resist the temptation to improve the model and migrate it simultaneously. The migration alone is a large enough project. Bundle it with methodology changes and you lose the ability to diagnose whether a difference in output comes from the new platform or the new approach. Migrate first. Improve second. The discipline to separate these is the single biggest predictor of success. (One exception here might be preparing for GPU acceleration or improved vectorisation etc.)

Joel Spolsky was right that rewrites are dangerous. Sonos learned it the hard way. Netscape learned it before them. But Spolsky was writing about software companies, where the old product is still shipping and generating revenue while the new one is being built. Actuarial model migrations are a different beast: you cannot run two production models indefinitely, the regulatory environment is evolving whether you migrate or not, and eventually the cost of not migrating exceeds the cost of migrating.

The trick is to go in with realistic expectations about how hard it will be, a clear-eyed understanding of what you are actually gaining (hint: it is not cleaner code), and enough humility to respect the load-bearing hacks you will encounter along the way. That comment that says “whatever you do, do not remove this”? Read it as a colleague speaking to you across time. They learned something the hard way. Your job is to learn it without repeating the pain.

I once read that meteorologists are better calibrated forecasters than stock pickers. By calibrated, I do not mean they are better or worse at forecasting, but that their level of confidence matches their accuracy. When they believe they are 80 % likely to be right, they are right about 80 % of the time. When they are uncertain, their forecasts are indeed less accurate. Stock pickers, and I suspect some economists, tend to be less well calibrated. They are not necessarily poor forecasters, but they are often overconfident, and that is a different problem altogether.

The point came from Superforecasting by Philip Tetlock and Dan Gardner, which explored how some individuals become remarkably well-calibrated forecasters through constant feedback and disciplined self-assessment. One reason meteorologists perform better is feedback. Stock pickers and economists can always point to exceptional events that explain why forecasts went wrong. Who knew there would be a drought, or that Russia would invade Ukraine, or that COVID-19 would shut down economies? There is always a convenient exogenous shock. Weather forecasters do not have that luxury. They live by feedback. They issue forecasts every day and are judged immediately by outcomes they cannot explain away. Their models evolve constantly because their results are tested constantly.

That contrast came to mind when I first encountered Computable General Equilibrium (CGE) models about a decade ago. They were intriguing. A little too clever, perhaps, and impressive in their scale. Thousands of equations, countless parameters, and from all of that emerges the predicted effect of an intervention on GDP, employment, or wages—sometimes quoted to two decimal places. At the time, it seemed remarkable, but I could not see the transparency I wanted. There was little evidence of how these models were calibrated, how reliable they were, or how confident one should be in their results.

I came across them again recently. I am not saying CGE models are unhelpful or that their results are wrong. They can be useful for exploring direction and mechanism – how substitution between sectors or between labour and capital might unfold. They are elegant frameworks for thinking through equilibrium interactions. What concerns me is the way they are sometimes used. There is often little effort to quantify uncertainty, measure accuracy, or report diagnostics that would tell us how well the model is performing. That makes me uneasy.

I would like to see genuinely robust studies that take a CGE model, generate predictions, and check them against what actually happens. Yes, there will always be surprises, but if the models barely outperform a simple trend line, the enormous complexity and opacity seem hard to justify.

Even before that, I would want to see what the model predicts with no intervention at all. CGE models are built to find a general equilibrium, but there is no guarantee the economy we feed in is itself in equilibrium. The first step should surely be to see what the model does on its own. If its natural equilibrium is very different from the starting point, that tells us something. Either the real economy is far from equilibrium, or the model’s structure is not reflecting it well.

I am not suggesting this problem occurs frequently, but the diagnostic results are rarely reported, and that makes me wonder whether it sometimes does. If the model is not allowed to reach its internal equilibrium before the intervention is applied, the change in GDP or other results could be partly driven by the model’s own adjustment toward equilibrium. It would be useful to know.

I would also like to see how stable the results are when the model is run with successive years’ data. If we input the economy as it stood each year, do the parameters and balances remain consistent? If not, that raises questions about the model’s stability. For example, can it predict the next year’s social accounting matrix (the SAM) with any meaningful accuracy? If it cannot, then I am not sure how much confidence we can place in the counterfactual results.

As I understand it, CGE models are calibrated rather than statistically estimated. They rely on factual data inputs, some parameters solved to achieve equilibrium, and elasticities imported from previous studies. Those elasticities differ across the literature, and the uncertainty around them is often considerable. How confident are we in those numbers? What happens if we vary them within plausible ranges? If small changes produce very different equilibria even before any intervention, then those elasticities are critical. If the results remain fairly stable, that is reassuring.

We could also test how much the estimated impact of a policy or intervention depends on these elasticity choices. If the results swing widely, confidence in the model’s precision should be low. And if the model’s internal uncertainty interval for GDP, wages, or productivity is wider than the intervention effect, that should be stated clearly.

In actuarial work, we routinely perform analysis of surplus to understand why results differ from expectations. We ask whether deviations stem from data, assumptions, or the model itself. That discipline drives improvement and professional scepticism.

Actuaries are not perfect. There are areas of our own work where experience analysis or model validation is weak. Some solvency projections (ORSA AvE notably, but there are others) , for example, are never properly reconciled to outcomes. That is disappointing, but at least the mindset of testing and validation is embedded in the profession. The actuarial control cycle is still taught.

We are also learning from “new” data science, which takes validation seriously. Models are trained and tested on different data, evaluated out of sample and out of time, and using techniques such as Monte Carlo Cross Validation (a new discovery for me) or k-fold cross-validation. These approaches help ensure that models perform reliably, not just on the data that built them but on data they have never seen.

I would like to see economists apply similar rigour to CGE modelling. Calibration is not validation. Without testing, backtesting, and clear communication of uncertainty, the apparent precision of these models can be misleading.

CGE models can be valuable tools, but their uncertainties and dependencies are often poorly understood, poorly expressed, and underappreciated.

Perhaps I am missing something. Perhaps these diagnostic checks are done quietly somewhere. I would be glad to be corrected by economists who understand these models better than I do. I am genuinely open to learning more.

Because if these models can withstand that kind of scrutiny, they deserve confidence. If they cannot, they still have value, but we should be honest about what they are: structured thought experiments with useful stories to understand. Not forecasts – and those second decimals places should never be shown.

As the parametric insurance market evolves, so too must our approach to capital modeling. The challenges are significant, but so are the opportunities for innovation and more accurate risk assessment.

Queue management at the motor vehicle licence renewal office offers a window into a realm of complexity that extends far beyond the waiting room. Exploring priority queuing and weighted fair queuing (WFQ), we uncover a world of probabilistic models, dynamic rules, and real-world applications that challenge conventional wisdom.

Upon arrival, patrons encounter priority queuing, where elderly individuals receive absolute precedence to accommodate potential mobility challenges. While this prioritisation addresses (quite reasonably!) the needs of a vulnerable group, it can lead to excessively prolonged waits for others, depending on the arrival rate of elderly patrons and the dynamics of the queue. Consideration of different objective functions for various demographics, including the young and old, could unveil alternative optimal solutions.

Actuaries are used to implicitly allowing for objective functions that aim to minimise mean squared errors, but this isn’t the only possibility. Aside from different penalties for waiting times for all participants, explicitly considering different costs to different groups of individuals makes it more interesting.

In contrast to absolute priority queuing, WFQ introduces a probabilistic element, ensuring that even low-priority individuals have a chance of being served at every point in time. The weights can be tuned for those different groups, still achieving prioritisation for the elderly. While wait times for low-priority customers will still be longer than priority customers, they remain finite, underscoring the equitable nature of this approach. I can imagine an algorithm where the weight depends on the wait – the longer the duration of the existing wait the more an individual is prioritised.

The network packets to which WFQ most commonly applies probability aren’t conscious, and therefore their experience of the wait isn’t important. For humans, the conscious experience of knowing that you don’t have to wait until every individual in a priority group is served before you might make the wait more palatable. I’m not a fan of gamification; overall this seems to have result in evil outcomes, greedily addictive algorithms and a worse society overall. However, the introduction of a little randomness and the possibility of a serendipitous early shot at renewing a licence might be a net positive.

Beyond the motor vehicle licence renewal office, the principles of queue management find relevance in diverse fields, including insurance companies. By leveraging similar concepts, insurers can prioritise claims payment or customer service based on factors like severity or urgency, but also explicit objective functions to optimise for target response times within particular bands, aiming for prompt resolution while maintaining fairness and efficiency. Sometimes the maths will demonstrate that based on demand and supply of support calls, failing these objectives is inevitable. Modelling can determine how often that may occur and then you can weigh up the costs of more call centre agents with the unappealing customer experiences.

These waiting processes should ring bells from prior study of queuing processes, of exponential, poisson, negative binomial and gamma distributions. These provide a mathematical framework for understanding and optimising queue management, revealing the hidden complexities behind seemingly simple processes. While you may not think about claims processes as queuing processes, there are some surprising linkages there too.

For me, delving into the dynamics of network switching provided some fun insights. In network architecture, switches play a critical role in directing data packets efficiently across networks. However, the limited capacity of switches can lead to packet loss during times of congestion. Buffers within switches temporarily store packets to alleviate congestion, but if overwhelmed, they are forced to drop packets, resulting in packet loss and network underperformance.

A smarter (yet initially counterintuitive step) is to apply Random Early Detection (RED), a mechanism that deliberately drops packets in anticipation of the switch being overwhelmed. Under RED, these strategically dropped packets provide information to network endpoints to manage the speed of transmission to prevent larger problems down the line. To an extent, this can can be likened to passengers on an overbooked flight being paid to take a different flight. However, unlike RED, this approach does not alert endpoints or other passengers to slow transmission rates, highlighting the unique challenges and solutions in network management.

While queue management may appear remote from your regular work, some exploration might be worth the time spent. By embracing probabilistic models, dynamic rules, and real-world applications, we may see a way to apply our skills in new areas, or even learn from the significant advancements in network modelling in our financial services world.

Back to my world – often the best model is no model. If you must have a model, a simple model is better than a complex model. Simplifications aren’t just not a bad thing, they are good and necessary.

Complexity has massive downsides – run-time, increased risk of errors, more involved documentation requirements, longer development time, longer handover time, longer pick-back-up-months-later time, and harder to explain to clients.

Finding the appropriate simplifications is key to so much of what we do.  Approximations are not even just “harmless errors” they are a critical feature.

And yes, she totally killed her husband.

But trying to imagine scenarios, particularly very severe scenarios, isn’t straightforward. We don’t have many examples of very extreme events.

Some insurers will dream up scenarios from scratch. It’s also common to refer to prior events and run the current business through those dark days. The Global Financial Crisis is a favourite – how would our business manage under credit spread spikes, drying up of liquidity, equity fall markets, higher lapses, lower sales, higher retrenchment claims, higher individual and corporate defaults, switches of funds out of equities, early withdrawals and surrenders and increased call centre volumes?

Downwards counterfactual analysis is the:

• analysis
• of events different (counter to) actual facts
• that are worse (downwards) than reality

For stress testing, we could take a past scenario and make it worse. What might have happened in 2007/2008 if African Bank had failed then, rather than a few years later? What would have been the result of an SA government sovereign downgrade at the same time, or if a major insurer got their hedging wrong enough to be really in trouble? What if the Cape Town water crisis hit at the same time and resulted in a cholera outbreak?

Some of those options are natural progressions of the factual scenario, whereas others (water + cholera) are mere add-ons.  The strongest sort of counterfactual analysis is where the original scenario was better than it might have been due to good luck. Tweaking that to an equally likely or even more likely but worse scenario powerfully shows how close we came to real disaster. A change in the prevailing wind during the Fukushima nuclear disaster is an obvious one.

Lloyds and RMS have put out a fascinating paper on downwards counterfactual analysis called “Reimagining history”.

It’s interesting stuff and well worth a full read.

I firmly believe in anthropogenic climate change. I am not an expert, but my reading has convinced me of the seriousness of the issues, the overwhelming evidence that it is humans at fault. Having young children makes me seriously concerned about our ability to remedy the mess we’re in for their sake.

Part of my very inwards focussed, selfish research was on the practical impact for where I live. Unfortunately for my cycling aspirations, this is current on the bottom slopes of Table Mountain.  Fortunately for my flood risk, the same.

The screen capture below is of an extreme, and hopefully not realistic, 30m rise in sea levels.  The website I used for visualising the impact of sea level rises makes no claims to accuracy, but it is interesting all the same.

The fact that my house is on dry land doesn’t nothing to address the loss of the Cape Town Foreshore and most of the city.  The picture is worse in the Netherlands of course, but all of our reclaimed “foreshore” is obviously gone.

However, the part that I missed until I zoomed out a little is that my dream of living on an island may just come true.  Not exactly what I had hoped for playing Kokomo.

My training and experience has given me high confidence that this is the right way forward for typical actuarial data.  Here I’m talking in force policy data files, movements, transactions, and so on.  This is really well structured data that will be used many times by different people and can easily be processed once, “on write”, stored in the data warehouse to be reliably and simply retrieved whenever necessary.

The particular issue was effectively about a single transaction, a single claim, being recorded as two separate items because of administration system limitations that required the claim to be processed as two equal amounts rather than a single claim. I can understand wanting to possibly also store the two separate transactions for audit trail purposes, but in terms of the claim for movements and analysis runs and checking against policy rules, it needs to reflect the product, business, actuarial model and customer reality of a single claim.

If the data is permanently stored as two separate, equal transactions, the end consumer of that data will always need to know to combine them, that they’re not duplicates (well they are, but they’re not – see how confusing it can be?). It really is standard practice for data warehouses to have “schema on write” in place. This talks to the structure of the data, but also the cleaning and transformation, application of business and validation rules to the data to fit the data definitions exactly.

And then I heard about Data Lakes. At first, these same a little like a new buzzword for data warehouse. But in fact it is quite different. The idea behind a data like is to store large amounts of unstructured data, as it comes, and figure out what to do with it later.

This is “schema on read”.

If you can imagine an actual lake, full of different things, sand and mud and fish and seaweed, but also boats and piers and old anchors and cool water and warm water and maybe the occasional Loch Ness Monster. Some of it may be structured, most of it not, and you don’t need to define what you will store ahead of time. You just throw it all in the data lake and “figure out what to do with it later”

The value of a data lake is that it is flexible. Nothing is lost from the original data. If a brand new purpose for the data arises that wasn’t even imagined when the data was originally stored, no avenues have been closed off. This is not true for a data warehouse.

However, for the types of data used for a simple actuarial valuation, locking down the exact data to be used to ensure results are consistent, that all valid policies are valued, that the analysis of experience and surplus work correctly, a rigid, reliable data source is required.

If a data lake is to be used, then another store of the highly structured, cleaned and transformed and validated data will be required to support the time-sensitive standard valuation processes. Whether this is  data warehouse feeding off the data lake, or just a simple data base, I suppose if up for debate. But a data lake on its own holds too many monsters to be used raw.

• The risk of contagion, where failure of an entity leads to distress or failures of others [micro prudential]
• The risk of an event that can trigger serious consequences for the real economy. [macro prudential]

If you are a narrowly focussed prudential regulator, you might be concerned with the micro-prudential view. After the Global Financial Crisis acted as a reminder of the Great Depression, the impact on the real economy of failures within the financial sector now cannot be ignored.

Individual entities can fail and, possibly, be well managed through their failure to limit the financial impact on customers. The failure of several entities could similarly be limited in the breadth of its impact. Mass job losses, decline in GDP, failure of businesses outside the financial sector and other leakage into the real economy has costs measured as percentage of GDP foregone, likely over several years. Inevitably this a much larger hit than the failure of even several entities.

I don’t view this as a choice, either the micro view or the macro view. The failure of several entities is more likely to drive real economy effects. As I’ll discuss when in a later post when I talk about “Tsunamis”, no individual entity has to fail for their to real economy implications. Similarly, the failure of several financial services entities that doesn’t have significant real economy impacts is clearly undesirable.

The most useful view is of both of these as systemic risks and outcomes to be avoided.

A recent article in The Actuary magazine addressed whether “de-risking in members’ best interests?“  I say “recent” even though it’s from August because I am a little behind on my The Actuary reading.

In the article, the authors demonstrate that by modelling the impact of covenant risk, optimal investment portfolios for Defined Benefit (DB) pensions actually have more risky assets than if this covenant risk is ignored.

The covenant they refer to is the obligation of the sponsor to make good deficits within the pension fund. Covenant risk then is the risk that the sponsor is unable (typically through its own insolvency) to make good on this promise.

On the surface it should seem counterintuitive that by modelling an additional risk to pensioners, the answer is to invest in riskier assets, thus increasing risk.

The explanation proffered by the authors is that the higher expected returns from riskier assets allow the fund to potentially build up surplus, thus reducing the risks of covenant failure.

I can follow that logic, particularly in the case where the dependence between DB fund insolvency and sponsor default is week. It doesn’t mean it’s a useful result.

The optimisation considered only one side of the equation – what is in members’ best interests. It ignores the other side of the problem – the financial impact (expectation and variability around that) for the sponsor’s financial position.

To take it to the extreme, if every sponsor just liquidated all its assets and transferred them to the DB fund, that would be a pretty good outcome for fund members. Not so much for the sponsor.

Without having seen the detailed model results, what I expect is happening is that the increased allocation towards risky assets is increasing expected returns (as it should) but also increasing the risk to the sponsor of having to put in additional funds. Any time this is done and the sponsor doesn’t default, there is no downside for fund members. The only risk to fund members is the combination of being underfunded and sponsor default.

The risk to the sponsor of increased frequency of injections required has been well established for decades. Depending on what you are willing to assume about risk premiums for risky assets and the assumed risk appetite, the higher expected return might outweigh the increased risk. This is not a point to be glossed over let alone left entirely untouched.

It also raises an awkward question about what they’ve modelled in terms of the covenant in the “no covenant risk” scenario. Surely if one models the covenant and not risk to the covenant, all benefits should always be paid to the members, regardless of asset mix?

I’m not at all convinced that these results are reliable. But either way, the fact that they are optimising and showing results for only one side of a two sided problem makes the approach utterly flawed.