To test the forecasting power of Markov analytics for predicting market share changes, I thought to test it on forecasting population shifts among states. After all that is a brand swiching question too with solid government data.

According to tax rolls, from 2022-2023, California lost 200,000 dependents. NY over 100,000. Is that really anything to worry about? NY for example is still not far off its peak population.

 Well, here’s the bad news for those states.  This is barely the tip of the iceberg.

A Markov switching analysis of state-to-state movement (including staying in the same state) lets me calculate the equilibrium population shares for each state.  What is equilibrium? It is the share a brand (or state) is headed towards eventually…just like it doesn’t matter how hot a cup of tea is, it will always cool off eventually to the temperature in the room…thermal equilibrium.

So what is the “temperature of the room” so to speak for California and NY? DISASTER.

Markov analysis shows that each will lose about 35% of their share over time. Why isn’t this better known? Because analysts use time series rather than Markov forecasting which is unfortunate. Time series doesn’t know the eventual equilibrium share.  Disasters are hidden from time series (like NY and California) because it cannot untangle the lower eventual share from a slow rate of change partially offset by population growth. But Markov analysis can (equilibrium and “rate of change” are parts of the output). In the cases of California, NY, Texas and Florida, the model is predicting the observed population changes almost perfectly.

In round numbers, for California, its current share of population is 12% which is likely to decline to about 8%. For NY, once the largest state in the Union, its share, a little above 5%, is likely to decline by 2 share points.  In terms of equilibrium shares, Texas is forecasted to become number 1, followed by California, with Florida close behind.  These represent massive population shifts over time.

It’s death by a thousand cuts for California and NY.

What does this have to do with marketing?

Equilibrium-based forecasts are insights that leaders need to know about. We could have used this math when I was running analytics at Unilever USA but, although we were committed to such analysis, we never took it this far.  When I joined, we had some brands that were once big that had been losing share slowly but steadily…Lifebuoy, Lux, Pepsodent, Imperial margarine to name a few…this type of analysis could have rung the alarm and perhaps led good leadership to change the course of the future. Just as New York was once the ‘Empire State,’ Pepsodent was once the emperor of oral care.

Markov analysis also pinpoints the problem areas. In the case of California and NY, it is partially retention rate (they have to hold onto their citizens better if they want to hold share…calculated va Markov analytics), but it is also really bad net switching with certain states. For California their main problem is Texas. For NY, their main problem is Florida. Markov switching clearly shows this. Similarly in marketing, there might be some competitors that are eating your lunch.  You need to know who, how bad, and what you can do about it.  Markov encourages this line of inquiry. It encourages analysis of competitive interaction patterns and market structure. Other approaches, that tend to be single brand thinkers, do not.

So, what should we Marketers learn from this exercise?

1. Markov modeling is an important prediction approach that adds understanding of trends that time series analysis cannot deliver.
2. Market structure is real and close competitors need to be carefully studied. Brands live in a closed system, not isolated with univariate metrics.
3. The rate of change in the marketplace can be forecasted as well as the destination share.  It will depend on the average repeat or retention rate of competitors.  In the case of population movement things develop slowly because retention rates are high. In marketing, brand shares will move faster towards equilibrium
4. Markov analysis requires longitudinal data turned into brand switching matrices. A greatly underutilized tool in marketing and analytics.

Marketers don’t need retrospective readouts; they need brand guidance systems. Instead of “tunnel vison/single brand” forecasting from historical sales and awareness levels, predict the movement of systems…and then change the future.