FLAWED MARKETS, RISKS OF SUPPLY FAILURE, AND IDEOLOGY.

Conventional wisdoms on the superiority of unfettered energy markets, and their ability to incentivise investment and deliver reliable electricity supplies, are coming under challenge as never before. The failure to deal adequately with the social costs and externalities of CO₂ emissions is one massive market failure, but even the resolution of that through carbon pricing does not address the structural flaw in many wholesale electricity markets. The policy imperatives for a low carbon economy are reinforcing many of these structural failures, but the seeds of trouble have been there for some time. The UK, in many respects the pioneer of market liberalisation, the EU which has since adopted these ideas with enthusiasm, and New Zealand, whose natural resource endowment (hydro) has allowed it to move a long way towards a low carbon power sector, present different issues, but all are forced to confront the same basic paradoxes in electricity economics. Failure to resolve these will ultimately threaten security of supplies, and the credibility of national regulatory frameworks.

Tariffs, pricing and markets underpin both efficient resource allocation and the basis for power sector investment, and have always deserved theoretical and practical analysis.  But there are two separate objectives. One is a set of market prices that incentivises investment. The other is market signals that ensure the efficient use of an existing stock of generation capacity. The fundamental dichotomy is the distinction between the short term and long term. The cost signals essential for production efficiency from existing assets relate only to short run marginal costs (SRMC), but adequate returns to investment, and to a significant degree retail tariffs, require prices that cover total costs[1] including capital costs. This is often described as long run marginal cost (LRMC).  Both objectives matter.  But it is the more limited SRMC, often equated to the short term variable costs of fuel, that has become the key to most wholesale markets, and in many ways the cornerstone on which liberalised market structures rest.

Wholesale prices based on SRMC are an outcome of the requirements for operational efficiency.  But it is intrinsic to SRMC pricing that it is not sufficient to reward investment; nor will it signal to consumers the full real costs of consumption which must include investment.  Allocative efficiency matters in the consumption of electricity as well as in production; and in the wider economy more cost reflective pricing for power will in principle reduce costs and improve economic efficiency.  For both these reasons, the conceptual basis for electricity tariffs has often been defined as LRMC, giving substantially higher prices that can cover investment and other fixed costs. The structure of consumer tariffs can however incorporate both SRMC and LRMC elements.

The UK 1990 model. When the institutional norms for the power sector changed towards liberalised market structures, the LRMC/SRMC issue was brought into sharp focus.  “Energy only” markets necessarily tend towards SRMC based outcomes, especially in periods of surplus capacity. This does not cover investment costs or incentivise new capacity investment[2].  In consequence some market designs have attempted, explicitly or implicitly, to build in features which will, at least in principle and over the long term, be capable of rewarding investment through a spot price alone. A prime example was the England and Wales pool introduced in 1990, using an administrative mechanism to define value of lost load (VOLL), and loss of load probability (LOLP), to provide prices which spiked dramatically. In principle at least this provided incentives for investment on the basis of long term price expectations.

The approach was essentially a clever attempt to reconcile SRMC and LRMC through a device which purports to act as a surrogate for the “market” in assigning a scarcity value to form part of a single “spot price”, albeit done by administrative means. However this approach proved hard to maintain in a regulatory context, partly because it implies and requires the possibility of very substantial price spikes, some of which must be expected to persist over long periods if they are to provide adequate returns on capital.  However even this model, with a single “spot” price, depended on an administrative intervention, external to the market, to set VOLL and measure LOLP. This in turn reflects a political or administrative view of the level of security or generation adequacy to which the system should operate.

Post liberalisation experience.  This central intervention was one of the features that made the 1990 model unpopular and led, in the UK, to the NETA/BETTA reforms. Implicit in the latter was the assumption that the market itself would somehow define an appropriate level of security. The outcome was neatly summarised by John Kay in the FT.

 “But privatisation failed to provide a stable framework for planning new electricity generation. The initial regime reflected careful thought about appropriate incentives for capacity installation, but this regime was swept away in 2001 in favour of a simpler one modelled on other commodity markets and known as NETA (New Electricity Trading Arrangements), subsequently to be Betta (British Electricity Trading and Transmission Arrangements). As so often in commodity markets, this structure worked rather better in the short run than over the long term.”^[3]

A return to central purchasing. Predictably the UK is now widely seen as facing very tight capacity margins and the possibility of a supply crisis. In response it has reverted to what is essentially a central purchasing regime through the introduction of a capacity market. This is an entirely rational response but it represents a major step away from the unfettered market philosophy that underpinned the original power sector reforms, and the first step to a centrally directed system. The challenge will be to ensure that this new function for government is conducted efficiently and effectively.

EU ambitions for energy only markets. The EU has generally opposed the idea of capacity markets, perhaps partly on ideological grounds, but more convincingly because national capacity markets are potentially a major barrier to a “single market” in electricity. The power sector has always been a national not an EU responsibility, so national capacity markets are a further barrier to integration. Importantly a single market that includes capacity can only make sense if there is a single security standard across the system. This would need to be set centrally and applied in all EU countries participating in the single electricity market. It seems unlikely that the German government, for example, would be happy to see such a fundamental choice made in Brussels.   

Will regulators allow price spikes? New Zealand experience.  A necessary (but not sufficient) condition for a market to be effective in inducing investment is that the political and regulatory framework can allow for major price spikes in which the only constraint on prices is the willingness of someone to pay. General experience is that this does not happen. New Zealand was brought to my attention this week, and is interesting because of the high proportion of zero marginal cost generation. As such it presents a foretaste of how this market question might play out in other jurisdictions, as the advent of low carbon technologies accentuates the gap between SRMC and LRMC, with SRMC falling to a very low [4] or zero level, while LRMC, ie the full cost of supplying power, rises. The story, for market enthusiasts, is not encouraging.

Price spikes do occur and are subject to regular complaints of an “undesirable trading situation”, allowing the regulatory authority to intervene and remedy the problem. So the natural “market” development of supply shortages, inducing higher prices to bring forward additional supply or curtail demand, is heavily constrained.

Prima facie this should make life difficult for the generation utilities. However most are vertically integrated into retail supply, and there have also been complaints about the margins prevailing in retail supply. If correct this suggests that any damage to the financial viability of generation is offset at least in part by the ability to sustain excessive margins in another part of the business, a situation that would be strongly redolent of complaints made about UK energy utilities.

Ideology. In both New Zealand and the UK, there is substantial tension between “energy only” free market enthusiasts and the development of centrally directed capacity markets. Central purchasing has entered the UK by stealth under Conservative or Conservative led coalition governments. In New Zealand the left of centre Labour and Greens proposed a central buyer model, only to be accused of Soviet-style economic vandalism.

We can expect these controversies to continue and to accelerate as the world moves further towards a low carbon, zero marginal cost world. But it is more and more evident that conventional assumptions about electricity wholesale markets are no longer “fit for purpose” and we shall in due course see further rounds of major reform.

The Oxford Martin School Programme on the Integration of Renewable Energy will be returning to these and other questions, not just for the UK but in a wider international context.

---

[1] To be wholly accurate, we should distinguish actual total costs from the concept of long run marginal cost, but for the purposes of this particular exposition the distinction need not concern us very much.

[2] This is most easy to demonstrate for peaking plant, which can only earn enough to cover its fuel cost even in the very few hours when it runs. But a similar revenue shortfall will apply to almost all plant to some degree.

[3] John Kay.  FT.  July 2013.

[4] CCS, if it is ever built, might provide an exception to this.