In this episode we explore how the ONS measures our natural environment and the green economy.     

 

Relevant datasets: ONS Environmental Accounts 

 

Transcript 

  

MILES FLETCHER  

Welcome again to Statistically Speaking, the official podcast of the UK’s Office for National Statistics. I'm Miles Fletcher and this time we're getting back to nature as we explore the work of the ONS in measuring the economic and social value of the natural environment.  

Is classical economic growth - as measured by gross domestic product or GDP - always achieved at the expense of the environment? What price can we put on the amenities our environment provides? What is the green economy and what are green jobs? And what are the key data to watch as policymakers strive for net zero carbon emissions, while also seeking to improve national prosperity?  

Our guides through the rich and perhaps under explored landscape of environmental data are ONS’s Deputy Director for Environmental Statistics Analysis, Ian Townsend; Head of Natural Capital Accounts, Gemma Thomas; and Sophie Barrand, Monetary Accounts lead in the Environmental Accounts team.  

Welcome to you all. 

Ian to come to you first. The ONS is mainly known for measuring the economy and the population of the UK. So, what exactly is its role when it comes to the environment? What are we seeking to achieve?  What do we do? What do we publish?   IAN TOWNSEND So the environment is quite a broad topic that links with a lot of other issues and a lot of different national and devolved government departments and other related bodies producing statistics on the environment. And with all that range of statistics, we tend to focus at the ONS on the intersections between our environment and both the economy and society. This includes measuring what we call the Low Carbon and renewable energy economy, how many green jobs there are, the greenhouse gas emissions produced by different economic sectors, and valuing the services that nature provides to us, as well as providing rapid insights into what people and business think about climate change in the environment and their actions or indeed otherwise.    MF And what are the major publications that come out of the ONS that people ought to be looking at to get a sense of what we're saying about the environment and its value?   IT So I mentioned a couple in the introduction there - things like low carbon and renewable energy economy, green jobs, etc... and our emissions figures. But perhaps one that is quite worth bringing to the fore is our natural capital accounts. So, it's something we've done for several years, which basically looks at the value that ecosystems provide to nature and ecosystems provide to us, and the services that provides. So, we bring this out as a report every year - have done so for several years - and that looks beyond the economy, beyond gross domestic product, to look at all those natural resources and we found that in 2021, the total value of all those natural assets was around one and a half trillion pounds. It’s such a big figure, I think it can be quite hard for people to grasp. But a useful comparison might be that it's not that far off the 1.7 trillion pounds that homes in the UK were valued at in 2021 as well.   MF It's very difficult to arrive at a financial figure or value like that. Can you just give us a brief explanation of how it’s calculated?    IT Sure. So, there are internationally agreed guidelines that we follow around how to measure or indeed account for the current value of what natural capital could provide for us and our current and future generations. And all that process, all those guidelines are aligned with how we measure the GDP in the economy. It's really quite a complex exercise and includes things like the value of trees, rivers, peatlands, and many other habitats and natural resources in them. We've been developing and improving these approaches for probably at least 10 years, and probably have some of the most developed accounts in this form globally. Our estimates have improved over the years. But there are some things that we don't cover. So, in a way, this is probably best seen as a kind of partial and minimum value, even though it's already very large. And it's also part of a wider mission that the ONS has to capture the value of what's called missing capital, things that we don't currently measure so well in gross domestic product. So that's including social capital as well as natural capital. So that's called ‘inclusive wealth’ and that's another publication the ONS produces that people might be interested to have a look at.   MF And it's important, I guess, to have this economic value of the environment so that can be compared against the traditional measure of economic progress and prosperity, which of course is GDP. And it's sometimes – and we've heard this in other podcasts - because GDP is like the big beast of the economic statistical world. It's very important, it's very influential, but of course it does have significant weaknesses and omissions, and notably its lack of account for environmental factors being notoriously one of those.   IT Sure. And actually, there's a process going on right now internationally that would bring some of that into the way that we measure some of the key economic indicators. But I think one of the key things you say is putting out there is a measure of how our natural capital assets are doing. But I think the other real benefit of these statistics, and particularly the natural capital accounts, is that it helps literally to account for nature to give an estimate of some of the benefits that the environment does provide. So, when people make decisions, they can take that into account. We're not exactly saying that nature has this given value. It's more that that's the value that we've estimated so far that provides those. It helps people to make sure that when they're taking their decisions, they take into account what would happen if we reduced or depleted that natural capital, and indeed, the benefits we might get in the future if we were to increase that natural capital as well.   MF Because – and this is the other side of the question – high per capita GDP often goes with high carbon emissions in an economy, doesn't it? This way, we can look at the other side of the balance sheet and say, well, yes, you might be achieving this high economic performance in traditional terms, but look at the cost on the other side and, as you say, this is part of a big international movement to recognize that called ‘beyond GDP’, which is a topic we've covered in previous podcasts already.   

 IT Sure.  I think, just getting back to your point around GDP and emissions one of the things that we produce in the ONS is a piece around a different emissions measurements there are, and actually if you look at those, you'll see that over time, all those measures have been reducing. So there is an element to which whilst the economy is expanding, we are actually reducing emissions on all three different measures that are available.    MF And we say the environment is - we're putting it at 1.5 trillion pounds – that's the capital value of the natural environment, broadly equivalent to the value of all the houses in the UK. Some people might say that's a low figure perhaps you can think we measure human capital being much, much greater than that but that's a debate for another time. But explain for us who is using this number, how does it inform decision making at the moment at national and local level? How might it influence policymakers in future do you think?    IT Sure, I think it's not necessarily about the big number at the top although that is one that will get a lot of interest from people and as you say, it might be an underestimate. I mentioned that there are some aspects of nature that we don't currently measure. But in terms of how it's used lots and lots of detail that's available underneath that key figure in the natural capital accounts we publish every year, at kind of macro level. The figures we use are important considerations for decision making by UK and devolved governments. So Defra published a policy paper, for example, on the accounts, I think, a couple of years back, which looked at what the key takeaways for policy there were from there, and we think that some of the figures were used in some of the bids that departments put in for the 2021 Spending Review. I think at the micro level figures are definitely used in the analysis of costs and benefits that are used for judging different government projects. It’s part of what Department for Environment, Food and Rural Affairs call enabling a natural capital approach - it is in that guidance, and that has a lot of impact at kind of micro level. We're also publishing more and more data from the natural capital accounts at a local level. And making that available means that councils and others can use that in their decision making as well. And we're also building our connections and encouraging use where we can and it's also something that's being worked on internationally as well, which we’re part of working with other national statistics offices and the UN.   MF Turning to Gemma then, you're the head of natural capital accounts at the ONS. Let's get into the nuts and bolts then of what contributes to this valuation, this figure, because by understanding that we can better understand what it contributes to the economy and what it contributes to our wellbeing. Talk us through those elements if you would.   GEMMA THOMAS Yeah, of course. So as Ian mentioned, there's some international standards that we abide by, but essentially what we do is we say, okay, what services does nature provide to us? And we identify those services, and then we attempt to put a value on them. Now those services can fall into three broad categories. The first would be provisioning services, which might be what you would expect the environment to provide and nature to provide such as things like timber, oil and gas, so it's all the goods that are provided by the Environment, water, renewable energy, and they are what we call provisioning services...    MF How we exploit the environment, to put it coldly.   GT You said that Miles, not me. [Laughing] But yes, it's where we capture the goods that the environment does provide. But the other things that nature does for us is it regulates, so regulated services are another part of services and where nature helps maintain the quality of the environment. And so a sort of obvious example of this can be where vegetation removes pollutants from the air, but also nature can mitigate noise. So if you're in an urban area, and there's lots of trees that can mitigate noise pollution, or heat, it can regulate heat. So that's another example. And the other example that is maybe not so obvious is what we call cultural services and they are the non-material benefits that we obtain, such as recreation. And actually, in many of our publications, we have found that those services provide the greatest asset value, the cultural services, actually almost two thirds in 2021 of all of that 1.5 trillion figure, 950 billion of that was from cultural services.   MF That's why I mentioned exploitation - like it or not it's the traditional economic exploitation of the environment, when we talk about those things that we take out of the environment. But what the ONS has established, certainly when it comes to the UK’s environment, it's actually the amenity value of the environment, which contributes more to our economy.    GT Yes, exactly. And actually, when you say about that exploitation, this is one of the benefits of the natural capital accounts because we measure actually the value of 16 different services at this moment in time and it allows us to be able to see things like actually, in terms of value, for example, it is more valuable to the UK to have pollution removal from woodlands than it is to obtain timber or wood fuel, and we put values on that. So yes, as you say, it's cultural and recreation. And that's in a couple of ways. There's expenditure on recreation, and tourism. But there's also, and this is the biggest asset that we've found, is health benefits from recreation that was worth in 2021, nearly half a trillion so 445 billion.   MF Crikey, that is a big number! Can you unpack that for us then, what are the health benefits that we're itemizing there?   GT Research has found, outside of the ONS, that spending time in nature has a positive impact on your health and wellbeing. But interestingly, you need to spend on average two hours a week in nature. So some of our figures, and this is the great thing that you can sort of dig under the hood a little bit, is you can see things like maybe more people are visiting nature, but if they're not doing it for long enough or regularly enough, they can't obtain that health benefit. And that is what our data has found in 2021 and 2022. There was a slight drop in the sort of health benefits gained from nature because people weren't going for long enough. In terms of measurement. We have experts in the team who are excellent and they work with lots of people across government, but essentially it's sort of calculated in the number of years of a life lived in perfect health and then what we do is we say how much would the NHS have to spend to provide the equivalent health benefits and that's how it's done. But yeah, it's just a really good example of how that sort of top figure can sound interesting but when you dig in is a real story and there of nuance.   MF I guess that also reflects the importance of when we're talking about the environment. It's not just the great outdoors, it's not the wide open spaces necessarily. It's about having environmental areas near towns and cities as well where people can get out and feel the benefits of exercise and fresh air and so forth.   GT Exactly. And actually it's really interesting you mentioned that, because some of the things that are found that can be quite difficult to explain to people is that actually nature can have more value in some situations in urban areas than in rural areas. So if you take the example of pollution, there's more pollution in urban areas and there are more people breathing in that pollution. So by having trees and woodlands, etc within them that can absorb that pollution, that means that there's a higher value in that. That's why we have to be careful in how we explain all of our figures because sometimes a higher value - not in general, not for the top level, but for some of them - is not necessarily a good thing. Because if you think about it, if there's absolutely no pollution, then trees wouldn't be removing that pollution, and the value of that would be zero. So it's a challenging thing to communicate and get across to users, yeah.    MF It's a fascinating equation, isn't it? And also in terms of outputs, of course, we can't ignore the growing influence of renewables, which of course is energy from the environment.   GT Yes, exactly. Yes. And that's a service that’s actually seen some really big growth in its asset size as we possibly would expect over recent years.  

MF 

Give us the numbers on that. 

GT 

The renewable energy was worth 40.7 billion, although I have to say - which will be surprising to some people - the oil and gas asset value is actually still higher than that at 111 billion. So, it shows that although things are changing and renewable energy has seen the biggest growth, at present oil and gas would still have a higher asset value.  

  MF We'll talk a little about the dash for net zero, or progress towards Net Zero at least, a little bit later. But nonetheless, it can't be ignored that is renewable electricity provision increased by we’re saying 275% between 2011 and 2021. The last estimates - that's 21,899 gigawatt hours. So almost a threefold increase and, you know, without being political the drift of policy is to increase that much, much more.   GT Yeah, we have. We have seen that, and you'll see that across all of our figures. Renewable is definitely a growing sector.   MF We’ve talked about what we get out in a traditional economic sense of course, but historically the biggest example has been agriculture, and farmland. There’s a big debate going on about the use of farmland and again, in classical GDP terms, the contribution of agriculture has not been great because it’s been maintained by subsidies traditionally - at the risk of upsetting the farming community. Could we say that the decline in closed farmland being reported is because less and less of our natural environment is devoted now to agriculture? Are we saying that we can move away from that as long as the other economic benefits of the environment are being delivered on the other side of the equation?   GT Well, I think with any sort of figures, you need to look at it in the round. So, you could look in isolation within the natural capital accounts, but in reality you'd need to look at, you know, employment and those sorts of things. I mean, the interesting thing about the natural capital accounts is that technically farmed land - we need to look at it in its raw state. I mean, we can't yet separate that with our figures, but it's about what nature provides us. So, for example, we wouldn't include farmed fish, but we would include wild fish catch. So, with the national capital accounts, what we aim to do is separate that out. But in terms of the amount of farmland we can record that and we can publish that and look at it, but I think for anyone making decisions, they probably need to look at the whole.   MF But certainly we're saying that woodland, for example, is contributing more to the economy in terms of its beneficial effects in offsetting carbon and removing pollution, than it actually contributes through the felling of timber.   GT Yes, yes, that is very clear from our figures.   MF And indeed there are some other takeaways from the publication as well as. In cruder financial terms, just living closer to nature can be good for the value of your house.   GT Yes, yes. So, where we don't have a price, we have to look at a way to value it. And so, we know that being close to nature or having a nice view can add value to your house. I think it added around 1.5% To the average house price in 2021, or just under 5000 pounds, 4700. It did vary though by the type of property and flats and maisonettes it added around 4%. So yes, being close to nature and accessing nature, having access to green space is really important and in fact we have publications on our urban accounts, which pull that out, and as well on our house prices, where those interested can look in more detail.    MF Of course this continues to be a big area of ‘work in progress’. What other measures could potentially be included, particularly as the response to global warming continues to be refined and developed?   GT So yes, at the moment, I believe it was mentioned earlier that we cast this almost as an underestimate. We know there are services we're not capturing. So, for example, flood regulating services. So how much nature helps in preventing floods, that is an area that we are hoping to develop, but it is quite challenging. We do look for areas where we can maybe produce more local stats to help the decisions at the local level. Because it's not just our data that's really important. It is also our methods. Our methods get used by sort of local authorities and those who maybe want to produce their own sort of natural capital accounts for their region. So, it's something we're looking to develop.   MF And I guess it's going to be vital if we're looking at an era of increased house building. Once again, we can look at the other side of the balance sheet and say, okay, this is this is going to be the environmental cost of the development before major decisions are taken.    GT Yes, essentially. It’s just about making it easier for decision makers to see that actually it does have this value, rather than before when it would be anecdotal. Now by having these figures, you can measure those up and use it in sort of cost benefit analysis. Yeah.   

[Music Break] 

 MF Well, that's a fascinating look under the bonnet of natural capital, a concept that's clearly evolving in stature, in a really fascinating and important way. Another area that the ONS is very much involved in, and a very important set of publications that the ONS produces are the environmental accounts. Sophie Barrand joins us now. Sophie, you're the lead on monetary accounts in that team. Just to start, can you explain for us what's covered in the environmental accounts; what's the publication for?   SOPHIE BARRAND Yeah, so our environmental accounts ultimately looks to measure the contribution of the environment to the economy and the impact of economic activity on the environment. So, when we're thinking about changes in an area that's constantly evolving, it's rather a tricky thing. So, in general, what we consider is how our measures work as different pieces of this net zero puzzle where each informs the other. So, our accounts produce information on UK greenhouse gas emissions, things like economic measures of low carbon activities, government revenue from environmental taxes, and things like the use of different energy types and whether we're moving to more renewable alternatives, just to name a few. So ultimately, we're looking to quantify the flows that impact on the environment through both physical and monetary ways.    MF So natural capital if you like is the stock, and in the Environmental Accounts we’re looking at the flows, we’re looking at the nature of change.   SB Absolutely, yes. So, we're looking at measures of things that can sometimes be considered as the green economy. So, if we were to try and put a value on things like the environmental goods and services sector, the services and goods that sort of flow within that space, we look to produce annual estimates of that data. And within that we can also draw from different measures from each parts of our accounts. And because we produce our data on what's called a residency basis, that is any activity as part of UK registered businesses, it means that it's aligned with our national accounts data so we can start to compare across our measures, and consider things like how does our data compare to measures such as GDP or GVA and where we sort of fit within that space. So, it's really interesting to uncover some of these trends.   MF 

And when we talk about progress towards net zero, this is where the action happens, isn't it? This is where progress towards that is tracked.   SB So within net zero, the measures that typically get used to action net zero is the territorial approach to measuring greenhouse gas emissions. So those are figures produced by DESNZ and what we do is we communicate that information on a residency basis to align with the national accounts side of things.   MF DESNZ being the government department concerned here.    SB Yes, so the Department of Energy Security and Net Zero. And what we can do with our accounts is that we can compare with regards to broader measures such as GDP and GVA, how we can capture some of those trends, but what we're ultimately seeing across all three of the measures of greenhouse gas emissions is that they are falling over time, which we can go into a bit more detail on.    MF Yeah, well let's get stuck into that detail because it's a crucial area of public policy right now. And it's surely going to continue to be over the next few years. What are the data telling us at the moment then, what are the most compelling findings around greenhouse gas emissions and carbon generally?   SB Yeah, so what we've seen is a reduction in overall greenhouse gas emissions since 1990. Though we have experienced an increase in recent years that does reflect the economy coming back from the period affected by the Coronavirus pandemic, and what we've seen is that the largest contribution to the decrease across our time series has come from energy industries such as electricity and gas supply, and what we've seen as well is the compositional shift within this space where there's been a shift away from fossil fuel use and a move towards renewable sources. So, for example, in 2022, we saw the energy from renewable sources accounted for nearly 14% of all energy use in the UK. And although that figure has been increasing, we are still seeing a large reliance on fossil fuels, and it remains at 81% in 2022.    MF You talk about the substantial reduction in emissions, what about the suggestion that is sometimes made that countries, like the UK, have effectively exported their emissions simply by closing down their heavy fossil fuel driven industries?   SB Just in terms of exporting emissions, it's a really tricky space.   

MF 

How so? 

SB 

So, for example, within our statistics, because we cover a residency basis, we do capture some parts that might sort of deviate into that space. Because we measure UK registered businesses and residents the effect of things like UK tourism and the emissions data for that is going to be captured within that space. But then for example, when we look at the aviation industry within the UK, we do capture some of that so where we have the UK registered businesses such as British Airways, activity from that space across our accounts will be captured, but things like Ryanair wouldn't be captured because I believe it’s an Irish registered business.  

MF I was going to ask you how flights are accounted for, because we're told constantly, of course, that they are a big contributor to emissions. So if you're a British airline you count towards UK emissions, it’s not flights flying over Britain or flights terminating or starting in Britain. It's a much more economic analysis, then, it's aeroplanes that are essentially British owned, is that how we do it?   SB Exactly yeah, so it's UK registered. So if they're a company that are registered as being a UK business, then it will capture that activity, but there's no sort of physical boundaries that you might experience with other measures. It's more of, as you say, an economic term where it's more about the registration of the business.   MF Well that does say something about what a global issue this is really. If it's not measured in that local way, it's an issue for the world to address collectively.   SB Absolutely. And by combining those international comparisons, it should give you that full picture of exactly what's happening there. We can do that across different measures of our accounts as well. And for example, within the environmental taxation space, we put our taxation data in the context of the sort of broader international space where we compare against EU countries for example, so with all of our measures, what we're really looking to do is to sort of piece together this net zero puzzle, and each of our measures look to inform the other, and it is not without that broad picture that you can actually start to understand some of these.   MF Certainly where we are now, it's households that are the biggest emitters of greenhouse gases, and previously of course it used to be the industrial sector. So it explains the policy focus doesn’t it on households and vehicles as well, because those are the big emitters of greenhouse gases as far as the UK is now concerned.   SB Yes. So you touched on households there, as you say they now emit more greenhouse gas emissions than any other economic sector. And while households do tend to have limited influence over emissions related to things like heating their homes, we have seen that half of all household emissions do come from travel. And as you can imagine, we saw dips in these in the Coronavirus period. Where the biggest emitters used to be energy and manufacturing there has been a switch from oil to gas and things around improved processes. And the more recent increase that we've seen in renewables has seen big declines in greenhouse gas emissions from these two sectors in particular.   MF Have we seen any lasting impacts from the pandemic or have things just reverted back to exactly as they were before?   SB So it depends on which measure you're looking at as to how continuous the impact of the Coronavirus pandemic was with our statistics. So for example, with our environmental taxes, we saw quite a big dip there after a relatively stable annual increase. Since then it's pretty much bounced back to what it was in 2019 and we’ll look to future years to see whether this continues. Transport is now responsible for 16% of all UK greenhouse gas emissions and because of its increased importance in our stats when all transport stops during the pandemic greenhouse gas emissions did fall dramatically by about 11% or so, and with the recovering of transport we have seen the greenhouse gas emissions have increased in recent years.    MF So to that extent people have reverted to their same old pre-pandemic ways, which raises a whole question of how policymakers can influence behaviors and of course the classic way for policymakers to stop people doing things is to tax them. And the effectiveness, or otherwise, of environmental taxes Sophie is something we also look into, because that's a very important factor in all this. Tell us what we’ve found about all that.   SB The intention of those is to create these behavioural shifts and allow people to be making those decisions to move towards more environmentally friendly alternatives. So what we found is that through this taxation we have raised 52 and a half billion pounds in 2023 in the UK, with fuel duty being the single largest tax contributor to this at nearly half the amount. This coincides quite nicely with what we're seeing on the greenhouse gas emissions data where consumer expenditure from car transport is contributing largely to this. In the international space, when we look at our environmental tax figure as a percentage of GDP, this now amounts to 2% of GDP. So that sits a bit lower than the EU average, which currently sits at 2.3%.    MF What other environmental taxes are there then that are included, which again, might be varied or even increased in the years to come?   SB That's a very good question. So we're constantly considering how our statistics might evolve, and how our definitions might evolve. Some examples about environmental taxes include things like energy taxes, so things like your fuel duty and taxes on the type of energy sources that you're using. We then also have things like transport taxes. So these are air passenger duty, and things like that. But then we also have what we call pollution and resource taxes. So one of the most recently introduced ones of those is plastic packaging tax. So we do split up the different tax types in our release.   MF And again, on a local level, there's the whole question of how effective low emission zones are, and the crackdown on old diesel cars and so forth. Is that local picture available if you're a local authority seeking to investigate the effectiveness of measures like that?   SB So we don't currently disaggregate our data into below UK level unfortunately, but we are constantly discussing these things and I believe there's a discussion this year around the low emission zones and how we classify these. We generally follow the national tax list when it comes to categorizing our different taxes. So we're regularly reviewing these and considering which could be considered environmental and which wouldn’t.   MF Interesting. Finally then with you Sophie, the whole question of jobs and employment, because if you look at traditional economic measures you want to have lots of people in productive industries making things, creating economic value. When the environmental factor comes in of course, you can see reasons why it's not such a great idea to have lots of people working in high emitting industries, and that’s where the whole concept of green jobs - and on the other side of the fence, non-green jobs – is something the ONS has been developing its work around as well.   SB Yes, absolutely. So in looking at employment, which is an area of the economy with high user interest, particularly with regards to the green job space, we can look at things like jobs in high emitting industries as an example. So what we've seen is that five industry sections have accounted for around 82% of total industry greenhouse gas emissions across the UK, and of these five industries they employed around 16%. So that's one in every six or so UK employees.   MF Now there are some quite substantial variations around the UK regionally, when it comes to the numbers of people employed in these non-green jobs, aren’t there?   SB Yeah, so what we're seeing is that one in five Midlands workers were in high emissions industries in this time compared with one in 12 workers in London.   MF And Gemma then, this idea of a green job and a non-green job, how do we define those two things?   GT Well Miles, it's really, really challenging, and it's something that ONS has been working on for a long time and it's been a real leader in the field. You’d think, “Oh, green job”, and that term is used a lot. But in reality, it can mean different things to different people. ONS has done a lot of work over the years like with the low carbon economy, measuring that where that is looking at things in the low carbon and renewable energy space. But we realised that people wanted more and they wanted this definition and after a lot of work and discussion, we have agreed a definition where it's as: ‘Employment and activity that contributes to protecting or restoring the environment, including those that mitigate or adapt to climate change.’ And I think the key thing is it's wider than net zero. It's also about nature. It's also about jobs that are protecting nature, not just those that are helping to reduce emissions. And then in terms of non-green or brown as it's sometimes called, that is a challenge. And that's where Sophie has talked about, we look at high emitting industries. Because in reality you need to be practical, and those are the industries that maybe we'll see the most change - the industries that have high emissions. The real challenge, though, is that our current classification systems, you know, some of those green jobs will occur in those high emission industries. You think of things like the energy sector, not only does it have oil and gas jobs, but it also has renewable energies, and our way that we currently do economic statistics doesn't allow us to break that down. And that's the big challenge and that's what people are grappling with including us at the ONS.   MF Thank you Gemma. And Ian then, finally from you. This whole question of nailing down precisely what is a green job, what is the green economy - as we've heard from Gemma, it's very dynamic and there is a lot of work going on in this area. But where is this all going internationally? How are we collaborating with other statistical organisations around the world on this, and is there anything we can learn from other countries?   IT I think in many ways we're near the front of the pack on this internationally. So a lot of the work we've been doing is about sharing our experiences, and in the kind of broader space of environment measurement we're very involved with the UN in particular, particularly on natural capital accounts. And we're also looking at some major revisions to some of the systems that underlie how we do these things. So for example, there's a big shift internationally in the way that we measure GDP and there's an element of that which is around the environment, so we're engaged with that too. So quite a lot going on in that space. We also have regular bilaterals with other national stats institutes as well, to share learning and understand what other people are doing. And obviously, the more organisations doing this work, the more mutual learning we can benefit from as well.   MF And how do you see things developing from here on, in terms of ONS’s measurement of the environment?   IT I think we’ve been able to do quite a lot over the last few years, building on great work over a much longer period. And there's a lot that we could be doing- of course the challenge is always about channeling the resources available to the highest impact activity that we can be doing. I think one of the things we're going to be looking at is to move to accredited ‘official statistics’ status for the natural capital accounts. It might sound a bit inward looking but it's a real mark of quality that we'd like to have, and it would set it alongside economic statistics that have that kind of accreditation. We're also reflecting on, of course, the new government's priorities following the general election. One of the things that we've been experiencing quite a lot of demand for, and particularly from local and combined authorities across the country is around greater granularity for our environmental data - so data at Local Authority / Combined Authority levels. So that might be in the mix. It’s something we've been thinking about and gradually releasing more for. We’ll also continue to be working across the ONS and across the wider government statistics service on the topic as well, so do watch out for collaborations in that space. And of course, as we've just talked about, the international aspects as well. And of course, we always welcome interest and feedback in our work, so if people are interested, please do get in touch.   MF I’m sure they will. And it’s worth pointing out to our listeners that once a quarter, the ONS publishes key statistics on Wellbeing and Beyond GDP, which can help people understand the environment in the context of the economy. And I think on the same day you can also check out the latest quarterly greenhouse gas emissions data.   That’s it for another episode of ‘Statistically Speaking’. Thanks to Gemma Thomas, Sophie Barrand and Ian Townsend for joining us. And of course as ever thanks to you for listening. 

You can subscribe to future episodes of this podcast on Spotify, Apple podcasts and all the other major podcast platforms.   

You can also follow us on ‘X’ - previously known as Twitter - via the @ONSfocus feed.    

I'm Miles Fletcher and from myself, and our producer Steve Milne... goodbye.   ENDS 

With the public release of large language models like Chat GPT putting Artificial Intelligence (AI) firmly on our radar, this episode explores what benefits this technology might hold for statistics and analysis, as well as policymaking and public services. 

Joining host, Miles Fletcher, to discuss the groundbreaking work being done in this area by the Office for National Statistics (ONS) and across the wider UK Government scene are: Osama Rahman, Director of the ONS Data Science Campus; Richard Campbell, Head of Reproducible Data Science and Analysis; and Sam Rose, Deputy Director of Advanced Analytics and Head of Data Science and AI at the Department for Transport. 

Transcript

MILES FLETCHER

Welcome again to Statistically Speaking, the official podcast of the UK’s Office for National Statistics. I'm Miles Fletcher and, if you've been a regular listener to these podcasts, you'll have heard plenty of the natural intelligence displayed by my ONS colleagues. This time though, we're looking into the artificial stuff. We'll discuss the work being done by the ONS to take advantage of this great technological leap forward; what's going on with AI across the wider UK Government scene; and also talk about the importance of making sure every use of AI is carried out safely and responsibly. Guiding us through that are my ONS colleagues - with some of the most impressive job titles we've had to date - Osama Rahman is Director of the Data Science Campus. Richard Campbell is Head of Reproducible Data Science and Analysis. And completing our lineup, Sam Rose, Deputy Director of Advanced Analytics and head of data science and AI at the Department for Transport. Welcome to you all. Osama let's kick off then with some clarity on this AI thing. It's become the big phrase of our time now of course but when it comes to artificial intelligence and public data, what precisely are we talking about? OSAMA RAHMANSo artificial intelligence quite simply is the simulation of human intelligence processes by computing systems, and the simulation is the important bit, I think. Actually, people talk about data science, and they talk about machine learning - there's no clear-cut boundaries between these things, and there's a lot of overlap. So, you think about data science. It's the study of data to extract meaningful insights. It's multidisciplinary – maths, stats, computer programming, domain expertise, and you analyse large amounts of data to ask and answer questions. And then you think about machine learning. So that focuses on the development of computer algorithms that improve automatically through experience and by the use of data. So, in other words, machine learning enables computers to learn from data and make decisions or predictions without explicitly being programmed to do so. So, if you think about some of the stuff we do at the ONS, it's very important to be able to take a job and match it to an industrial classification - so that was a manually intensive process and now we use a lot of machine learning to guide that. So, machine learning is essentially a form of AI. MILES FLETCHERSo is it fair to say then that the reason, or one of the main reasons, people are talking so much about AI now is because of the public release of these large language models? The chat bots if you like, to simpletons like me, the ChatGPT’s and so forth. You know, they seem like glorified search engines or Oracles - you ask them a question and they tell you everything you need to know.  OSAMA RAHMANSo that's a form of AI and the one everyone's interested in. But it's not the only form – like I said machine learning, some other applications in data science, where we try in government, you know, in trying to detect fraud and error. So, it's all interlinked.   MILES FLETCHERWhen the ONS asked people recently for one of its own surveys, about how aware the public are about artificial intelligence, 42% of people said they used it in their home recently. What sort of things would people be using it for in the home? What are these everyday applications of AI and I mean, is this artificial intelligence strictly speaking?  OSAMA RAHMANIf you use Spotify, or Amazon music or YouTube music, they get data on what music you listen to, and they match that with people who've been listening to similar music, and they make recommendations for you. And that's one of the ways people find out about new music or new movies if you use Netflix, so that's one pretty basic application, that I think a lot of people are using in the home.  MILES FLETCHERAnd when asked about what areas of AI they'd like to know more about, more than four in 10 adults reported that they'd like to know better how to judge the accuracy of information. I guess this is where the ONS might come in. Rich then, if I could just ask you to explain what we've been up to, what the Data Science Campus has been up to, to actually bring the power of artificial intelligence to our statistics. RICHARD CAMPBELLThanks Miles. Yeah, a few things that ONS has been doing in this very broad sphere of artificial intelligence, and it's really in that overlap area that Osama mentioned with data science, so I'd pick out a few sorts of general areas there. So, one is automation. You know, we're always keen to look at how we can automate processes and make them more efficient. It frees up the time of our analysts to conduct more work. It means that we are more cost effective. It means that our statistics have better quality. It's something we've done for years but AI offers some new opportunities do that. The other area which Osama touched on is the use of large language models, you know, we can get into the complexities of data. We can get much more out of data; we can complete tasks that would have been too complex or too time consuming for real data scientists. And this is good news, actually, because it frees up the data scientists to add real valuable human insights. Some of the places we've been using this. So, my team for example, which is called reproducible data science and analysis, and we use data science and engineering skills to develop computer systems to produce statistics where the data is a bit big, or what I tend to call a bit messy or a bit complex for our traditional computer systems. We use AI here through automation, as I mentioned, you know, really making sure that we're making systems as efficient and high quality as possible. Another thing we're interested in doing here is quite often we’re doing something called re-platforming systems. So, this is where we take a system that's been used to produce our statistics for years and years and look to move it on to new technology. Now we're exploring with Osama's team the potential for AI to do a lot of the grunt work for us there to sort of go in and say, right, what is going on in this system? How is it working, how we can improve it? One other thing I'll mention, if Osama doesn't mind me treading on the territory of his team, is the Stats Chat function that we've used on the ONS website. So, this is using AI to enable a far more intelligent interrogation of the vast range of statistics that we've got, so it no longer requires people to be really knowledgeable about our statistics. It enables them to ask quite open questions and to be guided to the most relevant data.  MILES FLETCHERBecause at the moment, if you want to really explore a topic by getting into the depths of the data, into the granular data, you’ve really got to know what you're looking for haven’t you? This again is an oracle that will come up with the answers for you and just present them all ready for your digestion.  RICHARD CAMPBELLThat's right. And I tend to think of these things as a starting point, rather than the whole answer. So, what it’s enabling you to do is to get to the meat of the issue a lot quicker. And then you can focus your energy as a user of our statistics in doing the analysis that you want rather than thinking “how do I find the right information in the first place?” MILES FLETCHEROsama, that sounds like an intriguing tool. Tell us precisely how it works then, what data does it capture, what's in scope? OSAMA RAHMANSo the scope is publicly available documents on the ONS website. And there's a specific reason for that. So, these AI tools, you can have it look at the whole internet, you can have it look at subsets of data, you can point it to specific bits of data, right? And what's important for us is actually the work of the ONS, that statistics we produce are quality assured and relevant. And by providing these guardrails where you know, Stats Chat only looks at ONS published data, we have a degree of assurance that the data coming back to the user is likely to be of good quality and not based on who knows what information. MILES FLETCHERBecause when you use, to name one example, ChatGPT for example, the little warning comes back saying “ChatGPT can make mistakes, consider checking important information.” And I guess that's fundamental to all this isn't it. These tools, as intelligent as they might be, they're only as good - like any system - as the information that's going in the front end.  OSAMA RAHMANThat's absolutely correct, which is why we have these guardrails where, you know, the functionality on Stats Chat is focused on published ONS information.  MILES FLETCHERThat does mean that something that's offered by an organisation like the ONS does have that sort of inbuilt potential to be trustworthy and widely used. But of course, you might say, to have a really good tool it's got to be drawing on masses of information from right across the world. And it's interesting how, and you mentioned that it's open-source data, of course, that's most available for these tools at the moment, but you're seeing proprietary data coming in as well. And this week, as we're recording this, the Financial Times, for example, has announced that it's done a deal with one of the big AI firms to put all of its content into their database. Do you think there's scope for organisations like the ONS around the world to collaborate on this and to provide you know, really powerful tools for the world to exchange knowledge and data this way? OSAMA RAHMANSo there is collaboration going on. There's collaboration, both within government - we're not the only department looking at these sorts of tools; there's also collaboration internationally. I think the difference you know... our information on our website is already publicly available. That's why it's on the net, it is a publication. But there's a difference in situation with the FT where, you know, a lot of the FT information is behind a paywall. MILES FLETCHERYeah, it has a sort of democratising tendency that this publicly available information is being fed into these kinds of sources and these kinds of tools. That's big picture stuff. It's all very exciting work that's going on. But I'll come back to you Rich just for a second. What examples practically, because I think that the Stats Chat project is still a little way off actually being available publicly, isn't it? RICHARD CAMPBELLYeah, I think it is still a little way off. So, I think the key thing that we're doing at the moment and something we've done for years, but AI is helping is the use of automation principles. Just making things quicker. Now in a data science context, this might be going through very, very large data sets, looking for patterns that it would take an analyst a huge amount of time and probably far too much patience than they would have to find. MILES FLETCHERSo for example, in future then we might find that - and this is one issue that recurs in these podcasts - obviously about the limitations of official statistics is they tend to lag.  This is another way of making sure that data gets processed faster. And therefore, the statistics are more timely, and therefore the insights they provide are really much more actionable than perhaps they might be at the moment.  RICHARD CAMPBELLYeah, that's spot on. There's potential in there for pace of getting the statistics from the point that the data exists to getting it into published statistics. There's potential there for us to be able to combine and bring more sources together. There's also some behind the scenes stuff that helps as well. So, for example, quite often we are coding up the systems to produce new or improved versions of official statistics. And we're looking at the possibility of AI speeding up and supporting that process, perhaps for example, by giving us an initial draft of the code. Now, why does that matter for people in the public, you know, does anybody actually care? Well, what it means is that we can do things quicker and more to the point we can focus the time of our expert data scientists and other analysts in really helping people understand the data and the analysis that we're producing.  MILES FLETCHEROkay, so lots of interesting stuff in the pipeline there. But I’d like to bring in Sam now to talk about how AI is actually being used in government right now. Because in your work Sam at the Department for Transport, you've actually been working on some practical projects that have been gaining results in the real world. SAM ROSEWe have - we've been doing loads actually, and my poor team probably haven't had any time to sit still for the last 18 months or so. And I think like most ministerial departments, we're doing lots and lots of work to automate existing processes, so much like Rich has alluded to in your space, we're looking at the things that take up most of the time for our policy colleagues and looking at how we can automate those. So, for example, drafting correspondence, or automating policy consultation processes, or all of that kind of corporate memory type stuff. Can we mine big banks of data be it text or otherwise and summarise that information or generate new insights that we wouldn't have been able to do previously? But I think slightly more relevant maybe for you guys, is the stuff we're doing on creating new datasets or improving datasets. So, a few things. We're training a machine learning model to identify heavy goods vehicles from Earth observation data. And that's because we don't have a single nationally representative data set that tells us where these heavy goods vehicles park or stop outside of existing kind of service stations, and what we want to understand is where are those big areas of tarmac or concrete where they're all parking up as part of their routine journeys, so that we can look at when we're rolling out the green infrastructure for heavy goods vehicles, we're looking at where the important places that we need to put that infrastructure are. And that data doesn't exist at the moment. So we're using machine learning to generate a new dataset that we wouldn't otherwise have. MILES FLETCHERAnd how widespread are these kinds of projects across government in the UK now? SAM ROSESo I think that there are loads of different things and I wouldn't be able to speak on behalf of everybody but I know lots of different areas of government are looking at similar kind of automation and productivity projects like our kind of drafting all of the knowledge management area. I think there's things like Osama alluded to where DEFRA for example, I think they're using Earth observation data to assess biodiversity for example. So, there's lots of stuff that's common between lots of government departments, and then there's lots of stuff that's very specific to individual departments. But all along the way there's lots of collaboration and working together to make sure we're all learning continuously and where we can collaborate on a single solution that we are. MILES FLETCHERI guess one of the central public concerns about the spread of AI once again that it will cost jobs, that it will do people out of the means of making a living that they've become used to. And I guess from government's point of view, it's all about doing much, much more with the resources that we have and making government much more effective.  SAM ROSEYes, absolutely. And it's not necessarily - and I think Rich mentioned this earlier - it's not necessarily about doing our jobs for us. It's about improving how we can do our jobs and being able to do more with less, I think, so freeing up the human to do the bit that the human really needs to do and enabling the technology to do their very repeatable very automatable parts of the job. And indeed, in some instances, this technology can actually do the work better than humans. So be it identifying really complex patterns and datasets, for example. Or a good example from us in transport is we've trained machine learning model to be able to look at images of electric vehicle charge point installations and be able to identify that similar or the same image that has been submitted more than once. Now that's estimated to have saved over 130 man years of time, you know, that's not a task that we would have been able to do with just humans. MILES FLETCHERAnd you would have to be pretty alert as a human and have a very high boredom threshold to process all that material yourself and spot the fraudsters. SAM ROSEYeah, well, quite. And that's, I think, a really nice example of where again, it's not taking our jobs, but it's enabling us to do something that we wouldn't have been able to do previously and improve the service that we're providing. MILES FLETCHERNow, our ability collectively, whatever sort of organisation we're involved in, our ability to make the most of AI depends on of course having the right skills, and Osama I guess this is where the Data Science Campus comes in as the government's Centre of Excellence for data science, principally, but I guess also in this context, artificial intelligence as well. What work have you been involved in to make sure that the supply of those skills and knowledge is on tap for government? OSAMA RAHMANSo firstly, I would say we are a (one) centre of excellence within government. I think you know, what's been brilliant to see since the campus was set up has been that actually more and more government departments have excellent data science, AI teams. Sam leads one at DfT. There is, of course, 10DS (or 10 Data Science) at number 10 [Downing Street]. There's a Cabinet Office team. So, there's lots of teams that now work in this area. Some of the stuff we've been doing is we have various training programmes that we have run. We have senior data masterclasses so that actually, senior leaders within government can understand better the power of data. 10DS, Sam's area, have all been running hackathons, which actually improve skills as well. So, it's no longer just us who are building capability. I think it's great to see that across government and across departments there are teams improving skills within their departments, bringing in others from outside to work with them. So, there's a lot going on there. SAM ROSEJust really quickly, it's important to think that skills are not just skills of data scientists, but skills of everybody's ability to use this kind of technology. There's a lot of work going on at the moment looking at what we need to do both internally to government, but also out there in all of our sectors to make sure that our workforce has the skills it needs to be able to more rapidly kind of adopt and be able to take advantage of all the benefits that this technology brings to us. I mean from a very personal point of view, and I don't really know all of the answers to this, but you know, I'm thinking about what actually, if large language models can help us to generate efficient code, then actually, what skills do I need in my data scientists? If it's not writing code, is it actually the analytical thinking and being able to understand how to apply these kinds of technologies? So, I think it changes what we need in the workforce that we have.  MILES FLETCHERInevitably, though, if we're talking about this kind of technology being rolled out across government and thereby increasing the power of government to know more about more people, then concerns obviously, about the ethical use of data come in...  RICHARD CAMPBELLMaybe if I can just come in on that one Miles. Using data safely and responsibly - it's built into our very DNA in ONS and across government. And our keenness to sort of learn how to do new tools new techniques is always going to be tempered by our need to ensure that we are responsibly using the data that's been entrusted to us. And I think we need to sort of strike a balance here. We need to ensure that we don't take this responsibility as an excuse to not try and adopt new technology such as AI, but it also means we have to do so with care and responsibility and to do it at an appropriate pace. The key thing, I think, for me is ensuring that we can retain control of the data that we've been entrusted with. And so, understanding what AI is doing with that data, considering what data we're giving access to it, what data is being processed, and what data is being generated. And this is really at the forefront of our minds and our collective use of this. I think our approach - and Osama touched on this earlier - is to sort of be novel and start with open source and non-sensitive data first, so that will help us learn how we can effectively use it before we go on to some of the more sensitive data that we hold. SAM ROSEWe have to have ethics and data protection at the heart of everything we do, which then does have the tendency I think necessarily to reduce the pace of our ability to roll things out a little bit. But as government we do, I think have more responsibility. We can't have those kind of oops moments that some of the big tech companies have had when they're trying to reverse engineer the data to remove bias and that you know, things like that that then fundamentally undermine the output of their models. I think when you're doing a job that affects individual people, and providing services that affect citizens then we don't really have the luxury of getting it wrong like that, and we have to try to make sure we get it right first time. So, all of the things that Richard said about starting with, you know, safer datasets and working our way up before we deploy these models is kind of fundamental to how we're going to learn and ensure that we're doing it safely and securely MILES FLETCHEROsama what's your take on the ethics question?  OSAMA RAHMANFirst of all, I would echo everything just said. You know the Statistics Code of Practice is an annex to the Civil Service Code, it applies to all of us not just statisticians - I'll point that out. It is I think, not just in the ONS, I think for analysts and data scientists and specialists across government, this is kind of built into their DNA. Central Digital and Data Office has put together guidance and circulated it across government on the safe use of AI within government. So, within government, we do take this quite seriously. And then actually in terms of the use of some of these techniques, I think pointing these tools at data and information that we know is accurate is an important starting point - so having those guardrails. If it's going to be used for decision making, then having a human in the loop is quite important to make sure that the use is ethical. So, there's a bunch of safety checks that we do put in which I think allow for us to have some assurance that the use of these tools will be safe and ethical.  RICHARD CAMPBELLI think just as one additional point is you know; this isn't a new challenge for us. It's a different flavour of a challenge that we faced in considering new technology in the past. So, we can think in fairly recent times the use of cloud technology to securely and safely store data. If we go further back the use of the Internet, go back further, again, the use of computers to hold data. And what I think we've demonstrated time and time again, is that we do approach these things responsibly and maturely. But we do find opportunities to use all of them to improve the quality of statistics and analysis and the service that we offer the public. MILES FLETCHERLooking to the future then, and this is a very fast-moving future of course, I'd like to get your takes on also where you see us in five years’ time in 10 years’ time with this. I mean starting with the Office for National Statistics – Osama and Rich particularly on this. How will we start to see the published statistics and the big key topics, but also the granular insights that we provide on all kinds of areas. How will we see that changing and developing do you think? Where are you going to put your money? RICHARD CAMPBELLI think predicting the future in this way is quite a dangerous game. I’m thinking back to you know, if we had this podcast in the year 2000 and we asked ‘’how would the internet form part of our working lives?’ We would have predicted something which would have been quite different from the impact that it had. Saying all that I think it will make a fundamental difference to the way that we work. I see that it will be integrated in the day-to-day tasks that we do in a similar way that we used computers to speed up and change the way that we produced statistics. I think it will enable our users to far better interact and engage with data and analysis. So, it will be less of us producing a specific finalised product for them, and more for them to be able to sort of get in ask questions, probe and really, really interact. And I think lastly, it will give us more potential to work and analyse data because one thing, and I think this is really important to say, AI will give more opportunities for analysts. It won’t take them away. It will give them more space, more tools to work with to produce better, more complex, more useful datasets and analysis for ONS and for its users. SAM ROSEI was just going to add that I think it will fundamentally change the nature of what we do. A little bit like Rich said, the sort of work that we do will be different, but really critically, I think in a few years’ time we won't really notice that change. I was thinking that most people have forgotten that 10 or more years ago before you left the house to go somewhere new, you would have consulted your map. Whereas actually nobody, or very few people, do that anymore. So, I think we're going to forget very quickly that lots of what we will be doing will be AI driven. MILES FLETCHERSo it's a big evolutionary step forward, if not quite a revolution. Do you agree with that Osama? OSAMA RAHMANAbsolutely, because some of us have actually been using sort of transformers-based models, which is what these large language models are based on for... My team has been working with those for at least the last eight years. But I wanted to just pick up on what Rich just said. And it is an evolution right. And you can't separate the tools from the data. And one of the things we're getting now is data that is much more granular and of much higher velocity than the data we were used to. So that allows us to look at things at a more local level, at a more timely level. What I do completely agree with Rich on is actually a lot of these tools and methodologies allow the technical production of statistics to get more efficient, which then allows you to produce more statistics at a disaggregated level - at a regional level or local authority area level or looking at different sub populations. It allows us to update statistics more frequently. But then also what it allows us to do, because it's not just about the production of the statistics, it's about what those statistics actually tell you is going on. And I think it allows the people we have at the ONS and other government departments to spend more time on the real value added which is “what does this mean?”  MILES FLETCHERIt's interesting if you're researching a particular topic, it must be good to sort of evolve your methodology quickly and to refine your processes on the run as it were to explore a particular topic. One thing of course we need in statistics is consistency of methodology and approach. Does that limit do you think, either of you, the ability for statistics to get more insightful to get more germane to issues because we have to stick to accepted methodologies to provide that consistency over the long run? RICHARD CAMPBELLI don't think it does Miles. I mean, you're right there. There's always a challenge for us in that, that consistency is really important, that comparability in a time series. Equally, users do want us to look for improvements, more detail, whether that's granularity or whatever else. And actually, we've got a really good successful track record of both maintaining the consistency of our statistics, while at the same time introducing new and improved methods. We do it with GDP. We do it with inflation, we do it with population, it's something that we do time and time again. And, actually, I think automation AI offers up some really exciting opportunities here in terms of methods that can be applied. There's actually an element of it, which will help us in the understanding and documentation and consistent application of the methods as well. It’s perhaps one of the less – if you don’t mind me using the word - “sexy” applications of AI but using it to ensure that our documentation is absolutely spot on and done quickly. To ensure that we are applying methods really quickly and consistently. I think AI offers us potential to do that even better. MILES FLETCHER“Sexy” in the particular way that we refer to progress in data science.

RICHARD CAMPBELL

Yes, quite. OSAMA RAHMANCan I just come in on this? And it's possibly worth using a specific example of putting out statistics on prices. In the old days you’d basically have people going out into the field, and that's where you'd find a basket of goods, and using pen and paper would collect prices. Now where a lot of national statistical organisations are going to is actually getting scanner data, because most things when you pay for them nowadays in many parts of the world, it's scanned first, electronically rather than rung up through a cash register of some sort. So, scanner data provides a lot of information about what is being purchased, and at what price it's being sold at various retail outlets. And so, you have this data which again is much more granular and has much higher velocity then price data you can collect through surveys, and you know, how you integrate that into the production of pricing statistics and other economic statistics is really, you know, a really interesting question and work that a lot of national statistical organisations are working on. So, there's still the basic methodology remains the same. It's you know, kind of defined a basket of goods, but you expand the scale of the basket, we'll get prices on at what each of the elements are those baskets are being sold at, and then produce a price measure an inflation measure, right. But these tools and the increasing quantity of data allow us to do that. But you know, the basic methodology is kind of the same, but actually the increase in this data allows us to do that in kind of a different way. It's an evolution. MILES FLETCHERIt does all suggest though, that perhaps the survey might finally be replaced - the big social surveys that the ONS runs. Do you think that the surveys days are numbered, therefore because of AI? 

RICHARD CAMPBELL

No.

OSAMA RAHMAN

No.

[LAUGHTER] MILES FLETCHER

A resounding no.

RICHARD CAMPBELLThat was a resounding no, and it's not a pre-rehearsed one. And maybe I'll just take us back Miles. So, if we went back about the best part of 10 years, everyone was talking about big data. You know, the days of a survey was gone. What we needed was these big, complex, sometimes quite messy data sources that were collected for a variety of other reasons, and that we could utilise those to sort of answer all of the statistical questions that we had. Now, what we found out actually is that yes, these data sources can give us a lot of potential; data science is helping us make the most of them; AI is helping us make even more from them. What we also learned though, is that they work best when they're complementing the surveys, rather than trying to replace. Think of it a bit as horses for courses. Actually, though, I want to give an example of where AI might be able to help us improve the response rates on surveys. So, AI might be able to help respondents navigate through some of the surveys, helping them understand what it is that they're being asked. Helping them answer a bit more efficiently. So that might actually remove a barrier that some people, some businesses have to respond to surveys. So, you never know we might see a bit of an uptick in response rates with a bit of AI’s help.  OSAMA RAHMANAnd I think the other thing I would add is what surveys are particularly good at is getting information on the extremes of the distribution. It's great if you think everything's going to be generated through digital footprints data and online services, but actually not everyone... some people... apparently dumb phones are coming back into fashion. Or there are groups that you know, for whatever reason, are not picked up in other forms of data. And actually, surveys are really important for accessing, getting information about hard to reach groups at the end of the distribution. MILES FLETCHERI think that’s kind of reassuring that for all the promise of AI in this brave new world, that we hope won't be a dystopian future, but whether it will deliver all those things that we've been talking about in terms of better insights, faster statistics, and all that. It's still good to hear though, isn't it, that there is no substitution from speaking to real human beings directly.? OSAMA RAHMANI agree entirely. MILES FLETCHERWell, that's it for another episode of Statistically Speaking and, in summary, I suppose the use of AI feels like a natural evolution with a number of potential benefits, and potentially huge benefits, but with its adoption we need as always to be thoughtful and ethical. So, thanks to all our guests: Sam Rose, Osama Rahman, and Rich Campbell, and of course, thanks to you as always for listening. You can subscribe to future episodes of this podcast on Spotify, Apple podcasts and all the other major podcast platforms. You can also follow us on X - formerly known as Twitter - via the @ONSfocus feed. I’m Miles Fletcher and from myself and producer Steve Milne. Until next time, goodbye. 

ENDS

The ONS podcast returns, this time looking at the importance of communicating uncertainty in statistics. Joining host Miles Fletcher to discuss is Sir Robert Chote, Chair of the UKSA; Dr Craig McLaren, of the ONS; and Professor Mairi Spowage, director of the Fraser of Allander Institute. 

 

Transcript 

MILES FLETCHER 

Welcome back to Statistically Speaking, the official podcast of the UK’s Office for National Statistics. I'm Miles Fletcher and to kick off this brand new season we're going to venture boldly into the world of uncertainty. Now, it is of course the case that nearly all important statistics are in fact estimates. They may be based on huge datasets calculated with the most robust methodologies, but at the end of the day they are statistical judgments subject to some degree of uncertainty. So, how should statisticians best communicate that uncertainty while still maintaining trust in the statistics themselves? It's a hot topic right now and to help us understand it, we have another cast of key players. I'm joined by the chair of the UK Statistics Authority Sir Robert Chote, Dr. Craig McLaren, head of national accounts and GDP here at the ONS, and from Scotland by Professor Mairi Spowage, director of the renowned Fraser of Allander Institute at the University of Strathclyde. Welcome to you all.  

Well, Sir Robert, somebody once famously said that decimal points in GDP is an economist’s way of showing they've got a sense of humour. And well, that's quite amusing - particularly if you're not an economist - there's an important truth in there isn't there? When we say GDP has gone up by 0.6%. We really mean that's our best estimate. 

  SIR ROBERT CHOTE It is. I mean, I've come at this having been a consumer of economic statistics for 30 years in different ways. I started out as a journalist on the Independent and the Financial Times writing about the new numbers as they were published each day, and then I had 10 years using them as an economic and fiscal forecaster. So I come at this very much from the spirit of a consumer and am now obviously delighted to be working with producers as well. And you're always I think, conscious in those roles of the uncertainty that lies around particular economic estimates. Now, there are some numbers that are published, they are published once, and you are conscious that that's the number that stays there. But there is uncertainty about how accurately that is reflecting the real world position and that's naturally the case. You then have the world of in particular, the national accounts, which are numbers, where you have initial estimates that the producer returns to and updates as the information sets that you have available to draw your conclusions develops over time. And it's very important to remember on the national accounts that that's not a bug, that's a feature of the system. And what you're trying to do is to measure a very complicated set of transactions you're trying to do in three ways, measuring what the economy produces, measuring incomes, measuring expenditure. You do that in different ways with information that flows in at different times. So it's a complex task and necessarily the picture evolves. So I think from the perspective of a user, it's important to be aware of the uncertainty and it's important when you're presenting and publishing statistics to help people engage with that, because if you are making decisions based on statistics, if you're simply trying to gain an understanding of what's going on in the economy or society, generally speaking you shouldn't be betting the farm on the assumption that any particular number is, as you say, going to be right to decimal places. And the more that producers can do to help people engage with that in an informed and intelligent way, and therefore mean that decisions that people take on the basis of this more informed the better.  

  MF  So it needs to be near enough to be reliable, but at the same time we need to know about the uncertainty. So how near is the system at the moment as far as these important indicators are concerned to getting that right?   SRC  Well, I think there's an awful lot of effort that goes into ensuring that you are presenting on the basis of the information set that you have the best available estimates that you can, and I think there's an awful lot of effort that goes into thinking about quality, that thinks about quality assurance when these are put together, that thinks about the communication how they mesh in with the rest of the, for example, the economic picture that you have, so you can reasonably assure yourself that you're providing people with the best possible estimate that you can at any given moment. But at the same time, you want to try to guide people by saying, well, this is an estimate, there's no guarantee that this is going to exactly reflect the real world, the more that you can do to put some sort of numerical context around that the more the reliable basis you have for people who are using those numbers, and thinking about as I say, particularly in the case of those statistics that may be revised in future as you get more information. You can learn things, obviously from the direction, the size of revisions to numbers that have happened in the past, in order to give people a sense of how much confidence they should place in any given number produced at any given point in that cycle of evolution as the numbers get firmer over time.   MF  If you're looking to use the statistics to make some decision with your business or personal life, where do you look for the small print? Where do you look for the guidance on how reliable this number is going to be?   SRC  Well, there's plenty of guidance published in different ways. It depends, obviously on the specific statistics in question, but I think it's very important for producers to ensure that when people come for example to websites or to releases that have the headline numbers that are going to be reported, that it's reasonably straightforward to get to a discussion of where do these numbers come from? How are they calculated? What's the degree of uncertainty that lies around that arising from these things? And so not everybody is obviously going to have an appetite for the technical discussion there. But providing that in a reasonably accessible, reasonably findable way, is important and I think a key principle is that if you're upfront about explaining how numbers are generated, explaining about the uncertainty that lies around them in as quantified way as you can, that actually increases and enhances trust in the underlying production and communication process and in the numbers rather than undermining it. I think you have to give the consumers of these numbers by and large the credit for understanding that these things are only estimates and that if you're upfront about that, and you talk as intelligently and clearly as you can about the uncertainties - potential for revision, for example - then that enhances people's confidence. It doesn't undermine it.   MF  You mentioned there about enhancing trust and that's the crux of all this. At a time we're told of growing public mistrust in national institutions and so forth, isn't there a risk that the downside of talking more about uncertainty in statistics is the more aware people will become of it and the less those statistics are going to be trusted?   SRC  I think in general, if you are clear with people about how a number is calculated, the uncertainty that lies around it, the potential for revision, how things have evolved in the past - that’s not for everybody, but for most people - is likely to enhance their trust and crucially, their understanding of the numbers that you're presenting and the context that you're putting around those. So making that available - as I say, you have to recognise that different people will have different appetites for the technical detail around this - then there are different ways of presenting the uncertainty not only about, you know, outturn statistics, but in my old gig around forecasts of where things are going in the future and doing that and testing it out with your users as to what they find helpful and what they don't is a valuable thing to be doing.   MF  You've been the stats regulator for a little while now. Do you think policymakers, perhaps under pressure to achieve certain outcomes, put too much reliance on statistics when it suits them, in order to show progress against some policy objective? I mean, do the limitations of statistics sometimes go out of the window when it's convenient. What's your view of how well certainty is being treated by those in government and elsewhere?    SRC  Well, I think certainly in my time as a forecaster, you were constantly reminding users of forecasters and consumers of that, that again, they're based on the best available information set that you have at the time. You explain where the judgements have come from but in particular, if you're trying to set policy in order to achieve a target for a particular statistic at some point in the future, for example, a measure of the budget deficit, then having an understanding of the uncertainty, the nature of it, the potential size of it in that context, helps you avoid making promises that it's not really in your power to keep with the best will in the world, given those uncertainties. And sometimes that message is taken closer to heart than at other times.   MF  Time I think to bring in Craig now at this point, as head of national accounts and the team that produces GDP at the ONS to talk about uncertainty in the real world of statistical production. With this specific example, Craig, you're trying to produce a single number, one single number that sums up progress or lack of it in the economy as a whole. What do you do to make the users of the statistics and the wider public aware of the fact that you're producing in GDP one very broad estimate with a lot of uncertainty built in?   CRAIG MCLAREN  Thanks, Miles. I mean, firstly, the UK economy - incredibly complex isn't it? The last set of numbers, we've got 2.7 trillion pounds worth of value. So if you think about how we bring all of those numbers together, then absolutely what we're doing is providing the best estimate at the time and then we start to think about this trade off between timeliness and accuracy. So even when we bring all of those data sources together, we often balance between what can we understand at the point of time, and then equally as we get more information from our businesses and our data suppliers, we evolve our estimates to understand more about the complex nature of the UK economy. So where we do that and how we do that it's looking quite closely at our data sources. So for example, we do a lot of surveys about businesses, and that uses data provided by businesses and that can come with a little bit of a what we call a time lag. So clearly when we run our monthly business surveys that's quite timely. We get that information quite quickly. But actually when we want to understand more detail about the UK economy, we have what we call structural surveys, and they're like our annual surveys. So over time, it can take us a couple of years actually to get a more complete picture of the UK economy. So in that time, absolutely. We may revise the estimate. Some businesses might say, well, we forgot about this. We're going to send you a revised number. We look at quite closely about the interplay between all the dynamics of the different parts of the economy, and then we confront the data set. So I think by bringing all this information together, both on the timeliness but also as we get a more complete picture, we start to refine our estimates. So in practice, what we do find is as we evolve our estimates, we can monitor that. We do look quite closely at the revisions of GDP, then we can produce analysis that helps our users understand those revisions and then we quite heavily focus on the need for rapid information that helps policymakers. So how can policymakers take this in a short period of time, but then we provide this information to understand the revision properties of what we would call that about how our estimates can change and evolve over time as we get additional information going forwards.   MF  So let's just look at the specifics, just to help people understand the process and how you put what you've just explained so well into action. Craig, the last quarterly estimate of GDP showed the economy contracted slightly.   CM  That's exactly right Miles and I think where we do produce our estimates in a timely basis, absolutely they will be subject to revision or more information as we get them. So this is why it's important, perhaps not to just focus on a single estimate. And I know in our most recent year in the economy, when that's all pretty flat, for example, or there's sort of a small fall, we do have a challenge in our communication. And that becomes a little bit back to the user understanding about how these numbers are compiled. And also perhaps how can you use additional information as part of that? So as I mentioned the UK economy is very complex, GDP is a part of that, but we also have other broader indicators as well. So when we do talk about small movements in the economy, we do need to think about the wider picture alongside that.   MF  Okay, so the last quarterly estimate, what was the potential for revision there? Just how big could that have been?   CM  We don't formally produce what we call range estimates at the moment. We are working quite closely with colleagues about how we might do that. So if you think about all the information that comes together to produce GDP, some of that is survey base which will have a degree of perhaps error around it, but we also use administrative data sources as well. So we have access to VAT records anonymized of course, which we bring in to our estimates. So the complex nature around the 300 different data sources that we bring in to make GDP means that having a range can be quite a statistical challenge. So what we do is we can actually look at our historical record of GDP revisions, and by doing that, in perhaps normal times, are quite unbiased. And by that, I mean we don't expect to see that to be significant either way. So we may revise up by perhaps 0.1 or down by 0.1, but overall, it's quite a sort of considered picture and we don't see radical revisions to our first estimates over time.    MF  You're saying that when revisions happen they are as likely to be up as they are to be down and there's no historical bias in there either way, because presumably, if there was that bias detectable, you would have acted some time ago, to make sure it was removed from the methodology.  

CM 

Exactly. Exactly.  

MF 

Just staying with this whole business of trying to make a very fast estimate because it is by international standards, a fast estimate of a very, very big subject. How much data in percentage terms would you say you’ve got at the point of that first estimate as a proportion of all the data you're eventually going to get when you produce your final number? 

  CM  It does depend on the indicator Miles. So the UK is one of the few countries in the world that produces monthly GDP. So we are quite rapid in producing monthly GDP. Robert did mention in the introduction of this session that with monthly GDP we do an output measure. So this is information we have quite quickly from businesses. So our monthly GDP estimate is based on one of the measures of the economy. So that uses the output measure. We get that from very rapid surveys, and that has quite a good coverage around 60 or 70% that we can get quite quickly. But then as we confront with our different measures of GDP, that's when the other sources come in. So we have our expenditure measure which takes a bit longer and then we have our income measure as well. So we have this process in the UK working for a monthly GDP which is quite rapid. We then bring in additional data sources and each of these measures have their own strengths and weaknesses until we can finally confront them fully in what we call an annual framework. And then often that takes us a couple of years to fully bring together all those different data sources so we can see the evolution of our GDP estimates as additional data comes in.    MF  Now looking back to what happened during the pandemic, of course, we saw this incredible downturn in the economy as the effects of lockdown took effect on international travel that shuddered to a halt for a while and everyone was staying at home for long periods. The ONS said at that point, it was the most significant downturn it had ever recorded. But then that was closely followed of course when those restrictions were eased by the most dramatic recovery ever recorded. Just how difficult was it to precisely manage the sheer scale of that change, delivered over quite a short period, relatively speaking, just how good a job did the system do under those very testing circumstances?   CM  It was incredibly challenging and I think not just for official statistics of course but for a range of outputs as well. Viewing it in context now, I think when the economy is going relatively stable, perhaps a 0.1 or 0.2 change, we might start to be a bit nervous if we saw some revisions to that but if you think about I believe at the time was around 20% drop in activity and actually the challenge of ensuring that our surveys were capturing what was happening in the economy in the UK, and in the ONS we stood up some additional surveys to provide us with additional information so we could understand what was happening. We still have that survey that's a fortnightly survey. So the challenge that we had was to try and get the information in near real time to provide us with the confidence and also obtaining information from businesses that are not at their place of work, so they weren't responding to our surveys. So we had to pivot to using perhaps telephone, collecting information in a different way really to understand the impact the economy. So when we look back now, in retrospect, perhaps a 20% drop should that have been 21 or 22%. It's all relative to the size of the drop is my main point I would make. So in the context of providing the information at the time, we were quite fortunate in the survey on the data collection front to really have a world leading survey for businesses that provided that information in near real time, which we could then use to understand the impacts on different parts of the UK economy. And I think now when we get new information in an annual basis, we can go back and just confront that data set and understand how reliable those estimates were, of course.   MF  Of course the UK was not alone in making some quite significant revisions subsequently to its initial estimates, what was done, though, at the time to let the users of the statistics know that because of those circumstances, which were so unusual, because the pace of change you were seeing was so dramatic, that perhaps there was a need for special caution around what the data was seeming to say about the state of the economy?   CM  Exactly, and it was unprecedented of course as well. So in our communication and coming back to how we communicate statistics, and also the understanding as well. We added some additional phrasing, if you like Miles, to ensure people did sort of understand and perhaps acknowledge the fact that in times like this, there is an additional degree of uncertainty. So the phrasing becomes very important, of course to reflect that these are estimates they're our first estimate at the time, they perhaps will be maybe more revised than perhaps typically we would expect to happen. So the narrative and communication and phrasing, and the use of the term ‘estimate’, for example, became incredibly important in the time of the pandemic. And it's also incredibly important in the context of smaller movements as well. So while we had this large impact on COVID, it was our best estimate at the time, and I think it's important to reflect that, and as we get more and more understanding of our data sources, then those numbers will be revised. So what we did do was really make sure that was front and centre to our communications just to reflect the fact that there can be additional information after the fact but this is the best estimate at the time and there's a degree of uncertainty. And we've continued that work working closely with colleagues in the regulator to understand about how best we can continue to improve the way that we communicate around uncertainty in what is a complex compilation process as well.   MF 

Professor Mairi Spowage. You've heard Sir Robert talking earlier about the importance of understanding uncertainty in statistics and the need to make sure our statistical system can deal with that, and explain it to people properly. You've heard Craig also there explain from a production point of view the length to which the ONS goes to deal with the uncertainty in its initial estimates of GDP and the experience of dealing with those dramatic swings around the pandemic. What is your personal take on this from your understanding of what the wider public and the users of economic statistics have a right to expect? What do you make of all that?  

MAIRI SPOWAGE 

So I think I’d just like to start by agreeing with Robert, that explaining uncertainty to users is really important. And in my view, and certainly some research that some of my colleagues at the Economic Statistics Centre of Excellence have done, which show that actually it increases confidence in statistics, because we all know that GDP statistics will be updated as more information comes in when these are presented as revisions to the initial estimates. And I think the more you can do to set expectations of users that this is normal, and sort of core part of estimation of what's going on in the economy, the better when these revisions inevitably happen. We very much see ourselves as not just a user of statistics, but also I guess a filter through which others consume them. We discuss the statistics that ONS produce a lot, and I think we like to highlight for example, if it's the first estimate that more information will be coming in where revisions have happened. And particularly when you're quite close to zero, as we've been over the last year or so, you know, folks can get quite excited about it being slightly above or below zero, but generally the statistics are in the same area even though they may be slightly negative or slightly positive.  

 

MF 

Yes, and I'd urge people to have a listen to our other podcast on the whole subject of ‘what is a recession’ to perhaps get some more understanding of just how easily these so called technical recessions can in fact be revised away. So overall then Mairi, do you think the system is doing enough that people do appreciate, particularly on the subject of GDP, of course, because we've had this really powerful example recently, is doing enough to communicate the inherent uncertainty among those early estimates, or perhaps we couldn't be doing more?  

 

MS 

Yeah, absolutely. Obviously, there's different types of uncertainty and the way that you can communicate and talk about uncertainty when you're producing GDP statistics is slightly different to that, that you might talk about things like labour market statistics, you know. I know there are a lot of issues with labour market statistics at the moment, but obviously, the issues with labour market statistics in normal times is really about the fact it's based on a survey and that therefore has an inherent uncertainty due to the sampling that has to be done. And it might mean that a seemingly you know, an increase in say unemployment from one quarter to the next isn't actually a significant difference. Whereas with GDP, it's much more about the fact that this is only a small proportion of the data that will eventually be used to estimate what's happened in this period in the economy. And over time we’ll sort of be building it up. I think the ONS are doing a good job in trying to communicate uncertainty in statistics but I think we could always do more. I think having you know, statisticians come on and talk about the statistics and pointing these things out proactively is a good idea. So much more media engagement is definitely a good idea. As I said, we try and through you know informal means like blogs and podcasts like this, to talk about the data that have been produced. And you know, when there are interesting features of it, which are driving some of the changes and to what extent those might change. So, one of the features over the last year for 2023 has been the influence of, you know, things like public sector strikes on the data, because when there's less activity in the public sector that also changes the profile of growth over the year quite a lot. And that's been very influential over 2023. So I think it's important that there's more discussion about this and, to be honest, more knowledge in economic circles about how these statistics are put together. Or you know, I'm an economic statistician rather than an economist per se, and I think the more knowledge and awareness that can be amongst economic commentators on these issues, I think the better because if we’re upfront about the uncertainty, I think it increases the confidence when these revisions inevitably happen.  

 

MF 

Perhaps then it is the way the statistics are told in the media and elsewhere? Of course, they're invested by those observers with more authority perhaps than they deserve. Particularly, of course, it must be very tempting if you're a politician and the numbers are going your way, then obviously you want people to believe they are absolutely 100% accurate.  

 

MS 

Absolutely. We're in a funny situation at the moment. I mean, you know, our research institute focuses a lot on the Scottish economy. And the data for Scotland for 2023 shows... Yes, it shows two quarters of contraction and two quarters of growth, but they're not joined together. So there wasn't a technical recession in Scotland. But you know, over the year, basically, the Scottish and UK economies have had a really poor year with hardly any growth. But you know, I haven't seen it yet, but I'm expecting that there will be some people, you know, sort of crowing about that, like it's really showing that the Scottish economy is doing better or something when it's not really. So there will always be politicians who try to you know, over interpret changes in the data. Another example would be the first estimates of quarterly growth in the first part of 2023 showed 0.4 growth in Scotland compared to 0.1 in the UK, and there were politicians saying that Scotland was growing four times as fast as the UK. These things will happen, but you know, one of our roles to be honest is in our regular blogs and communications with the policy community, particularly in Scotland, but also beyond, is to point these things out and say that they're a bit silly. That no doubt these things will be revised and come closer together and nobody should get too excited about them.  

 

MF 

Thinking particularly about when you're looking at levels of geography different from the UK for yourselves in Scotland and from where I'm sitting here in Wales as well, for that matter. Do the data tend to become more or less accurate, should we have more or less confidence in the sort of datasets we're seeing for those different levels of geography?  

 

MS 

Well, generally it becomes more unreliable, and it's subject to more uncertainty. A lot of the data that's used is based on business surveys for estimating what's going on in the economy. And there are two areas of uncertainty there. The samples at smaller geographies are smaller so it's greater uncertainty because of sampling variability. But there's also a key problem on the data infrastructure in the UK that business data - this is across GB because Northern Ireland's is collected slightly differently - is collected on units which are GB wide. So it does make estimating what's going on in the parts of GB quite challenging. And there are some additional estimation procedures that need to be done to actually say what's going on in Scotland or in Wales. So it does add an additional layer of uncertainty to any sort of economic estimation at sub-UK geographies. 

 

MF 

I should add at that point that improving the quality of regional sub-national data has always been an important part of the ONS’s work and continues to be part of its strategic drive. But Sir Robert, from what you've seen recently, particularly over the last year, the way that GDP estimates have been used in the media and in politics, and particularly the whole business of comparing quite small differences in GDP change internationally and the significance that's invested in that, the relative growth rates between one country or another. Has there been too much discussion around that; has too much weight being put on that recently from where you've been sitting?  

 

SRC 

Well, I think just to pick up on the point that Mairi was making, you can end up investing, you know, much too much significance in comparisons of what's going on in one place and in another place over a relatively short period of time in which there's likely to be noise in those numbers. So, as she said, the idea that you know, taking one area where the growth is 0.4 in a quarter and another where it’s 0.1 and saying that one economy is growing four times more quickly, while strictly true on the basis of those that is really not an informative comparison, you have to look over a longer period for both. When you get to international comparisons, there's the additional issue of the extent to which although there are international standards and best practices as to how, for example, national accounts are put together. The way in which this is actually carried out from place to place can be done in different ways that make those sorts of comparisons again, particularly over short periods, but also when the economy is doing strange things as it was during the course of the pandemic, particularly tricky. So in the GDP context, obviously, there was the question mark about having big changes in the composition of what the education and health sectors were doing as we went into the period of lockdown and therefore judging how the output of those sectors had changed was a really very tricky conceptual judgement to make. And one of the issues that arose about trying to make international comparisons is that different people will be doing that in different ways, depending in part on how they measure the outputs of health and education under normal circumstances. So if you are going to do international comparisons, it’s certainly better to look over a longer period. So you're avoiding being misled by short term noise but also having a care to the way in which methodologies may differ, and that that may matter that sometimes more than it does allow others to see if this is actually a meaningful comparison of like for like.  

 

MF 

It's also worth pointing out, as I think we have in previous podcasts, that the UK is one of the few economies that does actually seek to measure the actual output of public services, whereas some countries just make broad assumptions about what those sectors have been doing. But it's also worth mentioning, I think that some countries simply don't revise as much as we do because their system makes an initial estimate, and then they don't return to it for some years in the case of a number of countries.  

 

SRC 

Yes, that's true. And so then the question is sometimes - and I think this arose relatively recently in the UK context - of the set of revisions that you look at and change the international comparison, but you know that some countries have not yet essentially done the same set of revisions. For example, the way in which you try to pull together the estimates of output income and expenditure at times afterwards as you have more information from annual surveys you have more information on incomes for example, from the tax system. So, again, at any given moment, even though you know, you're in both cases, trying to say well, what's our best sense of what was going on a year ago, different countries will be at different stages of the statistical production process and the proportion of the eventual total information set on which you base your estimates, you know, some countries will have incorporated more of that than less, and so a revision that you're doing this month, somebody else may not do for six months, and that again, complicates the picture, and really again, suggests that looking at international comparisons at too high frequency or too much in the recent past, there are bigger uncertainties and caveats that you ought to be placing around big calls and big interpretations based on that.  

 

MF 

Yes, and while it's hard enough to know where you are at any given point in the economy, it's even harder of course - infinitely harder you might say - to work out where on earth you're going to go next. You've spent a lot of time in the forecasting business, how are forecasters, and I know the Bank of England in particular is taking a good look at this at the moment - the data it relies upon in order to make its forecast - what can the statistical system be doing to support organisations with that unenviable task of having to look into the future and guide us on what is going to happen next?  

 

SRC 

Well, I think from the perspective of the forecasters themselves, many of the same principles that we've been talking to in terms of how the statistical system should communicate uncertainty applies in spades, in the case of forecasts where explaining how you've reached the judgements that you have, the uncertainty that you know, past forecast errors, that particular sensitivity of a forecast, a judgement that you'd be maybe making in some part of it, the more you can do to explain that increases people's trust rather than reduces it. From the perspective of the statistical producer helping the forecaster, I think, again, explanation, if you have got particular difficulties, particular reasons why you think there might be greater uncertainty than in the past around particular numbers, it's very important. The current evolution of the labour market statistics is a good example of that - you need to be talking to the big users and the big forecasters about the particular uncertainties, there may be at a given time so they can take account of that as best they can. On the other hand, having been a forecaster for 10 years, I certainly took the view that for forecasters to complain about revisions in economic data is like sailors complaining about waves in the sea. I'm afraid that is what you're dealing with. That's what you have to sail on and everybody makes their best effort to come up with the best possible numbers, but it's a fact of life. And your knowledge and understanding of what's going on in the past now, and how that informs your judgements in the future evolves over the time. It doesn't remain static and you're gazing through a murky cloud at some time, but that doesn't reduce the importance of doing the best job you can.  

 

MF 

Final word then for the forecasters and for everybody else. The statistics are reliable but understand their limitations.  

 

SRC 

Yeah.    

MF  Well, that's it for another episode of Statistically Speaking. Thanks to Sir Robert Chote, Professor Mairi Spowage, and Dr. Craig McLaren, and of course, thanks to you, as always for listening.   You can subscribe to future episodes of this podcast on Spotify, Apple podcasts, and all the other major podcast platforms. And you can follow us on X, previously known as Twitter, via the @ONSfocus feed.   I'm Miles Fletcher and from myself and producer Steve Milne. Goodbye    

ENDS. 

In this episode Miles is joined by the National Statistician, Sir Ian Diamond, to reflect on what has been a busy and transformative year at the Office for National Statistics. 

 

Transcript  

 MILES FLETCHER  This is “Statistically Speaking”, the official podcast of the UK Office for National Statistics, I’m Miles Fletcher. This is our 20th episode, in fact, a milestone of sorts, though not a statistically significant one. What is significant is that we're joined, once again, to look back at the highlights from another 12 months here at the ONS by none other than the National Statistician himself, Professor Sir Ian Diamond. Ian, thanks for joining us again. The year started for you with being reappointed as the national statistician. As 2023 developed, how glad did you feel to be back?   SIR IAN DIAMOND 

Of course, you know, I was hugely privileged to be invited to continue. It's one of the most exciting things you could ever do and I will continue to do everything in my power to bring great statistics to the service of our nation.   MF  To business then, and this time last year, we sat in this very room talking about the results of Census 2021, which were coming in quite fresh then. And we've seen the fastest growth of the population, you told us, since the baby boom of the early 1960s. Over the course of the year much more data has become available from that census and this time, we've been able to make it available for people in much richer ways, including interactive maps, create your own data set tools. What does that say about the population data generally and the way that people can access and use it now? How significant is that there's that sort of development?   SID  Well I think we need to recognise that the sorts of things that we can do now, with the use of brilliant technology, brilliant data science and brilliant computing is enabling us to understand our population more, to be able to make our data more accessible. 50, 60, 70 years ago, 150 years ago, we would have just produced in about six or seven years after the census, a report with many, many tables and people would have just been able to look at those tables. Now, we're able to produce data which enables people to build their own tables, to ask questions of data. It’s too easy to say, tell me something interesting, you know, the population of Dorset is this. Okay, that's fine, but actually he wants to know much more about whether that's high or low. You want to know much more about the structure of the population, what its needs for services are, I could go on and on. And each individual will have different questions to ask of the data, and enabling each individual to ask those questions which are important to them, and therefore for the census to be more used, is I think, an incredibly beautiful thing.   MF  And you can go onto the website there and create a picture...   SID  Anyone can go onto the website, anyone can start to ask whatever questions they want of the data. And to get very clearly, properly statistically disclosed answers which enable them to use those data in whatever way they wish to.   MF  And it's a demonstration of obviously the richness of data that's available now from all kinds of sources, and behind that has been a discussion of, that's gone on here in the ONS and beyond this year, about what the future holds for population statistics and how we can develop those and bring those on. There's been a big consultation going on at the moment. What's the engagement with that consultation been like?   SID 

Well the engagement's been great, we’ve had around 700 responses, and it addresses some fundamental questions. So the census is a really beautiful thing. But at the same time, the census, the last one done the 21st of March 2021, was out of date by the 22nd of March 2021, and more and more out of date as you go on and many of our users say to us, that they want more timely data. Also by its very nature a census is a pretty constrained data set. We in our country have never been prepared to ask for example, income on the census yet this is one of the most demanded questions. We don't ask it because it is believed that it is too sensitive. And so there are many, many, many questions that we simply can't ask because of space. There are many more questions that we simply cannot ask in the granularity that we want to. We've been doing some work recently to reconcile the differences between estimates in the number of Welsh speakers from surveys with estimates on the number of people in the census who report they speak Welsh. Frankly, it would be better if we were able to ask them to get information in a more granular way. And so while the census is an incredibly beautiful thing, we also need to recognise that as time goes on, the technology and the availability of data allowing us to link data becomes much more of a great opportunity that we have been undertaking a lot of research, a lot of research which was asked for by the government in 2013, following the report by Chris Skinner, the late Chris Skinner, Joe Hollis, and Mike Murphy, which is a brilliant report. We said at the moment we need to do another census in 2021. That's what we have done and I believe it to be one of the best coverages there has ever been. And yet we need to assess whether administrative data could be used in future to provide more timely, more flexible and more accessible data and that's what the consultation is about. I will be making a recommendation to the UKSA (UK Statistics Authority) board in the future. In the near future we have to say, and I think it is worth saying that what the consultation says to us is that people are very, very, very much in favour of the direction of travel but at the same time as yet accepting our prototype, unconvinced about the data flows and the sustainability of those data flows to enable us to do it and so, we are looking at how to respond to other very important analyses and we will do so in the near future.   MF  When can the people who contributed to that consultation, roughly when should they expect to hear from us?

SID  I think the expectation is we'll publish something by the end of quarter one in 2024.  

MF  Surveys have continued to be a very important part of what the ONS does, these very large national surveys, and yet one of the biggest challenges of the has been maintaining coverage and particularly response rates and obviously, particularly with the Labour Force Survey recently that has been a particular issue for the ONS hasn’t it. Where do things stand now as we move into modernising the traditional Labour Force Survey and moving to a new model because it's an issue statistics bodies around the world have been dealing with, it's harder to get people to complete surveys like they used to.    

SID  I think it's a fair point that response rates globally are a challenge and response rates globally, not only in national statistics issues, but in the private sector organisations that also collect data, are a challenge. So we need to recognise that. A part of that is that historically, one could find people at home, knock on doors, have that conversation with people, and perhaps post pandemic people are less willing to have a conversation at the house. Also, people are very busy. They work in multiple occupations. They are not always in, they live in housing accommodation which is more and more difficult to access. This there is no kind of single magic bullet here that we could press all we would have. The first thing to say Miles is that we recognise that and that's why we worked with our colleagues at His Majesty's Treasury to provide a project to go to what we call a transformed Labour Force Survey. And I think that that's a hugely exciting project for a number of reasons. One, the labour force survey which has been around for a long time, the questionnaire had become a little bit unwieldy. And also we wanted to enable people to have much more flexibility at the time of which they answered the question. We are in the field with the pilots for that service. We've been pretty good. There are good response rates. There are also some challenges around getting the questions right. These aren’t challenges that stress me, that's why you do a pilot, but at the end of the day we're hoping to be able to transform into that new Labour Force Survey early in 2024, in the first half of 2024. We're working very closely in doing that with our major stakeholders and the Bank of England, His Majesty's Treasury and the Office for Budgetry Responsibility (OBR), is you take a joint decision on when people feel comfortable that we have had enough dual running to enable us to move forward. The other question that I'd have to raise around surveys more generally, is on inflation, which we have all been subjected to in many, many areas in the last couple of years, inflation in survey collection has increased massively and so in the last year we've had to make real judgments about how we maintain quality. And in the next few years, we will really be needing to think through exactly how we conduct our surveys and the cost of doing so.   MF  Yes. Of all the people who should be aware of inflation are the people who report it, and certainly the impacts of those relatively high rates of inflation have impacted us as much as anybody else. The challenges not withstanding of running surveys, the interest of government bodies in getting that information directly from people does continue to underline the unique value of surveys. Some people say Oh, well, they surely they can get this information from other sources I've even seen it suggested that social media could provide the answers, but there is a unique value isn't there and actually getting a statistically representative sample of people and speaking to them directly.   SID  It depends Miles, I think it absolutely depends on what the question is you're trying to answer. If you're trying to get some answers to a question where the answer can be obtained through administrative data sources, then you don't need a survey. Surveys are difficult to conduct and difficult to pilot and plan and extremely expensive to undertake. So you should only do a survey if you can't get the information from somewhere else. Therefore, you know, I do think that we need to be very, very careful in thinking through when we need to do surveys. Does that mean to say we don't need to do surveys? Absolutely not. There are reasons why you need to do surveys. It may be that you need to really spend some time identifying whether someone really is eligible for hte questions you're going to ask or you may want attitudes. I don't know how to get someone's attitude without asking them. And so there are reasons why you would want to do a survey, but I would argue that you should only do a survey when you cannot get the data from elsewhere. And you also mentioned social media. Social media is an incredibly interesting and important source of data. Now, I wouldn't necessarily say it was statistically representative, but we absolutely have to be flexible in what we call data. We have to be sure of the quality of those data and we have to be sure that we are really aware of what the population is that are represented by those data. So we are using many, many, many types of data now that we would not have used 50 years ago, we simply couldn't have used things like telephony data, things like card data, things like data from satellites to address questions which those data are the best way of providing answers.    MF  And there are some fantastic examples of that around the ONS. If you look at how we've changed prices over the last couple of years, again, the measurement of inflation, bringing in new data sources most recently from the US car industry, from the rail industry as well and it all means that the estimates of inflation are now based on many hundreds of thousands of price points, where it used to be just a few things.   SID  It doesn't matter what the numbers are, frankly, it matters that you've got a good coverage it matters that you have the most appropriate method and that your data are as accurate as possible. And I do think it is incredibly important. We use a wider range of data sources. I think it's incredibly exciting what those data sources are, but we should only do so being unbelievably careful about what the metadata are that go with them, what the coverage is, why we are using them and whether or not they represent an improvement over what we could do before.   MF  Okay, so we've seen in the area of prices, the measurement of inflation, there's new innovative data sources coming from outside, coming from industry. What sort of an improvement does that represent in how we measure inflation, when it's such an important time for cost of living?    SID  Well, I think it helps because we have more accurate data. We have more timely data, we have data that are real. So on rail prices, we know what people pay as opposed to what the price as advertised necessarily is and I think that is important. And so being able to properly understand what the consumer is doing, therefore, what inflation is, is to me, incredibly important. I would say that all this effort that we're putting in would not necessarily just be about prices. Here it is about do we understand more about what is going on in the economy, and there are many more questions that we can ask from those data when you've got them, and simply from some of the fixed price point data that we have previously.   MF  Now one massive change we've seen lately, and this is another area we've managed to improve coverage, is of course the private rental sector. It’s become much more important as we've seen house prices coming under pressure and mortgages under pressure by high interest rates and so forth. It's revealed a very interesting picture of long-term change, and also in more contemporary terms, what's actually going on with the economy right now.   

SID 

Oh 100% 

 MF  Talking about areas where we've been able to form a new view of what's really been going on. An area that attracts a particular commentary during the course of the year is expenditure on research and development. Regarded as a very important area of activity if you're talking about productivity, future economic growth...we substantially upgraded our estimates of R&D. What was the story behind that? Why was that necessary?    SID  Well it was incredibly important because we looked very carefully at our data, we look very carefully at our samples, we looked at our coverage and we decided that we needed properly to to bring in a much wider range of business. And we were reflecting very much those businesses from a very wide range of areas who were able and available to claim R&D tax credits, and therefore to be able to get a decent sample, and the critical thing here is not only were we making good estimates, but we were able to understand much more about what, particularly for smaller tech and creative industry companies, was R&D. And I think that is something that we need to recognise particularly in those smaller companies where there's a much greater flexibility about what people would call R&D.   MF  It’s a reflection perhaps that startups are the sort of firms that do R&D these days, and less so the sort of industrial behemoths with huge R&D departments. But there was an interesting change nonetheless, and obviously considerable improvement in measuring that very important area. This all I guess comes under the umbrella of future proofing practices and systems and this all came under a refreshed data strategy that we launched during the course of the year. One of the fundamental principles underlying that, where is it taking us?   SID  I think, I mean, just where I've been coming from, are to do a much more holistic view of what data are and how we really use data which are most appropriate to answer the questions that we have, and we recognise that the economy and indeed society are changing very quickly, and therefore we need appropriate data to be able to answer those questions. For example, if you look at employment, there are many, many people in our society who have three, four, even five jobs, we need data which enable us to find out what the distribution of the number of jobs people have is, what they're spending their time doing, and how that impacts on our understanding of the labour force.   MF  Worth perhaps recognising some of the particular areas where new data has also been able to shed new light and particularly think of the payments industry which obviously digital payments happen very quickly. They provide almost a daily update on the state of consumer spending. With it obviously the state of the of the of the wider economy. We've managed to strike up partnerships with a huge cross section of the payments sector. What is the particular value of that? And what do we say to perhaps other data providers who might wish to enter into similar arrangements?   SID  Well I think we’d say we do everything ethically, and with complete privacy, but at the same time in the public good. And that is, to me, incredibly important. And so understanding what the consumer is spending money on understanding what the consumer is not spending money on, and the transitions, is incredibly important to enable policy which impacts very positively on all of our fellow citizens. So we are very proud of those partnerships. We value them greatly. We don't take them for granted. And those data, entirely ethically provided, with great security but at the same time enabling us to understand what is going on at an early stage in the economy is incredibly important.   MF  And of course it’s worth restating, as mentioned already, that of course all of these data are anonymized and aggregated, and no individual would ever be able to identify themselves or be identified from that fast payments data which of course is helping to inform economics policy.   MF  Providing data to the people who do make policy and around government and to make sure that policies are really informed by evidence of course that is the major purpose behind the new Integrated Data Service, which was accredited this year under the Digital Economy Act. And that's enabling data to be shared around government in a way that simply wasn't possible before.   SID  80 datasets available now and indeed, that number going up more or less by the day. And one of the most important things here is that there are very few challenges which government face which simply can be addressed by data from one department. Therefore, what we need to be able to do is to link data from different sources to enable us in a very granular way to be able to answer questions about topics for which the answer requires data from many sources. And the Integrated Data Service allows us to do that. It allows us to do at a pace and allows us to do it in a way which brings a wide variety of analysts to the party. And I think that, you know, this year major milestone in getting Digital Economy Act accreditation. And we will be looking to streamline the process of using it over the next year, as well as seeing more and more and more projects on it having successful results.   MF  And sharing between departments at the national level is important, but also it's been a long-term aim of the ONS to improve its coverage at local levels. And again, there's another important initiative kicked off this year, and that's the launch of ONS Local.   SID  Yes and I’d say that the two are linked. It doesn't matter whether you are at a national level or whether you are at a regional level, linked data are important, but we are very pleased working with funding from our colleagues at the Department of Housing, Levelling Up and Communities to have been able to place ONS staff in regions. So we're not talking about teams of people in Manchester or teams of people in Exeter, but we are talking about interlocuters in the southwest, northwest for example, who can really work with the leaders there to ensure that we've got local data for local leaders to make local decisions and that's incredibly important because the questions that people wish to ask are different in different parts of the country and therefore we need to recognise that so it is a good initiative, which I hope will bear fruit in 2024.   MF  And the importance of data in government has been underlined by a big initiative, which takes in everybody, not just statisticians and analysts, but everybody in the civil service, has been engaged in what's called the One Big Thing campaign to spend time learning about data that's important to the use of data. How has that initiative been going? The ONS has been a central part of that. How's it been going? How important is it?   SID  It is critical. We do not need every public servant to be able to be a brilliant statistician, but we need every public servant to be data literate. We need every public servant to be able to understand data and the best policy comes about when analysts and policymakers and potential beneficiaries work together. And that requires that you can have that data literate conversation. And so I think One Big Thing is a great thing.   MF  In fact that the need for people to better understand data became evident early this year, of course, when our GDP revisions were quite dramatically revised in the early part of the autumn as the estimates for the big peak pandemic years, 2021 and 22. There was quite a reaction from some parts of the media and beyond, who reported that our original figures were, because they had revised so dramatically, were simply wrong. I mean, that's not the case. revisions of course have always been integral part of the process. Indeed the OSR, the statistics regulator, found as part of its review our approach to be, and their words were appropriate and well managed, however, it also found the ONS could communicate better the uncertainty in those early estimates of GDP and that's a learning point for the future.   MF  We saw particular attention recently for the natural capital outputs, measures of the natural environment, and they attracted a degree of media interest we haven't seen so far, helped by the fact we're able to bring it to life with an analysis of time spent in nature and so forth, and you spoke to BBC Countryfile about that particular work. What's your overriding thoughts on that release?  Are we moving to the point where these kinds of measures are getting more exposure? Are they being recognised for their value?   SID  I thought the national Natural Capital stuff was brilliant. I've always thought, as I said last year, that we should put alongside GDP measures of the environment and measures of well-being, but you need a concise picture and that's where we're moving in the future.    MF  As we speak, we're heading into the bleak midwinter of 2023. The nation is doing all it can to avoid a seasonal bout of flu and the other viruses that traditionally do the rounds at this time of year. And that's seen a revival of our surveillance effort. The Winter Coronavirus Infection Study (WCIS). Tell us about that. What's the purpose of it and what's happening?   SID  Yeah, working very much for our colleagues at the UK Health Security Agency who asked us whether we would be prepared to stand back up some of the work we do on surveillance of winter flus, COVID and other issues. and we're of course pleased and proud to be asked. We’re using a different strategy to the one we were using in the past, this is very much simply a mail out of tests enabling people to take a test and then to make estimates, and at the moment the good news is that the estimates of positivity are relatively low, but the bottom line is we need to recognise that without some good hard data on those levels it's pretty impossible for government to plan, and so I think it’s a really exciting initiative. It's a smaller survey than one in the past. It's a survey which will make national estimates rather than many regional estimates, but it's one that we think is extremely exciting, and builds on some of the work we've done in the past.   MF  And now of course everyone knows how to self-administer a COVID test and that ability makes it much easier to run these big.   SID  Oh 100%. I do think we need to recognise the way in which the world moves on. And certainly, when we first set up the COVID infection survey in 2020. We were not aware of the extent to which people could self-administer, we learned pretty quickly that's why we were able to transition to self-testing, but I think we are in a world where we can do this at pace and provide estimates very, very quickly.   MF  Well, thank you very much for joining us. Great to have you with us again at the end of the year. You could choose just three words to sum up your 2023   SID  Exciting, full of change and high-quality statistics.   MF  And looking ahead to 2024, which pieces of work are you looking forward to most?    SID  The economy is changing quickly, society is changing quickly. We will continue to change and to be ever more effective. We've talked about some of the things we're bringing on board and looking forward to a brand-new website to improve our communication. And I think it's going to be a very exciting time.  

MF 

Professor Sir Ian Diamond, thanks very much for joining us.   That's it for another episode of Statistically Speaking, you can subscribe to future episodes of this podcast on Spotify, Apple podcasts and all the other major podcast platforms and also follow us on X, previously known as Twitter, via the @ONSFocus feed. I'm Miles Fletcher, and from myself, our producer Steve Milne, and everyone here at the ONS, we wish you seasonally adjusted greetings, goodbye.   ENDS 

The ONS led the way informing the UK response to the Coronavirus pandemic. But what lessons can be learned and how can we best prepare not only ourselves, but the rest of the world, for the next pandemic?  

 

Transcript 

MILES FLETCHER 

This is Statistically Speaking, the Office for National Statistics (ONS) Podcast. I'm Miles Fletcher, and as we approach the darkest months of winter, we're revisiting COVID-19.  

Now the ONS doesn't do predictions, and we're certainly not forecasting a resurgence of the virus, either here in the UK or anywhere else. But pandemic preparedness has been the driving force behind two important pieces of work that we're going to be talking about this time. Looking beyond our shores, how well equipped now is the world in general to spot and monitor emerging infections? We'll hear from Josie Golding of the Wellcome Trust on that, including how even weather events like El Nino could affect the spread of viruses. We'll also talk to my ONS colleague, Joy Preece about the pandemic preparedness toolkit, a five-year project backed by Wellcome to create and develop resources that will help countries with health surveillance in the event of future pandemics.  

But first, and closer to home, a new UK winter surveillance study to gather vital data on COVID-19 is now well underway. Jo Evans is its head of operations. Jo, this is a brand new COVID-19 survey the ONS is running in partnership with the UK Health Security Agency (UKHSA). What is the new survey and what's it going to be monitoring over the winter? 

JO EVANS 

So this is now the winter COVID infection study. And we're going to be going out to, I think we've got 145,000 people signed up, and we're going to ask them to take a lateral flow test to see if they are testing positive for COVID-19. Then we'll ask them to tell us a little bit about how they're feeling, what symptoms they have and some other household information - what work do they do? Do they have caring responsibilities? And so on.  

MF 

So we're gonna be getting people to take a test and everyone's familiar of course now with administering their own lateral flow test, that wasn't the case back in the early days of the pandemic, when it was a new thing for the vast majority of us. So they'll take a test that'll tell us whether they are positive or negative for COVID-19. And on top of that, we're going to be gathering data in the form of a questionnaire.  

JE 

That's right. And then this is a collaboration this time, so we'll be working with the UKHSA. I mean, we've worked with them on the COVID infection study before, but this time what we'll be doing is looking at those responses of how many people are telling us that they have COVID-19 And we'll be trying to understand that by where people live or their age group and so on, but we'll be sharing that information with UKHSA and they will then be looking at what the impact is on hospitals. So what they call the infection hospitalisation rate, how many people are going into hospital because they have COVID, so it'll really help us understand what pressures there are on the NHS over this winter period. 

MF 

And that will give us some inkling, once again, about how many people are infected but not actually displaying any symptoms? 

JE 

Absolutely. And we do ask people about their symptoms and if they tell us they test positive, we'll then be sending them a second questionnaire, a follow up, asking them to keep testing until they get two consecutive negative tests so that we can see how long they are testing positive, but we'll also ask them how long did their symptoms last and did they need to go and see a doctor, did they take any medication, so really trying to understand how they're experiencing that period of illness. 

MF 

So during those critical winter months, that'll give us some insight into what's really going on on-the- ground and in communities.  

JE 

That's right and we're running this study from November right through to March so that we can understand that, because COVID, unlike flu, it's not a seasonal virus, but we know that the NHS really suffers through the winter with those increased pressures, with more people needing their services. And this is about understanding what's happening out there. In the community, and what impact that is having on our healthcare services. 

MF 

Another very important aspect of that is we're going to be monitoring for people who say they're suffering the symptoms of what is popularly known as long-COVID, ongoing impacts of the virus, and that will fill a very important evidence gap won’t it. 

JE 

Absolutely. We will in a follow up questionnaire be asking people how long they've had COVID for and whether they have long-COVID. And interestingly, in some research we did when we were designing the questionnaire, long-COVID sufferers told us that they know precisely what date their symptoms started and how long they've had it because of the impact it's been having on their lives. So we are hopeful that this study will provide some really useful information. 

MF 

So 145,000 people taking part. Has it been difficult to get as many people as that involved? 

JE 

Do you know what, we got halfway there within the first 48 hours, people were so keen to take part in this study. We've really been surprised about that. 

MF 

It's probably a reflection of the success of the profile that the original study had. 

JE 

I think so, people are really keen to do their bit here and get involved in this study. And we've had a lot of participants, particularly in the older age groups, who have signed up so we will have to do something that we call ‘weighting of the data’ across the different age groups, but we do this all the time and we are also going out to those under 16s, right up to the over 70s. 

MF 

And as well as taking part in a very important public study, people get a COVID test for free and can see for themselves whether they've been affected. 

JE

Yeah, think that's one of the things people are keen to do, particularly over the winter periods when we're going to be mingling and visiting family, that reassurance really that you're going to test every month and find out whether or not you have COVID, I think we all want to make sure that we are virus free before we go and see our loved ones over Christmas, for example. 

MF 

Well, we're meeting to discuss this in mid-November. The first results are still a few weeks away but how are things going, we've got enough people? Are the tests out in the field yet? 

JE 

The tests are out in the field. I think we're looking to get two publications in before Christmas, so testing windows start next week. We're expecting around 25,000 people a week to take their tests and answer their questionnaire. 

MF 

And over the course of a month then, all 145 we hope will have been covered?  

JE 

Yes, I mean 145 is a fantastic number, and if we get all of them taking their test kits each month, then yes, that number will be higher. But even if we were looking at a 50 or 60% response rate, that is excellent for a social survey. 

MF 

Yes, and all the time, what we've heard in other contexts, is that it is difficult to get people to take part in surveys, but certainly in this case people can see the need for it and have come forward in their thousands. It's possibly worth pointing out though that you do have to be selected to take part, that's very important isn’t it, that we've never looked for volunteers. We've selected households randomly and that approach, that's very important to make this a really, really reliable survey isn't it? 

JE 

Yes, and as soon as there was information about the study in the newspapers earlier this year, we had people ringing up and asking to take part and we've had to explain to them that we want a nice random sample so that we can have a fully representative study. 

MF 

So ONS will be producing the figures then it's over to our colleagues in the UK Health Security Agency to interpret what that means from a public health point of view, and what response might be necessary. Absolutely. 

JE 

Absolutely. And they'll be producing some statistics as well. Looking particularly, as I said, at that infection hospitalisation rate. 

MF 

So are we expecting the virus to take off again, or is it just a just a precaution to be monitoring things in this way? 

JE 

When we started this, it was more about understanding if there would be that impact on the NHS over the winter. But then we did see back in September, a new variant, particularly in the US, and as you know, from looking at COVID over the past few years, when you see a new variant coming, sort of appearing in one country, you know that it will come here eventually. So, it's about keeping track of that really, although because we are doing lateral flow tests, we won't actually have information about what kind of variant people have, but it will just be to look to see whether we're seeing an increase in positivity in the community. 

MF 

Okay, so all eyes on the first result, and we wish you, and the team getting the survey together once again, every success on what is a highly valuable and important exercise. 

So we've heard how the new winter surveillance study is helping us track ongoing COVID infection here at home. But we're also using the experience the ONS gained during the last pandemic to prepare not only ourselves, but other countries around the world for another one, Josie, with that global perspective in mind, my first question to you just to get us started is what have been the biggest learnings, the biggest take homes if you like, for Wellcome from the pandemic. And what's your priority now as an organisation considering how best to respond to others? 

JOSIE GOLDING 

Thank you for having me today, I think this is good to be reflecting on COVID in the future. So the biggest take home message is, probably I can look at the positives and the negatives, so I'll go on the negatives first.  

So I think we had a lot of the tools for responding to outbreaks and bigger events but I think we weren't prepared to deal with such a massive pandemic that we saw at SARS-CoV-2, we had expected to prepare for something like influenza and of course we probably didn't use our imagination of how the impact would be so great, affecting people in so many different ways. I think we need to really use that imagination going forward, it’s about thinking through the variety of different impacts we could see across different populations. I think we've learned a lot on how we communicate with the public, with the key people who are involved, and take those lessons because I think we did struggle. I think globally, not just Wellcome as one of the actors on communicating the importance, and the push to be better prepared to respond to these pandemics.  

One of the successes, and I'll put this up from a Wellcome point of view, really was the true integration of research into the response. And you know, this has been building up for many years from the Ebola West Africa outbreak in 2014. And tested again, and tested and tested and refined, on how we do this across the small research community who are engaged in those relatively smaller outbreaks to now a complete game change on how people expect research to be integrated into outbreak responses through pandemics.  

So I think that's now set the new status quo, and before I had to convince people of the importance, I think the importance now speaks for itself.  

MF 

Yes, it was notable in the early stages of the pandemic, those countries, notably in East Asia that have had experience of major respiratory viruses, and dealing with those on a public health point of view, didn't seem to be much better prepared than us in the West, who perhaps have underestimated the risks? 

JG 

It is absolutely true. You know, it is testing the system over and over and over again. So you know who your stakeholders were, you knew how to get things done quickly and at speed. And I think that's the one piece we have to keep remembering that we can keep preparing, but you still need to keep testing the system to ensure that it works in practice. But through it all I would say, you know, one of the things that Wellcome is taken away from SARS-CoV-2 is really the belief that we can't predict exactly what's going to come next when it comes to emerging infectious diseases. We have to keep that in mind, but actually the way to test the system time and time again, is dealing with the health priorities right now. So things like antimicrobial resistance. We know this has been a growing threat for many years. It's had some setbacks through SARS-CoV-2 and the pandemic, you know, we need to really re-energise the community to really take this seriously and to finance and to conduct the research that's required. But there are other threats that are, you know, common health issues, common infectious diseases that countries are dealing with, and we should be integrating the readiness, haemorrhagic fevers, viral fevers, other viruses, whatever it may be, into how we deal with those everyday infectious diseases. 

MF 

And what's the legacy been from an analytical point of view of the first few years for the period that is now known as peak COVID? Have we got that to draw on now because we're seeing the virus continuing to emerge? We're seeing potential threats from new variants and possibly other viruses. 

JG 

I don't think it's evenly applied across the globe on taking advantage of the systems. The approaches that were built up during SARS-CoV-2, some countries are able to maintain some of the resources that have been built or pivot into other health priorities. But that is a bit of a gap that we are seeing. I'll give an example of what I think is a great statistic, you know, for pathogen genomic sequencing and how that was used to track variants and making that as close to real time as you could find through the accelerator and diagnostics working group that mapped out the capacity in countries to be able to conduct pathogen genetic sequencing. And I think at the time, this is going back to 2022, that 77% of the world's countries were able to conduct sequencing when that's a massive game change for a tool that really wasn't a, partly an add on, into how you would do some of the epidemiological research at the beginning of outbreaks. So I think being able to pivot that tool and make sure that these types of facilities and the training and the expertise that people have built up over time can be sustained, working with those communities to be able to identify what are the real use cases for pathogens. And so I think, yes, some of it has probably not been evenly distributed, but we could always be doing more to be able to ensure we can better understand the variants as they come about, but also, what does it mean for a variant you know, how, what changes will that make, what impact will that have on our health? 

MF 

Hearing the UK with our partners, the UK health security agency, we are preparing, as you well know I'm sure, to run a further study going into the winter. What is the role of studies all like that? Are they uniform now across the world or this is not as similar surveillance programmes going on? Or do we remain a bit of a one off in doing this in the UK? 

JG 

I don't believe that this is evenly spread around the world. We ourselves at Wellcome had made a decision to continue funding our SARS-CoV-2 work on the genomics as well as the characterization of these variants as they come out. And what difference does it make in people who've been vaccinated or with other health conditions? We know when we've engaged with the research community across a variety of countries around the world, it ends up being very novel that this research has continued to happen. So I do think there is a gap, and it is becoming more challenging for public health institutes, WHO and others, to gather this information to understand are the vaccines still effective when we have these new variants, are they more transmissible, and other impacts that we would assess for those new variants. So I do think it's becoming more limited, and so of course, we need to make sure that the data we do generate is of high quality. 

MF 

The focus has been very much on COVID, but of course as we've seen historically and in other countries, other viruses have emerged and have serious public health consequences in those countries globally. What other emerging diseases do we need to have our eye on at the moment? 

JG 

Since SARS-CoV-2 really picked up we've had a global impacts event that affected you know, very select communities around the world, and is still ongoing, but not to the same level. We have the ongoing threat of avian influenza, we have El Nino upon us, which is likely to further impact the rates of cholera that we're going to see as well as impacting temperatures, so mosquito borne viruses and other types of arboviruses, potentially broader than that, so it is happening right now. I don't think we even need to sit back and think what it could be. And there are many events that we need to be preparing for. And particularly with something like El Nino as a particular weather event but thinking about the climate crisis. This is only going to grow we need to really collect the evidence now to understand what difference will it make what risks will it pose by experience, and geographical distribution further afield. 

MF 

Yes, can you unpack that a bit for us, because most people will be aware of El Nino as a meteorological phenomenon. How does that translate into public health impacts? 

JG 

The whole background and where we've been watching and waiting for more certainty and whether this was actually going to happen this year, but it's a very high, I think it's now greater than 90% certainty it's going to happen from this part of the year onwards, and and it will vary depending on where in the region it will impact you for droughts or flooding. And of course we need to better understand well, what impact would that have on cholera? Cholera is a prime example. While it's not directly linked to El Nino as it stands right now, we have seen such a change in the cholera distribution in Malawi being a great example where it's seeing rises in cases outside of the expected weather event. So you'd expect it in flooding season but you're seeing it more in dry seasons. So El Nino will make this worse potentially. It's being able to track it and understand the issue we have with events like El Nino is that we don't have enough information on it. We need to be better from a researcher's perspective, we need to just understand the researchers in those countries that are likely to be affected, their opportunity to gather as much evidence about the impact of El Nino so that when it comes around again, we'll be better able to apply what we've learned now. 

MF 

Without wishing to sensationalise, what do you think the risks are of another big global pandemic of the sought we saw with COVID-19? 

JG

I’m a virologist by training so I'm always thinking that viruses hold the opportunity for some of the greatest opportunity for change. I'm always hopeful that you know these risks like the SARS-CoV-2 are actually quite rare events, to see something take off and to be able to transmit that successfully to humans. We see there are many events where we have what people refer to as spillover events between animals to humans, but it takes quite a change and we don't fully understand what the change might be or why it adapts for people to be more susceptible. So I think it is a risk, it’s a known risk even for SARS-CoV-2 which could change drastically. It's a very early stage in understanding this virus and how it operates. So I think we just have to be prepared, to be continually preparing, for the event that it could happen. I think influenza is the greatest one that it would be surprising if we didn't have a global event for influenza of some kind in the next few years. We've been preparing for this for quite some time. 

MF 

Is that the one that was anticipated then? Because if you look back historically, and this was the big comparison of course that was made with the COVID pandemic, it was so called Spanish flu wasn't it after the First World War, which was a huge global pandemic. 

JG 

Yes, it has been the one that we've always focused on. And if we look at the way that we've managed to monitor the change in the evolution of the virus, and to build the infrastructure globally, to be able to do the research and track that within laboratories and share information on that to help inform the vaccine production, which is very seasonal, influenza has changed every year. This is decades in the making. It's been ongoing for decades, and still, you know, we still have problems with making sure we have enough vaccine at the right time in the right supply to be accessible to all. So even for something that we know is likely to come we still struggle to get to that level of preparedness and it takes a lot of effort and time and it will continue, hopefully SARS-CoV-2 will help evolve those structures and I know a lot of the interest has been to combine, where we can, with coronaviruses respiratory like illnesses in the future to make it more efficient, but it's a big undertaking to really map out what you can do for a single pathogen. So, we have to work to see where we can build in those efficiencies across multi pathogen approaches. 

MF 

So one response and this is a project that you're working on with ONS and we're going to talk about now, and bring Joy in to explain to us, and that's the pandemic preparedness toolkit. The ONS is developing alongside Wellcome as I say, Joy, you're part of the team at ONS creating the toolkit just to take you from the very beginning, how did the ONS get involved in this and why is ONS well placed to facilitate this work? 

JOY PREECE 

Well, this was a proposal that we put together for Wellcome in the aftermath of, I think some of the early years of the pandemic response, and the of course well-known Coronavirus Infection Study (CIS) which I was part of, and I think what we really learned during COVID, the during big years, 2020 and 2021 in the UK, was how important really active data monitoring was when a disease is multiplying exponentially. That's really frightening stuff. You can't afford to just sit back and wait and see. So the key to any successful response has to be figuring things out like the reproduction number really, really quickly. You know, you needed to know how many people were affected, how fast infections were increasing, how the numbers of infections related to numbers of deaths, and we saw during COVID that it wasn't enough to just wait until people were already so ill that they were turning up in a healthcare setting in a hospital or you know, even worse kept to have systems that could be producing those kinds of statistical insights early from a community setting. And so that's the unique approach that ONS really took here. Linking up our statistical offices with the public health agencies and the decision makers. They're using our experience with surveys, with administrative data, with data modelling and data science, drawing on connections that we had with academics, with expert epidemiologists to try and get answers to those important questions as quickly as we could. And I think the unique thing that the ONS and other statistical offices around the world can bring to this is the very fact that we aren't part of the public health systems. So we bring here in ONS that expertise in a social research settings or community settings here, and you know, even apart from numbers of infections, there's topics like employment patterns, travel and tourism opinions and lifestyle habits, which tells you really important things about how people are interacting and behaving which gives you the ability to do some really, really clever modelling or things like disease communicability as that's kind of the background ONS brought to this from our experience during COVID. But, of course, we have experience as well supporting capacity building with overseas national statistical institutes. 

MF 

Now regular listeners to these podcasts will know we recently spoke to our colleagues in Ghana, about everything they're doing in partnership with our own, so there's countries like Ghana, who are very much part of this pandemic preparedness project as well.  

JP 

That's right. So what we are looking to do from the back of everything we learned in the UK is to go out and work with, initially we've identified that we want to be working with three different partner countries to co-create a toolkit that can be generalizable, and that can be accessed globally. And what's really important here is that it isn't about, oh, well here's a model that the UK used and therefore it's applicable to everybody because yes, we just heard from Josie that's not the case. Different countries have different contexts, different experiences of different diseases and have built different infrastructures and skills as a result. So what we really want to do is generate that pool of knowledge internationally and co-create a toolkit that allows countries, based on their unique context, to draw from it and we're talking here of practical guidance. Statistical Methods, knowledge products, case studies, training materials, really this is about capacity building to support that kind of infectious disease surveillance, but in a way that may look very different depending on the country’s context. So it's about an international community of practice here. 

MF 

Well, that's why it's a toolkit. Not a template for how to deal with a pandemic. 

JP 

That's absolutely right. So we're thinking about this under I think, three headings. So data collection on one hand, statistics and modelling on another but crucially, also the relationship building between statistical producers and public health professionals, and that's really essential, because you want to help toolkit users build those relationships of trust between analysts and the public health decision makers that needs to be already there. It needs to be there as a fundamental before a crisis hits because otherwise the opportunities for that kind of productive collaboration that ONS was able to do during the COVID pandemic, just become so so limited. 

MF 

And it’s about sharing of learnings as well, sharing of intelligence... 

JP 

No, absolutely, absolutely right. This is why I talk about it as a co-created toolkit. This is something that will be kind of jointly delivered in collaboration between ourselves and the three countries that we end up working with. We're going through a process right now to kind of identify volunteers and select countries that we'll work with, but also our first stage of that we are launching with a couple of kind of lessons learned workshops are where we're inviting statistical Institute's from around the world and a number of large international NGOs and experts in the field to come and talk about their experience of you know, disease surveillance of COVID and pandemic response and of other disease response. I start drawing out you know, what is there that we can find in common, what are there that are common challenges that have been common enablers to make your situation better, what is there that collectively we identify as the key criteria for a toolkit that will have the most value to the most countries. 

MF 

And Josie, from Wellcomes point of view, what are what are Wellcome’s ambitions for this piece of work? 

JG 

We're very keen to understand how that toolkit can be of value to others, but also think through what is that epidemic preparedness model? So how would you apply it in the future to whatever that disease may be? So we never know maybe during the lifespan of this project, there will be opportunities for the countries to be able to test it out to see what works for them in in real time. I mean, I hope there’s not another pandemic, but we have to just work on that assumption that as we go along, during this particular project, there could be something that might have to test it. But we'll see. 

MF 

There was of course much soul searching about the effectiveness of particularly early response to COVID in certainly this country, in the UK and in other Western countries. But, of course, looking back, and now we have the data, it was the global south that was disproportionately affected. 

JG 

It's fair to say, and I think there's still an unanswered question why some countries were affected more and some were less, and I think Joy has be very clear about the different contexts that countries operate in, but that includes also the populations as well and the other diseases they might have seen, what other health issues. I know, there's been less cases observed on the African continent and of course, that is down to the ability to test, but to a degree it is a different population structure that we're seeing. So yes, we want to make sure that these types of tools are equitably shared, and applied to whatever the health requirements are for their systems. And I think this is the exciting part of this project. And I guess my main kind of message that I repeat to everybody is the only way to prepare for a future pandemic or a future epidemic is simply to deal with the health issues that we have right now. So make sure that we're thinking about things like MMR, and thinking about the impact of climate and understanding better how it's changing the dynamics of infectious diseases such as mosquito borne, or other viruses or malaria, you know, the common issues that many countries are facing and just act now rather than just planning for something. I want to see some real tangible research and systems being built. I think that's why the ONS approach for this is important because it’s about just getting on with it. And not just, you know, coming up with a theoretical model, it is actually working with the countries to see how you're going to apply it now. So we have to just keep focusing on that it could be tomorrow. So just get going. 

JP 

Well, and I would just say aye because I completely agree Josie because it's very easy to get caught up in talking about a pandemic response. But of course, a pandemic response, you can only draw on the resources that are already there. There is no time in a crisis situation to be developing things that are substantially new. So what we're really talking about when we talk about a pandemic preparation is about supporting improved health statistics for all sorts of purposes. You know, data and modelling and communicating and understanding the statistical insight and actually having that really good disease, that has a multiplier benefit for a whole range of health outcomes. Whether or not we see a pandemic tomorrow, we should be planning, even if it doesn't happen tomorrow. And I think that's the critical thing in this, this isn't a once in 100 years. This is an event that is happening on a daily basis, when people are catching diseases and communicating diseases on a daily basis, and providing improved tools to support that has a benefit even in the absence of a large-scale event. 

MF 

And that's it for this episode of Statistically Speaking, next time, as the end of the year approaches, we'll be joined once again by the UK’s National Statistician. If you've got a question for us then please ask us via @ONSfocus on the ‘X’ social media platform, or Twitter for us traditionalists. 

Thanks to all of our guests today and our producer Julia Short. You can of course subscribe to new episodes of Statistically Speaking on Spotify, Apple podcasts and all other major platforms. 

ENDS

In this episode we talk about the growth of data use in the media and the potential impact of misinformation on the public’s trust in official statistics.  

Navigating podcast host Miles Fletcher through this minefield is Prof Sir David Spiegelhalter, from the University of Cambridge; Ed Humpherson, Head of the Office for Statistics Regulation; and award-winning data journalist Simon Rogers. 

 

Transcript 

 

MILES FLETCHER 

Welcome again to Statistically Speaking, the official podcast of the UK’s Office for National Statistics, I'm Miles Fletcher. Now we've talked many times before in these podcasts about the rise of data and its impact on our everyday lives. It's all around us of course, and not least in the media we consume every day. But ‘what’ or ‘who’ to trust: mainstream media, public figures and national institutions like the ONS, or those random strangers bearing gifts of facts and figures in our social media feeds? 

To help us step carefully through the minefields of misinformation and on, we hope, to the terra firma of reliable statistical communication, we have three interesting and distinguished voices, each with a different perspective. Professor Sir David Spiegelhalter is a well-known voice to UK listeners. He's chair of the Winton Centre for Risk Evidence Communication at the University of Cambridge and was a very prominent voice on the interpretation of public health data here during the COVID pandemic. Also, we have Ed Humpherson, Director General of regulation and head of the Office for Statistics Regulation (OSR), the official stats watchdog if you like, and later in this podcast, I'll be joined by award winning data journalist and writer Simon Rogers, who now works as data editor at Google. 

Professor, you've been one of the most prominent voices these last few years – a fascinating few years, obviously, for statistics in which we were told quite frankly, this was a golden age for statistics and data. I mean, reflecting on your personal experience as a prominent public voice in that debate, when it comes to statistics and data, to be very general, how well informed are we now as a public, or indeed, how ill-informed on statistics?  

 

DAVID SPIEGELHALTER 

I think things have improved after COVID. You know, for a couple of years we saw nothing but numbers and graphs on the news and in the newspapers and everywhere, and that went down very well. People didn't object to that. In fact, they wanted more. And I think that has led to an increased profile for data journalism, and there's some brilliant ones out there. I'm just thinking of John Burn-Murdoch on the FT but lots of others as well, who do really good work. Of course, in the mainstream media there is still the problem of non-specialists getting hold of data and getting it wrong, and dreadful clickbait headlines. It is the sub editors that wreck it all just by sticking some headline on what might be a decent story to get the attention and which is quite often misleading. So that's a standard problem. In social media, yeah, during COVID and afterwards, there are people I follow who you might consider as - I wouldn't say amateurs at all, but they're not professional pundits or media people - who just do brilliant stuff, and who I've learned so much from. There are also some terrible people out there, widespread misinformation claims which are based on data and sound convincing because they have got numbers in them. And that, I mean, it's not a new problem, but now it is widespread, and it's really tricky to counter and deal with, but very important indeed.  

 

MF 

So the issue aside from - those of us who deal with the media have heard this a hundred times - “I don't write the headlines”, reporters will tell you when you challenge that misleading kind of headline. But would you say it’s the mainstream media then, because they can be called out on what they report, who broadly get things right? And that the challenge is everything else - it's out there in the Wild West of social media?  

 

DS 

Yeah, mainstream media is not too bad, partly because, you know, we've got the BBC in this country, we’ve got regulations, and so it's not too bad. And social media, it's the Wild West. You know, there are people who really revel in using numbers and data to make inappropriate and misleading claims.  

 

MF 

Is there anything that can be done? Is it the government, or even those of us like the ONS who produce statistics, who should we be wading in more than we do? Should we be getting out there onto the social media platforms and putting people right?  

 

DS 

It's difficult I mean, I don't believe in sort of censorship. I don't think you can stop this at source at all. But just because people can say this, it doesn't give them a right for it to be broadcast wide, in a way and to be dumped into people's feeds. And so my main problem is with the recommendation algorithms of social media, where people will see things because it's getting clicks, and the right algorithm thinks persona will like it. And so we just get fed all this stuff. That is my real problem and the obscurity and the lack of accountability of recommendation algorithms right across social media is I think, a really shocking state of affairs. Of course, you know, we come on to this later, but we should be doing something about education, and actually sort of pre-empting some of the misunderstandings is something I feel very strongly about with my colleagues. You’ve got to get in there quick, and rather than being on the backfoot and just reacting to false claims that have been made, you've got to sort of realise how to take the initiative and to realise what misunderstandings, misinterpretations can be made, and get in there quickly to try to pre-empt them. But that of course comes down to the whole business of how ONS and others communicate their data.  

 

MF 

Because when you ask the public whether they trust them - and the UK statistics authority does this every two years - you ask the public if they trust ONS statistics, and a large proportion of them say they do. But of course, if they're not being presented with those statistics, then they're still going to end up being misled.  

 

DS 

Yeah, I mean, it's nice to get those responses back. But, you know...that's in terms of respondents and just asking a simple question, do you trust something or not? I think it's good to hear but we can't be complacent about that at all. I’m massively influenced by the approach of the philosopher, Baroness Onora O’Neill, who really makes a sharp distinction between organisations wanting to be trusted and revelling in being trusted, and she says that shouldn't be your objective to be trusted. Your objective should be to be trustworthy, to deserve trust, and then it might be offered up to you. And so the crucial thing is trustworthiness of the statistics system and in the communications, and that's what I love talking about, because I think it's absolutely important and it puts the responsibility really firmly back to the communicator to demonstrate trustworthiness.  

 

MF 

So doing more as stats producers to actually actively promote data and get people to come perhaps away from the social platforms, and to have their own websites that present data in an accessible way, in an understandable way, where people can get it for nothing without requiring an expensive subscription or something, as some of the best of the media outlets would require.  

 

DS 

The other thing I'd say is there's no point of being trustworthy if you’re dull, as no one's going to look at it or take any notice, and other media aren't going to use it. So I think it's really worthwhile to invest, make a lot of effort to make what you're putting out there as attractive, as vivid and as grabbing as possible. The problem is that in trying to do that, I mean, that's what a lot of communicators and media people want to do, because of course they want people to read their stuff. But what that tends to do largely is make their stuff kind of opinionated and have a very strong line, essentially to persuade you to either do something or think something or buy something or vote something. So much communication has to do with persuading that I think it's just completely inappropriate. In this context, what we should be doing is informing people. 

 

In a way we want to persuade them to take notice, so that's why you want to have really good quality communications, vivid, get good people out there. But in the end, they’re just trying to inform people, and that's why I love working with ONS. I just think this is a really decent organisation whose job is just trying to raise the...to obviously provide official statistics...but in their communications, it's to try to raise the level of awareness raise the level of discussion, and by being part of a non -ministerial department, they're not there, the comms department, to make the minister look good, or to make anyone look good. It’s just there to tell people how it is. 

 

MF 

Exactly. To put that data into context. Is this a big number or is this is a small number, right? Adjectives can sometimes be very unhelpful, but often the numbers don't speak for themselves, do they. 

 

DS  

Numbers never speak for themselves, we imbue them with meaning, which is a great quote as well from Nate Silver. 

 

MF 

And in doing that, of course, you have to walk the same line that the media do, in making them relevant and putting them into context, but not at the same time distorting them. There's been a big debate going on recently, of course, about revisions. And if you've listened to this podcast, which we'd always advise and consume other articles that the ONS has published, we've said a lot about the whole process of revising GDP, and the uncertainty that's built into those initial estimates, which although helpful, are going to be pretty broad. And then of course, when the picture changes dramatically, people are kind of entitled to say, oh hang on, you told us this was something different and the narrative has changed. The story has changed because of that uncertainty with the numbers, shouldn't you have done more to tell us about that uncertainty. That message can sometimes get lost, can’t it?  

 

DS 

Yeah, it's terribly important. You’ve got to be upfront. We develop these five points on trustworthy communication and the first one was inform, not persuade. And the second is to be balanced and not to have a one-sided message to tell both sides of the story, winners and losers, positives and negatives. And then to admit uncertainty, to just say what you don't know. And in particular, in this case, “provisionality”, the fact that things may change in the future, is incredibly important to emphasise, and I think not part of a lot of discussion. Politicians find it kind of impossible to say I think, that things are provisional and to talk about quality of the evidence and limitations in the evidence, which you know, if you're only basing GDP on a limited returns to start with, on the monthly figures, then you need to be clear about that. And the other one is to pre-empt the misunderstandings, and again, that means sort of getting in there first to tell you this point, this may change. This is a provisional judgement, and you know, I think that that could be emphasised yet more times, yet more.  

 

MF 

And yet there's a risk in that though, of course the message gets lost and diluted and the... 

 

DS 

Oh no, it always gets trotted out - oh, we can't admit uncertainty. We can't tell both sides of story. We have to tell a message that is simple because people are too stupid to understand it otherwise, it's so insulting to the audience. I really feel a lot of media people do not respect their audience. They treat them as children - oh we've got to keep it simple, we mustn't give the nuances or the complexity. All right, if you're going to be boring and just put long paragraphs of caveats on everything, no one is going to read that or take any notice of them. But there are ways to communicate balance and uncertainty and limitations without being dull. And that's what actually media people should focus on. Instead of saying, oh, we can't do that. You should be able to do it. Good media, good storytelling should be able to have that nuance in. You know, that's the skill.  

 

MF 

You’re absolutely right, you can't disagree with any of that, and yet, in communicating with the public, even as a statistics producer, you are limited somewhat by the public's ability to get used to certain content. I mean, for example, the Met Office recently, a couple of years back, started putting in ‘percentage of chance of rainfall’, which is something that it hadn't done before. And some work on that revealed just how few people actually understood what they were saying in that, and what the chances were actually going to be of it raining when they went out for the afternoon’s work.  

 

DS 

Absolute nonsense. That sorry, that's completely I mean, I completely rely on those percentages. My 90-year-old father used to understand those percentages. Because it’s a novelty if you are going to ask people what they understand, they might say something wrong, such as, oh, that's the percentage of the area that it's going to rain in or something like that. No, it's the percentage of times it makes that claim that it's right. And those percentages have been used in America for years, they're completely part of routine forecast and I wouldn't say the American public is enormously better educated than the British public. So this is just reluctance and conservatism. It's like saying oh well people don't understand graphs. We can't put up line graphs on the news, people don't understand that. This is contempt for the public. And it just shows I think, a reluctance to make an effort to explain things. And people get used to stuff, once they've learned what a graph looks like, when they see it again, then they'll understand it. So you need to educate the public and not, you know, in a patronising way, it's just that, you know, otherwise you're just being misleading. If you just say, oh, you know, it'll rain or not rain you're just misleading them. If you just say it might rain, that's misleading. What does that mean? It can mean different things. I want a percentage and people do understand them, when they've got some experience of them.  

 

MF 

And what about certainty in estimates? Here is a reaction we add to the migration figures that ONS published earlier in the summer. Somebody tweeted back to say, well estimates, that’s all very good but I want the actual figures. I want to know how many people have migrated. 

 

DS 

Yeah, I think actually, it's quite a reasonable question. Because, you know, you kind of think well can’t you count them, we actually know who comes in and out of the country. In that case it’s really quite a reasonable question to ask. I want to know why you can't count them. And in fact, of course ONS is moving towards counting them. It's moving away from the survey towards using administrative data to count them. So I think in that case, that's quite a good question to ask. Now in other situations, it's a stupid question. If you want to know if someone says, oh, I don't want an estimate of how many people you know, go and vote one way or do something or other, I want to know how many, well then you think don’t be daft. We can't go and ask everybody this all the time. So that's a stupid question. So the point is that in certain contexts, asking whether something is an estimate or not, is reasonable. Sometimes it's not and that can be explained, I think, quite reasonably to people.  

 

MF 

And yet, we will still want to be entertained. We also want to have numbers to confirm our own prejudices.  

 

DS 

Yeah, people will always do that. But that's not what the ONS is for, to confirm people's prejudices. People are hopeless at estimating. How many, you know, migrants there are, how many people, what size ethnic minorities and things, we know if you ask people these numbers, they’re pretty bad at it. But people are bad at estimating all numbers. So no, it's ONS’s job to try to explain things and in a vivid way that people will be interested in, particularly when there's an argument about a topic going on, to present the evidence, not one side or the other, but that each side can use, and that's why I really feel that the ONS’s migration team, you know, I have a lot of respect for them, when they're changing their format or consulting on it, they go to organization's on both sides. They go to Migration Watch and the Migration Observatory and talk to them about you know, can they understand what's going on, is this data helping them in their deliberations.  

 

MF 

Now, you mentioned earlier in the conversation, education, do we have a younger generation coming up who are more stats literate or does an awful lot more need to be done?  

 

DS 

A lot more needs to be done in terms of data education in schools. I'm actually part of a group at the Royal Society that is proposing a whole new programme called mathematics and data education, for that to be put together within a single framework, because a lot of this isn't particularly maths, and maths is not the right way or place to teach it. But it still should be an essential part of education, understanding numbers, understanding data, their limitations and their strengths and it uses some numeracy, uses some math but it's not part of maths. The problem has always been where does that fit in the syllabus because it doesn't, particularly at the moment. So that's something that every country is struggling with. We're not unique in that and, and I think it's actually essential that that happens. And when you know, the Prime Minister, I think quite reasonably says people should study mathematics until 18. I mean, I hope he doesn't mean mathematics in the sense of the algebra and the geometry that kids do, get forced to do essentially, for GCSE, and some of whom absolutely loathe it. And so, but that's not really the sort of mathematics that everyone needs. Everyone needs data literacy. Everyone needs that. 

 

MF 

Lies, damned lies and statistics is an old cliche, it's still robustly wheeled out in the media every time, offering some perceived reason to doubt what the statisticians have said. I mean looking ahead, how optimistic are you, do you think that one day we might finally see the end of all that?  

 

DS 

Well my eyes always go to heaven, and I just say for goodness sake. So I like it when it's used, because I say, do you really believe that? You know, do you really believe that, because if you do you're just rejecting evidence out of hand. And this is utter stupidity. And nobody could live like that. And it emphasises this idea somehow, among the more non-data-literate, it encourages them to think that numbers they hear either have to be sort of accepted as God given truths or rejected out of hand. And this is a terrible state to be in, the point is we should interpret any number we hear, any claim based on data, same as we’d interpret any other claim made by anybody about anything. We’ve got to judge it on its merits at the time and that includes do we trust the source? Do I understand how this is being explained to me? What am I not being told? And so why is this person telling me this? So all of that comes into interpreting numbers as well. We hear this all the time on programmes like More or Less, and so on. So I like it as a phrase because it is so utterly stupid, then so utterly, easily demolished, that it encourages, you know, a healthy debate. 

 

MF 

We're certainly not talking about good statistics, we're certainly not talking about quality statistics, properly used. And that, of course, is the role of the statistics watchdog as we're obliged to call him, or certainly as the media always call him, and that's our other guest, Ed Humpherson. 

  

Ed, having listened to what the professor had to say there, from your perspective, how much misuse of statistics is there out there? What does your organisation, your office, do to try and combat that?  

  

ED HUMPHERSON 

  

Well, Miles the first thing to say is I wish I could give you a really juicy point of disagreement with David to set off some kind of sparky dialogue. Unfortunately, almost everything, if not everything that David said, I completely agree with - he said it more fluently and more directly than I would, but I think we are two fellow travellers on all of these issues.  

  

In terms of the way we look at things at the Office for Statistics Regulation that I head up, we are a statistics watchdog. That's how we are reported. Most of our work is, so to speak, below the visible waterline: we do lots and lots of work assessing reviewing the production of statistics across the UK public sector. We require organisations like the ONS, but also many other government departments, to be demonstrating their trustworthiness; to explain their quality; and to deliver value. And a lot of that work just goes on, week in week out, year in year out to support and drive-up evidence base that's available to the British public. I think what you're referring to is that if we care about the value and the worth of statistics in public life, we can't just sort of sit behind the scenes and make sure there's a steady flow. We actually have to step up and defend statistics when they are being misused because it's very toxic, I think, to the public. Their confidence in statistics if they're subjected to rampant misuse or mis explanation of statistics, it's all very well having good statistics but if they go out into the world and they get garbled or misquoted, that I think is very destructive. So what we do is we either have members of the public raise cases with us when they see something and they're not they're not sure about it, or indeed we spot things ourselves and we will get in contact with the relevant department and want to understand why this thing has been said, whether it really is consistent with the underlying evidence, often it isn't, and then we make an intervention to correct the situation. And we are busy, right, there's a lot there's a lot of there's a lot of demand for work. 

  

MF 

Are instances of statistical misuse on the rise? 

  

EH 

We recently published our annual summary of what we call casework - that's handling the individual situations where people are concerned. And we revealed in that that we had our highest ever number of cases, 372, which might imply that, you know, things are getting worse. I'd really strongly caution against that interpretation. I think what that increase is telling you is two other things. One is, as we as the Office for Statistics Regulation, do our work, we are gradually growing our profile and more people are aware that they can come to us, that’s the first thing this is telling you; and the second thing is that people care a lot more about statistics and data now, exactly as Sir David was saying that this raised profile during the pandemic. I don't think it's a sign that there's more misuse per se. I do think perhaps, the thing I would be willing to accept is, there's just a generally greater tendency for communication to be datafied. In other words, for communication to want to use data: it sounds authoritative, it sounds convincing. And I think that may be driving more instances of people saying well, a number has been used there, I want to really understand what that number is. So I would be slightly cautious about saying there is more misuse, but I would be confident in saying there's probably a greater desire to use data and therefore a greater awareness both of the opportunity to complain to us and of its importance.  

  

MF 

Underlying all of your work is compliance with the UK code of practice for statistics, a very important document, and one that we haven't actually mentioned in this podcast so far…  

  

EH 

Shame on you, Miles, shame on you.  

  

MF 

We're here to put that right, immediately. Tell us about what the code of practice is. What is it for? what does it do?  

  

EH 

So the Code of Practice is a statutory code and its purpose is to ensure that statistics serve the public good. And it does that through a very simple structure. It says that in any situation where an individual or an organisation is providing information to an audience, there are three things going on. There's the trustworthiness of the speaker, and the Code sets out lots of requirements on organisations as to how they can demonstrate they're trustworthiness. And it's exactly in line with what David was saying earlier and exactly in line with the thinking of Onora O’Neill – a set of commitments which demonstrate trustworthiness. Like a really simple commitment is to say, we will pre-announce at least four weeks in advance when the statistics are going to be released, and we will release them at the time that we say, so there is no risk that there's any political interference in when the news comes out. It comes out at the time that has been pre-announced. Very clear commitment, very tangible, evidence-based thing. It's a binary thing, right? You either do that or you do not. And if you do not: You're not being trustworthy. The second thing in any situation where people are exchanging information is the information itself. What's its quality? Where's this data from? How's it been compiled? What are its strengths and limitations? And the code has requirements on all of those areas. That is clarity of what the numbers are, what they mean, what they don't mean. And then thirdly, in that exchange of information, is the information of any use to the audience? It could be high, high quality, it could be very trustworthy, but it could, to use David's excellent phrase, it could just be “dull”. It could be irrelevant, it could not be important. And the value pillar is all about that. It's all about the user having relevant, insightful information on a question that they care about. That's, Miles, what the Code of Practice is: it’s trustworthiness, it’s quality and it’s value. And those things we think are kind of pretty universal actually, which is why they don't just apply now to official statistics. We take them out and we apply them to all sorts of situations where Ministers and Departments are using numbers, we always want to ask those three questions. Is it trustworthy? Is it quality, is it value? That's the Code.  

  

MF 

And when they've satisfied your stringent requirements and been certified as good quality, there is of course a badge to tell the users that they have been.  

  

EH 

There's a badge - the badge means that we have accredited them as complying with that Code of Practice. It's called the National Statistics badge. The term is less important and what it means what it means is we have independently assessed that they comply in full with that Code.  

  

MF 

Most people would have heard, if they have heard of the OSR’s work, they'll have seen it perhaps in the media. They'll have seen you as the so-called data watchdog, the statistics watchdog. It's never gently explained as it it's usually ‘slammed’, ‘criticised’, despite the extremely measured and calm language you use, but you're seen as being the body that takes politicians to task. Is that really what you do? It seems more often that you're sort of gently helping people to be right.  

  

EH 

That's exactly right. I mean, it's not unhelpful, frankly, that there's a degree of respect for the role and that when we do make statements, they are taken seriously and they're seen as significant, but we are not, absolutely not, trying to generate those headlines. We are absolutely not trying to intimidate or scare or, you know, browbeat people. Our role is very simple. Something has been said, which is not consistent with the underlying evidence. We want to make that clear publicly. And a lot of time what our intervention does actually is it strengthens the hand of the analysts in government departments so that their advice is taken more seriously at the point when things are being communicated. Now, as I say, it's not unwelcome sometimes that our interventions do get reported on. But I always try and make these interventions in a very constructive and measured way. Because the goal is not column inches. Absolutely not. The goal is the change in the information that's available to the public.  

  

MF 

You're in the business of correcting the record and not giving people a public shaming.  

  

EH 

Exactly, exactly. And even correcting the record actually, there's some quite interesting stuff about whether parliamentarians correct the record. And in some ways, it'd be great if parliamentarians corrected the record when they have been shown to have misstated with statistics. But actually, you could end up in a world where people correct the record and in a sort of tokenistic way, it's sort of, you know, buried in the depths of the Hansard parliamentary report. What we want is for people not to be misled, for people to not think that, for example, the number of people in employment is different from what it actually is. So actually, it's the outcome that really matters most; not so much the correction as are people left understanding what the numbers actually say.  

  

MF 

Surveys show - I should be careful using that phrase, you know - nonetheless, but including the UKSA survey, show that the public were much less inclined to trust in the words of the survey. Politicians use of statistics and indeed, Chris Bryant the Labour MP said that politicians who have been who've been found to have erred statistically should be forced to apologise to Parliament. Did you take that on board? Is there much in that? 

  

EH 

When he said that, he was actually directly quoting instances we've been involved with and he talks about our role very directly in that sense. Oh, yeah, absolutely. We support that. It will be really, really good. I think the point about the correction, Miles, is that it shows it's a manifestation of a culture that takes fidelity to the evidence, truthfulness to the evidence, faithfulness to the evidence, it takes that seriously, as I say, what I don't want to get into is a world where you know, corrections are sort of tokenistic and buried. I think the key thing is that it's part of an environment in which all actors in public debate realise it's in everybody's interests or evidence; data and statistics to be used fairly and appropriately and part of that is that if they've misspoken, they correct the record. From our experience, by and large, when we deal with these issues, the politicians concerned want to get it right. What they want to do is, they want to communicate their policy vision, their idea of the policy or what the, you know, the state of the country is. They want to communicate that, sure, that's their job as politicians, but they don't want to do so in a way that is demonstrably not consistent with the underlying evidence. And in almost all cases, they are… I wouldn’t say they're grateful, but they're respectful of the need to get it right and respect the intervention. And very often the things that we encounter are a result of more of a cockup than a conspiracy really - something wasn't signed off by the right person in the right place and a particular number gets blown out of proportion, it gets ripped from its context, it becomes sort of weaponized; it's not really as a deliberate attempt to mislead. Now, there are probably some exceptions to that generally positive picture I'm giving. but overall it's not really in their interests for the story to be about how they misuse the numbers. That's not really a very good look for them. They'd much rather the stories be about what they're trying to persuade the public of, and staying on the right side of all of the principles we set out helps that to happen.  

  

MF 

Your remit runs across the relatively controlled world sort of government, Parliament and so forth. And I think the UK is quite unusual in having a body that does this in an independent sort of way. Do you think the public expects you to be active in other areas, we mentioned earlier, you know, the wilder shores of social media where it's not cockup theories you're going to be hearing there, it's conspiracy theories based on misuse of data. Is there any role that a statistics regulator could possibly take on in that arena?  

  

EH 

Absolutely. So I mentioned earlier that the way we often get triggered into this environment is when members of the public raised things with us. And I always think that's quite a solemn sort of responsibility. You know, you have a member of the public who's concerned about something and they care about it enough to contact us - use the “raise a concern” part of our website - so I always try and take it seriously. And sometimes they're complaining about something which isn't actually an official statistic. And in those circumstances, even if we say to them, “well, this isn't really an official statistic”, we will say, “but, applying our principles, this would be our judgement”. Because I think we owe it to those people who who've taken the time to care about a statistical usage, we owe it to take them seriously. And we have stepped in. Only recently we're looking at some claims about the impact of gambling, which are not from a government department, but from parts of the gambling industry. We also look at things from local government, who are not part of central government. So we do we do look at those things, Miles. It's a relatively small part of our work, but, as I say, our principles are universal and you've got to take seriously a situation in which a member of the public is concerned about a piece of evidence.  

 

MF 

Professor Spiegelhalter, what do you make of this regulatory function that the OSR pursues, are we unusual in the UK in having something along those lines?  

 

DS 

Ed probably knows better than I do, but I haven't heard of anybody else and I get asked about it when I'm travelling and talking to other people. I have no conflict of interest. I'm Non-Executive Director for the UK Stats Authority, and I sit on the regulation committee that oversees the way it works. So of course, I'm a huge supporter of what they do. And as described, it's a subtle role because it's not to do with performing, you know, and making a big song and dance and going grabbing all that attention but working away just to try to improve the standard of stats in this country. I think we're incredibly fortunate to have such a body and in fact, we know things are never perfect and there's always room for improvement of course, but I think we're very lucky to have our statistical system.  

 

MF 

A final thought from you...we’re at a moment in time now where people are anticipating the widespread implementation of AI, artificial intelligence, large language models and all that sort of thing. Threat or opportunity for statistics, or both? 

 

DS 

Oh, my goodness me, it is very difficult to predict. I use GPT a lot in my work, you know, both for sort of research and making inquiries about stuff and also to help me do codings I'm not very good at. I haven't yet explored GPT-4's capacity for doing automated data analysis, but I want to, and actually, I'd welcome it. if it's good, if you can put some data in and it does stuff - that's great. However, I would love to see what guardrails are being put into it, to prevent it doing stupid misleading things. I hope that that does become an issue in the future, that if AI is automatically interpreting data for example, that it's actually got some idea of what it's doing. And I don't see that that's impossible. I mean, there were already a lot of guardrails in about sexist statements, racist statements, violent statements and so on. There's all sorts of protection already in there. Well, can’t we have protection against grossly misleading statistical analysis?  

 

MF 

A future over the statistics watchdog perhaps? 

 

DF 

Quite possibly. 

 

EH 

Miles, I never turn down suggestions for doing new work.   MF 

So we’ve heard how statistics are regulated in the UK, and covered the role of the media in communicating data accurately, and now to give some insight into what that might all look like from a journalist’s perspective, it’s time to introduce our next guest, all the way from California, award-winning journalist and data editor at Google, Simon Rogers. Simon, welcome to Statistically Speaking. Now, before you took up the role at Google you were actually at the forefront of something of a data journalism movement here in the UK. Responsible for launching and editing The Guardian’s data blog, looking at where we are now and how things have come on since that period, to what extent do you reckon journalists can offer some kind of solution to online misinterpretation of information? 

 

Simon Rogers   At a time when misinformation is pretty rampant, then you need people there who can make sense of the world and help you make sense of the world through data and facts and things that are true, as opposed to things that we feel might be right. And it's kind of like there is a battle between the heart and the head out there in the world right now. And there are the things that people feel might be right, but are completely wrong. And where, I think, Data Journalists can be the solution to solving that. Now, having said that, there are people as we know who will never believe something, and it doesn't matter. There are people for whom it literally doesn't matter, you can do all the fact checks that you want, and I think that is a bit of a shock for people, this realisation that sometimes it's just not enough, but I think honestly, the fact that there are more Data Journalists now than before...There was an EJC survey, the European Journalism Centre did a survey earlier this year about the state of data journalism. There are way more data journalists now than there were the last time they did the survey. It's becoming much more...it’s just a part of being a reporter now. You don't have to necessarily be identified as a separate data journalist to work with data. So we're definitely living in a world where there are more people doing this really important work, but the need, I would say it has never been greater.  

 

MF 

How do you think data journalists then tend to see their role? Is it simply a mission to explain, or do some of them see it as their role to actually prove some theories and vindicate a viewpoint, or is it a mixture, are there different types of data journalists?  

 

SR 

I would say there were as many types of data journalists as there are types of journalists. And that's the thing about the field, there's no standard form of data journalism, which is one of the things that I love about it. That your output at the end of the day can be anything, it can be a podcast or it can be an article or a number or something on social media. And because of the kind of variety, and the fact I think, that unlike almost any other role in the newsroom, there really isn't like a standard pattern to becoming a data journalist. As a result of that, I think what you get are very different kind of motivations among very different kinds of people. I mean, for me, personally, the thing that interested me when I started working in the field was the idea of understanding and explaining. That is my childhood, with Richard Scarry books and Dorling Kindersley. You know, like trying to understand the world a little bit better. I do think sometimes people have theories. Sometimes people come in from very sophisticated statistical backgrounds. I mean, my background certainly wasn't that and I would say a lot of the work, the stats and the way that we use data isn't necessarily that complicated. It's often things like, you know, is this thing bigger than that thing? Has this thing grown? You know, where in the world is this thing, the biggest and so on. But you can tell amazing stories that way. And I think this motivation to use a skill, but there are still those people who get inured by maths in the same way that I did when I was at school, you know, but I think the motivation to try and make it clear with people that definitely seems to me to be a kind of a common thread among most of the data journalists that I’ve met. 

 

MF 

Do you think that journalists therefore, people going into journalism, and mentioning no names, as an occupation...used to be seen as a bit less numerous, perhaps whose skills tended to be in the verbal domain. Do you think therefore these days you’ve got to have at least a feel for data and statistics to be able to be credible as a journalist?  

 

SR 

I think it is becoming a basic skill for lots of journalists who wouldn't necessarily consider themselves data journalists. We always said eventually it is just journalism. And the reason is because the amount of sources now that are out there, I don't think you can tell a full story unless you take account of those. COVID’s a great example of that, you know, here's a story that data journalists, I think, performed incredibly well. Someone like John Burn-Murdoch on the Financial Times say, where they’ve got a mission to explain what's going on and make it clear to people at a time when nothing was clear, we didn't really know what was going on down the road, never mind globally. So I think that is becoming a really important part being a journalist. I mean, I remember one of my first big data stories at the Guardian was around the release of the coins database – a big spending database from the government - and we had it on the list as a “data story” and people would chuckle, snigger a little bit of the idea that there'll be a story on the front page of the paper about data, which they felt to be weird, and I don't think people would be snickering or chuckling now about that. It's just normal. So my feeling is that if you're a reporter now, not being afraid of data and understanding the tools that are there to help you, I think that's a basic part of the role and it's being reflected in the way that journalism schools are working. I teach here one semester a year at the San Francisco Campus of Medill. There's an introduction to data journalism course and we get people coming in there from all kinds of backgrounds. Often half the class are just, they put their hands up if they're worried about math or scared of data, but somehow at the end of the course they are all making visualisations and telling data stories, so you know, those concerns can always be overcome.  

 

MF 

I suppose it's not that radical a development really if you think back, particularly from where we're sitting in the ONS. Of course, many of the biggest news stories outside of COVID have been data driven. think only of inflation for example, the cost of living has been a big running story in this country, and internationally of course, over the last couple of years. Ultimately, that's a data driven story. People are relying on the statisticians to tell them what the rate of inflation is, confirming of course what they're seeing every day in the shops and when they're spending money.  

 

SR 

Yeah, no, I agree. Absolutely. And half of the stories that are probably about data, people don't realise they're writing about data. However, I think there is a tendency, or there has been in the past, a tendency to just believe all data without questioning it, in the way that as a reporter, you would question a human source and make sure you understood what they were saying. If we gave one thing and that thing is that reporters would then come back to you guys and say ask an informed question about this data and dive into a little bit more, then I think we've gained a lot.  

 

MF 

So this is perhaps what good data journalists are bringing to the table, perhaps and ability to actually sort out the good data from the bad data, and actually, to use it appropriately to understand uncertainty and understand how the number on the page might not be providing the full picture. 

 

SR  

Absolutely. I think it's that combination of traditional journalistic skills and data that to me always make the strongest storytelling. When you see somebody, you know, who knows a story inside out like a health correspondent, who knows everything there is to know about health policy, and then they're telling a human story perhaps about somebody in that condition, and then they've got data to back it up - it’s like the near and the far. This idea of the near view and the far view, and journalism being the thing that brings those two together. So there’s the view from 30,000 feet that the data gives you and then the individual view that the more kind of qualitative interview that you get with somebody who is in a situation gives you. The two things together - that’s incredibly powerful. 

 

MF 

And when choosing the data you use for a story I guess it’s about making sound judgements – you know, basic questions like “is this a big number?”, “is this an important number?”  

 

SR 

Yeah, a billion pounds sounds like a lot of money, but they need to know how much is a billion pounds, is it more about a rounding error for the government. 

 

MF 

Yes, and you still see as well, outside of data journalism I stress, you still see news organisations making much of percentage increases or what looks like a significant increase in something that's pretty rare to start with.  

 

SR 

Yeah, it's all relative. Understanding what something means relatively, without having to give them a math lesson, I think is important.  

 

MF 

So this talk about supply, the availability of data journalism, where do people go to find good data journalism, perhaps without having to subscribe? You know, some of the publications that do it best are after all behind paywalls, where do we find the good stuff that's freely available? 

 

SR 

If I was looking from scratch for the best data journalism, I think there are lots of places you can find it without having to subscribe to every service. Obviously, you have now the traditional big organisations like the Guardian, and New York Times, and De Spiegel in Germany, there is a tonne of data journalism now happening in other countries around the world that I work on supporting the Sigma Data Journalism Awards. And over half of those entries come from small one or two people units, you know, practising their data journalism in countries in the world where it's a lot more difficult than it is to do it in the UK. For example, Texty in Ukraine, which is a Ukrainian data journalism site, really, and they're in the middle of a war zone right now and they're producing data journalism. In fact, Anatoly Barranco, their data editor, is literally in the army and on the frontline, but he’s also producing data journalism and they produce incredible visualisations. They've used AI in interesting ways to analyse propaganda and social media posts and stuff. And the stuff happening everywhere is not just limited to those big partners behind paywalls. And what you do find also, often around big stories like what’s happened with COVID, people will put their work outside of the paywall. But um, yeah, data is like an attraction. I think visualisation is an attraction for readers. I'm not surprised people try and monetize that, but there is enough going on out there in the world. 

 

MF 

And all that acknowledged, could the producers of statistics like the ONS, and system bodies around the world, could we be doing more to make sure that people using this data in this way have it in forms have it available to be interpreted? Is there more than we can do?  

 

SR 

I mean, there was the JC survey that I mentioned earlier, it’s definitely worth checking out because one thing it shows is that 57% of data journalists say that getting access to data is still their biggest challenge. And then followed by kind of like lack of resources, time pressure, things like that. PDFs are still an issue out there in the world. There's two things to this for me, on one side it's like, how do I use the data, help me understand what I'm looking at. On the other side is that access, so you know, having more kind of API's and easy downloads, things that are not formatted to look pretty but formatted for use. Those kinds of things are still really important. I would say the ONS has made tremendous strides, certainly since I was working in the UK, on accessibility to data and that's a notable way, and I've seen the same thing with gov.us here in the States. 

 

MF 

Well it’s good to hear the way the ONS has been moving in the right direction. Certainly I think we've been tough on PDFs.   SR Yes and to me it's noticeable. It's noticeable and you've obviously made a deliberate decision to do that, which is great. That makes the data more useful, right, and makes it more and more helpful for people.  

 

MF 

Yes, and at the other end of the chain, what about storing publishers and web platforms, particularly well you’re at Google currently, but generally, what can these big platforms do to promote good data journalism and combat misinformation? I mean, big question there. 

 

SR 

Obviously, I work with Google Trends data, which is probably the world's biggest publicly available data set. I think a big company like Google has a responsibility to make this data public, and the fact that it is, you can download reusable datasets, is incredibly powerful. I'm very proud to work on that. I think that all companies have a responsibility to be transparent, especially when you have a unique data set. That didn't exist 20 years earlier, and it's there now and it can tell you something about how the world works. I mean, for instance, when we look at something like I mean, I've mentioned COVID before, but it's such a big event in our recent history. How people were searching around COVID is incredibly fascinating and it was important information to get out there. Especially at a time when the official data is always going to be behind what's actually happening out there. And is there a way you can use that data to predict stuff, predict where cases are going to come up... We work with this data every day and we're still just scratching the surface of what's possible with it. 

 

MF  

And when it comes to combating misinformation we stand, so we're told, on the threshold of another revolution from artificial intelligence, large language models, and so forth. How do you see that future? Is AI friend, foe, or both?  

 

SR 

I work for a company that is a significant player in the AI area, so I give you that background. But I think in the field of data, we've seen a lot of data users use AI to really help produce incredible work, where instead of having to read through a million documents, they can get the system to do it for them and pull out stories. Yeah, like any other tool, it can be anything but the potential to help journalists do their jobs better, and for good, I think is pretty high. I'm going to be optimistic and hope that that's the way things go. 

 

MF 

Looking optimistically to the future then, thank you very much Simon for joining us. And thanks also to my other guests, Professor Sir David Spiegelhalter and Ed Humpherson. Taking their advice on board then, when we hear or read about data through the news or experience it on social media, perhaps we should first always ask ourselves – do we trust the source? Good advice indeed.  

 

You can subscribe to new episodes of this podcast on Spotify, Apple podcasts, and all the other major podcast platforms. You can also get more information, or ask us a question, by following the @ONSFocus on X, or Twitter, take your pick. I’m Miles Fletcher, from myself and our producer Steve Milne, thanks for listening.   ENDS 

In this episode, we explore ONS’s work with other countries to raise the world's statistical capabilities. 

 

Transcript 

MILES FLETCHER 

Hello and welcome again to ‘Statistically Speaking’, the Office for National Statistics’ Podcast. I'm Miles Fletcher, and in this episode we're going international.  

Now it hardly needs saying that global issues, climate change, population growth, inflation, to name a few are best understood and addressed with the benefit of good global statistics. So, to that end, the ONS works in partnership with a number of countries worldwide with the ultimate aim of raising the world's statistical capabilities. At the one end of Africa, for example, a continent where it’s deeply involved, that includes embedding state of the art inflation indices and other economic data in Ghana. On the other side of the continent, it's meant using AI and machine learning to track the movement of displaced populations in Somalia. How do you run a census in places where nobody has a permanent address?  

It's all fascinating work and here to tell us about it, Emily Poskett, Head of International Development at the ONS; Tim Harris of the ONS Data Science Campus’s international development team; and joining us from Accra, our special guest, Government Statistician of the Republic of Ghana and head of the Ghanaian Statistical Service Professor Samuel Annim. 

Emily then, to start give us the big overview if you would, set out for us the scale and the purpose of this international development work that the ONS is doing. 

 

EMILY POSKETT 

We work with countries around the developing world to support strong modern statistical systems wherever we see a suitable opportunity to do so. 

  

MF 

What form does that work take? Does it mean statisticians going out to these countries? 

  

EP 

Yes, it does, when that's the right way to go about things. So our work is usually through the form of medium-term partnerships with a small group of national statistical offices, or NSOs, from the developing world, and those partnerships are medium term over a number of years in order to build up a real understanding of the context in that country, that national statistical office’s vision for modernization and how the ONS can be of most help to achieve their own goals under their own strategy.  

That relationship will normally be led by a particular individual who spends time getting to know the context and getting to know the people, getting to know what ONS can do to help. A partnership might cover a range of topic areas from census to data science to leadership training to economic statistics, and the lead point of contact, the strategic advisor in many cases, will bring in the relevant experts from across ONS, and they'll work through virtual collaboration but also through on-site visits, and they will work out the best timing for those and the best delivery modality in order to ensure that the gains are sustained. Our primary focus really is to make sure that changes that we support in the partner organisation are sustainable, and the work that ONS does using the UK’s aid budget is really impactful and leads to long term change.  

We don't always work through direct partnerships, for example where we see opportunities to work alongside other organisations, so international institutions like the World Bank or other national statistics offices like Statistics Canada or Statistics Sweden, they might choose to bring us in to deliver small pieces of focussed technical assistance alongside their own programmes. One of our medium-term partnerships is with the United Nations Economic Commission For Africa (UNECA), and they work with all 54 countries of Africa, and they can choose to bring in our expertise alongside their own to target particular needs in particular countries. But I would say that 70% of our effort is through these medium-term partnerships. 

  

MF 

So the ONS is providing one part of a large patchwork of work, going on right across the developing world, but what is the ultimate purpose of that? What are all these countries trying to achieve together? 

  

EP 

Well, strong statistical systems are essential in all countries to aid effective planning and informed decision making. And this is even more important in developing countries where resources are often scarce and you're trying to use scarce resources to target a wide range of needs across the population. And that resource might include UK aid for example, and aid from other countries. The UK has been statistical capacity building for many, many years through different modalities, working with partners, and the ONS is just one implementing partner who can be called upon to provide that technical expertise. We're really proud to be a partner of choice for a number of developing countries and the ONS is seen worldwide as being a leader. We're really proud that countries like Ghana would choose to work with us, and that we want to do our bit to help them to achieve their own strategy and their own goals. 

  

MF 

Well, this seems like an excellent moment to bring in Professor Samuel Annim. Our great pleasure, great honour, to have you with us professor. From your perspective, and what you're looking to achieve in the Ghana Statistical Service, how important how useful is the work with ONS been for you? 

  

PROFESSOR SAMUEL ANNIM 

From the perspective of how it has been important for us, I mean, I look at it from several aspects. I got into office in 2019,  a year after the ONS and GSS collaboration had been established. And when I joined obviously, I had a sense of what I wanted to contribute to the office. Partnership that we've seen between National Statistical Offices over the years have always taken the dimension of statistical production partnerships, and what I simply mean by that is that they’re going in to help the service deliver on its core mandate. So for example, if price statistics are the priority, then that is the area you want to focus on, but our partnership with ONS took a different dimension. In addition to focusing on the traditional mandate of the Institute, which is the production of statistics, we really have over the period achieved some milestones from the perspective of transformation, which is of high priority to me, and secondly, from the perspective of injecting technology or contemporary ways of dispensing our duty as a National Statistical Office. So from an individual point of view, it has it has been beneficial to the mission that I have, and since then we have kept on working in the area of transformation. 

  

MF 

Listening to what you have to say there, it does sound as though some of the big challenges you face at the moment are not too dissimilar from the ones faced by ONS, all about modernising statistics, particularly using big data and new technology. 

  

SA 

Indeed, and I must say that it is a wave across all national statistical offices, because we are now trying to complement traditional surveys and censuses with non-traditional data sources i.e. Big Data, administrative data, citizens generated data and other geospatial resources. So collaborating is the key thing here, because this is new to the statistical community. So it's important we collaborate to learn how you are dealing with issues that are not consistent with the production of official statistics. Now as a global community, we are all thinking about how to use citizens generated science, I mean, getting citizens to provide us with data. And this is an area in which there isn't any National Statistical Office that can claim authority, because the approach and the processes are pretty not consistent with the guidelines for production of official statistics. So it's important to learn how countries are doing it and see how we can collaborate to get this done. 

  

MF 

Yes, in the last episode of our podcast, interestingly, we talked about the challenges of getting our citizens here in the UK to take part in surveys. Are Ghanaians friendly to what you're trying to achieve? Or are they perhaps sceptical as well and difficult to engage? 

  

SA 

I wouldn't say they're sceptical, I think they really feel part of it. And that is one of the strengths of citizen generated data, because if you package it in a way that it is more demand driven, rather than supply, you don't just go and tell them ‘do this because I know how it's supposed to be done’, but instead give them the platform to tell the National Statistical Office what their experiences are, provide them with platforms that they can easily engage so that they can feel part of the process and they really own the product. In our case, it is not a product that is owned by the statistical service but it is a product that is owned by the sub national agencies, and that is, as I said earlier, the beauty of citizen generated data. It is co-creation and co-ownership of the statistical product. So they are not sceptical, they are very receptive to it, and they are getting a better understanding of what we do as a National Statistical Office. 

  

MF 

Thinking internationally, thinking globally, what sort of shape do you think the world's statistical system is in now, as a result of partnerships like this or other developments, generally looking across Africa and looking beyond Africa, when we think about key issues, particularly climate change - how good is the statistical system now in tracking these very important changes, and the impacts they're having? 

  

SA 

We have as national statistical offices been very content with the traditional statistics - labour statistics, price statistics, GDP - and you do that either monthly, quarterly, or in some instances annually, and even the social indicators, I mean, it's only a few countries like the UK that has been able to do social indicators annually, for those of us in the Global South, a lot of the social indicators are being collected every five years, or every seven to eight years. So this was the way national statistical offices, up until about 2017 or 2018, were shaped. But with the data revolution that we saw around 2014, and since the World Development Report, the data for better lives document, that came out in 2021, clearly, we now have to approach statistics from a different point of view.  

And this is simply asking the question, how do I contextualise the statistics beyond what international communities would be expecting national statistical offices to do? I mean, now we are doing everything possible to ensure that we have a monthly GDP, and this is something that we are also learning from the partnership with ONS, because we are aware that they are developing models to ensure that beyond GDP they have some indicators that would readily give us insight on economic performance.  

And related to the issue of climate change that you are you talking about Miles, it's one of the areas that you cannot simply dispense your statistics in that one area as a standalone National Statistical Office, because this is something that has a continental dimension, something that has a global dimension. And at the moment we have data sitting in different silos, and the only thing that we can do is through partnership, see how we can bring these datasets together to help us get a better understanding of issues around climate change.  

So going forward, in my point of view, if we really want to sustain the transformation that we are seeing as a global Statistical Office, the only way out is through partnership, is through collaboration. And one of the things that I'm putting on the table is that we better begin to measure partnerships. Because we've treated partnerships as a qualitative engagement. And really, nobody knows which partnerships are working and which are not working. So if we're able to measure it, we can more clearly see the benefits of partnerships, although we all hold the view that it is the way to go. 

  

MF 

Interesting what you said about how we've traditionally concentrated on those classical measures of economic progress, and notably GDP. You might be interested to hear that the charity Oxfam, the big NGO, was in the news here in the UK recently when they said that GDP was ‘colonialist’, and it was ‘anti-feminist’, because it ignored the huge economic value of unpaid work, which they said is largely undertaken by women.  

Well, whether you agree with that or not, it does perhaps highlight the need for going beyond GDP and producing these alternative, and perhaps richer, wider measures of economic progress and economic value. 

  

SA 

I mean, I clearly associate with that submission, and we currently doing some work with the United Nations Development Programme on the National Human Development Report. And the focus of this report is exactly what you are talking about, Miles. We are looking at the current value of work, and we are looking at the future value of work. And we are going beyond the definition of who is employed, which strictly looks at whether the work that you are doing comes with remuneration or not, because once you broaden it and look at the value of work, you definitely have the opportunity to look at people who are doing unpaid work, and indeed their contribution to the progress that we are seeing as a human society, and the National Human Development Report has a sharp focus on this gender issue. They're going to look at that closely. And again, this is coming on the backdrop of an ongoing annual household income and expenditure survey that we are doing. So traditionally, government and international organisations would ask what is your employment and what is your unemployment rate? And then in this report, we tell them that we need to begin to look at those who are working, but we see they're not employed, simply because they are not working for pay or profit, and the proportion of people who are in there, and then once you disaggregate based on sex, age and geography, it's so revealing that we are losing a number of insights from the perspective of unpaid work. And so I fully subscribe to that view. 

  

MF 

That's interesting. Professor, for now thank you very much, and I hope you'll join the conversation again later, but we're going to broaden out to talk about, well, it’s actually a related development, Emily, talking about women and unpaid work, that's been another theme of ONS’s work with the UN Economic Commission for Africa. 

  

EMILY POSKETT 

There was a request put forward by national statistic offices around Africa to undertake leadership training, and this was part of the country's modernization vision. Countries recognise that in order to achieve modernisation, they need to have strong leadership. So they asked the UNECA to deliver leadership training and ONS partnered with UNECA to design and pilot this leadership training programme in a range of countries. As part of delivering that we noticed and recognised a lack of female leaders in a number of National Statistics Offices around the continent, and thought with partners about what we can do to help support that, so now as well as running a leadership training for the top tier of leadership in in each organisation, we also run a women into leadership training for potential future female leaders from within the staff. And it's been really, really successful. Some of the feedback that we've had from leaders in those organisations is that they've seen their female staff becoming more confident, more outspoken, more ambitious, putting themselves forward for positions, putting their ideas forwards as well, and generally feeling more confident to contribute in the workplace. We're really proud of that success and hope to roll it out in many more countries around the continent. 

  

MF 

A country that's the other side of Africa in a number of important senses, and that is Somalia, which of course if you've followed the news to any extent over the last few decades, you'll know the serious turmoil that's affected that part of Africa, Tim Harris, bringing you in, what's been going on in Somalia that the ONS has been involved in, particularly when it comes to measuring population and population movement. Tell us about that. 

  

TIM HARRIS 

Well, as you say, Somalia is a very fragile context. It's been affected by conflict and climate change and environmental issues for many years. And that's made it very challenging to collect information, statistical information, on a range of things. But particularly on population, which is a key underpinning piece of statistics which any country needs, and in fact, there hasn't been a census in Somalia since the 1970s, almost 50 years now, but there are plans to do a census next year, with support from the UN Population Fund, UNFPA, and other various institutions in Somalia and development partners, as well as the Foreign Commonwealth and Development Office. And Emily's international development team are also trying to form a partnership, or are forming a new partnership, with the Somalia National Bureau of Statistics. So there are plans to do a census next year, and we're really in the preparatory phase for that at the moment. And we're looking to see in our team how we can use data science, new techniques, new data sources, to try and help prepare to run that census.  

One of the particular issues in Somalia is that there are significant numbers of people who are displaced from where they usually live, by the conflict, by climate change. There's also been a drought for the last few years, and so there are hundreds of thousands, in fact millions, of people who are displaced from where they usually live. They tend to congregate in what we call Internally Displaced People Camps, or IDP camps. So they're not refugees, they haven't crossed an international boundary, but they are displaced from where they usually live. And these IDP camps tend to be quite fluid and dynamic. They're often in areas that are difficult to get to, so information on them is very difficult to obtain. They change very quickly, they grow, they contract, and a lot of them are on private land. So we're looking to see whether we can use new data techniques, and new data sources, to give us information about the broad scale of population in that area. 

  

MF 

And those new data sources are necessary presumably because it's very difficult to actually get out and physically see these people in those areas and count them physically. 

  

TH 

That's right, and they change very quickly. So if you're running a census, you want to know where your people are, so you can send the right number of enumerators to the right places, you can draw boundaries of enumeration areas and so on. 

  

MF 

You need an address register essentially, but these are people effectively without addresses. 

  

TH 

That's right. That's the way that you do it in the UK. It's not possible in many of these IDP camps in Somalia. So we're looking to see whether we can use high resolution satellite imagery, which we can task for a particular period of time, say in the next week or the next month. And we use that satellite imagery to see whether we can identify structures on the ground in these camps. And in fact, the UN Population Fund has been doing this in a very manual way for some time now. So someone looks at the satellite imagery, and they put a point on each tent, and they try and count the tents and the structures within these camps. That's obviously a very time-consuming way of doing it. So we're looking to see can we do it in a more automated way. 

So we've procured some satellite data and we've developed what we call training data. So in certain parts of the camps, we manually draw around the outlines of the tents, and we use techniques called machine learning. We show computers what areas are tents and what areas are not tents. And we try and train them using these algorithms to be able to predict in areas they haven't seen which areas are tents and which areas are not tents. So we're trying to develop models where we can use high resolution satellite imagery to predict the areas where there are tents, to produce numbers of tents, and in this way, we can help to estimate the broad numbers of people in these areas, and that can feed into the preparations for the census to help it run more smoothly and more efficiently. 

  

MF 

And trying to count an ever-shifting population allowance, in that we've got seasonal variations going on, some people unfortunately being evicted, and then you've got a population that would be nomadic anyway. 

  

TH 

People in these camps, they’re a whole mixture of people, some who've been forced to move because of drought, some have been forced to move because of conflict, and as you say, there are large numbers of nomadic people as well. And they have tended to also congregate in these IDP camps in recent times because of the drought and other climate conditions. 

  

MF 

And it's thought there could be up to 3 million people at the moment living under those conditions in Somalia. 

  

TH 

That's right. I mean, that I think highlights one of the particular issues, in that the numbers are very uncertain. So there is some information from camp management administrative data, there is some information from some limited surveys that the Somalia National Bureau of Statistics has undertaken, but the estimates from those two different sources produce very, very different results. And so this is what we're trying to do, to see whether objectively we can count the number of tents, and therefore have some objective measure of at least the number of tents and structures, obviously then we need to move to how many of those tents and structures might be occupied. How many people on average might be in each of those tents and structures. But can we add something to the information context that produces some more objective measures, at least of the number of tents and the number of structures in those areas. 

  

MF 

Well, that's the kind of cutting-edge stuff that the Data Science Campus is all about. But Emily, the ONS has been involved in other censuses in Africa over the longer term, hasn't it? 

  

EMILY POSKETT 

Yes, that's right, Miles. So we've been involved in a number of censuses around Africa, including Ghana, also Kenya, Rwanda and a number of other countries through our partnership with the UNECA. And we've been able to really support countries to move from using paper for data collection to using tablets for data collection, during what they call the 2020 census round. And between ONS and the UNECA, we've been able to support on a number of different aspects, including how to make the most of that tablet technology. So you don't just move from using paper to using tablets and do the same processes. There are a number of advantages to using tablets in terms of how you can monitor the quality of the data coming in in real time, and how you can speed up that data collection and that data processing, and we've been able to work with NSOs around the continent on that.  

We've also been using modern data matching techniques to support countries with their post enumeration surveys, which is a way of testing and improving the quality of the census. We've also been working with partners on using data visualisation and new techniques for improving the dissemination and user engagement with the products coming out of the census and therefore increasing the value of the census data products. 

  

MF 

It's interesting what you say about introducing tablet technology for data gathering in the field, to be honest, it’s not that long ago that the ONS actually moved to use that, rather than the traditional what was once described as ‘well-meaning people with clipboards’ going around asking questions. And it strikes me that in developing and working with these partner countries, the sort of methods the sort of technology being introduced, is not far behind where we're at really is it? 

  

EP 

No, absolutely. The ONS experts that get involved in these projects really learn a huge amount from the partners that they're working with as well, because often the partners we're working with have far fewer resources to deliver on similar goals. So the staff have to be incredibly innovative and use all sorts of different techniques and resources in order to achieve those goals. And people coming from ONS will learn a huge amount by engaging with partners. 

  

MF 

Well, on the modernisation theme, the census is another area where we've been working with Ghana isn't it, Professor?  

  

PROFESSOR SAMUEL ANNIM 

That's correct. We had support in all three phases of the census engagement, that is before the data collection, during the data collection, and after the data collection. We were very clear in our minds that we were going to use tablets for the data collection. And one of the things that we didn't know, or struggled with, had to do with the loading of the materials onto the tablet for the data collection.  

Our original plan would have taken us about six months or four months to do that. And it wasn't going to be new for Ghana. We had other countries that had taken that length of time just to load the materials onto the tablet to enable the data collection exercise. And through the ONS and UNECA collaboration, we got technical assistance to provision the tablets in a much shorter duration. If I recall correctly, it took about six weeks to get all the items on the tablet. And we had been using tablets for data collection, but we hadn’t been able to do remote real-time data monitoring because we didn't have a dashboard. We didn't know how to develop it. And through the partnership we were able to get a dashboard. The benefit of that was that after 44 days of exiting the field, we were able to put out a preliminary report on the census because during every day of the census we had a good sense of what the numbers were, whatever corrections we had to make we were making them. So after 44 days of exiting the field, we were able to announce the preliminary result. 

  

MF 

Wow, you had a provisional population total after 40 days? 

 

SA 

44 days, yes.  

 

MF 

44 days! Well, that puts certain countries to shame, I think, but anyway, let's not dwell on that. That's very impressive, Professor. And there's another project you've been working on, which I suppose is close to your heart as an economist, and that's the production of CPI? Modernising that? 

  

SA 

Absolutely. Absolutely. One of the first things that happened when I took office in 2019 was as part of the partnership, I visit ONS to understand what they are doing and how the collaboration can be deepened. And one of the things that we explored, and that was the first time I had heard of it, was how to produce a reproducible analytical pipeline. And all that simply means is that if you keep on doing something over and over again, you should think about automating the process. And that is the relationship that we have when it comes to CPI now. We have completely moved away from Excel. When I got in I said in addition to excel, let's use data, because when the process is not automated, and you have heavy dependence on human beings doing it, the likelihood of error is high. So we really bought into this and now we do our traditional ways, Excel, data, and then we do the reproducible analytical pipeline to compare the results. And ultimately, we're going to move away from the traditional XLS database and rely on this automated process. And this again, would allow us to hopefully reduce the length of time that it takes. So that is the extent to which we are modernising our CPI based on our collaboration. 

  

MF 

That’s very impressive. And so through the process of speeding up the lag time of those regular indicators, you get a much timelier picture of what's going on in the economy. 

  

EMILY POSKETT 

This is one of the areas we will be working with a number of partners on. This idea of using new technology to deliver reproducible analytical pipelines and really, this is where national statistics offices around the world, but particularly those with low resources, can really utilise new technology to save time and improve quality. And this is something that we're really excited to be working with a number of different offices on, on a number of different topics, to really save human resources and ultimately improve quality. 

  

MF 

Tim, bringing you in... 

  

TIM HARRIS 

I think this really illustrates one of the other benefits of data science. We've talked a lot about the mobile phone records, call detail records, and their use in Ghana for producing mobility statistics, talked about using satellite imagery and machine learning in Somalia. But data science, and the tools of the digital age, can do a lot more of the basic underpinning work in statistical modernisation really well, and I think we really need to focus on that and see where we can benefit from that. And the work that Professor Annim talks about, about automating the CPI, I think that is really important. For that we can use the tools of coding, lessons from software engineering, like version control, and auditing processes, to really help us to get much greater efficiencies in these key statistical processes which any statistics office undertakes.  

And we've been very pleased to work with the Ghana Statistics Service on automating their consumer price index. I think that we're seeing that it's speeded up the process, it's reduced the scope for human error, it’s enabled us to put in quality assurance checks. The process has enabled us to produce much more transparent processes, and processes that can be maintained over time, because people can understand and see what's been done rather than things being hidden in a black box. So this process of automating statistical processes is really important. 

I think the way we've engaged with the Ghana Statistics Service also highlights what we're trying to do in terms of building capacity for people within statistics offices to do this work for themselves. So partly we've done some of the work to help them automate. But we've also tried to build the capacity of Professor Annim and his colleagues so that they can then do this work themselves and take it forward, and not only within the consumer price index, but also seeing how they can plan more strategically about how this work can be done in other areas of statistics production. 

  

MILES FLETCHER 

Emily, what are the priorities for the future of this international development work of the ONS? 

  

EMILY POSKETT 

So our priority for the next phase of this work is to continue with the partnerships that we have and to build new partnerships. So Tim mentioned that we are working towards a new partnership with the Somalia National Bureau of Statistics. We're also considering new partnerships in Tanzania and Zimbabwe, to add to the ones we already have in Ghana, Rwanda, Kenya, Namibia and with the UNECA. And that's just in Africa. We're also looking to see what we can do to support in other regions. We have a partnership with Jordan, and a new one with the Palestinian Central Bureau of Statistics, and we're looking to do more in that region and beyond into Asia and the Pacific as well, but also looking to consolidate the kind of topics which we've worked on previously. So we've mentioned census, data science, women into leadership training, open SDG platform, and we're also looking to do more in new topic areas. So we're looking to do more in climate and environment statistics. We think this is a really important area that we're looking to do more in, in geography and geographical disaggregation of data. 

And I think we're looking to do more really on the usability of statistical outputs and dissemination of statistical outputs. I think a number of our partners do a really great job of collecting data, but there's a lot more that can be done to make use of new technology to better disseminate and improve the use of that data. So we're ambitious in the reach that we have with our small budget, but we want to make sure that we don't lose sight of sustainability, and by spreading ourselves too thinly, we could reduce the sustainability of the work that we do, and I think we're forever trying to balance off those two things. 

  

MF 

Professor Annim, perhaps I could give you the last word on this. How do you see the future of collaboration between the ONS and Ghana? 

  

PROFESSOR SAMUEL ANNIM 

We really want to push the collaboration beyond the two statistical agencies, and let me indicate that that’s started already. One of the things that we want to achieve is more utilisation of our data. I mean, we are fine with the production of it. We are technical people. We can continue to improve on it. But what I see with this partnership is to scale our relationship as two national statistical offices. Our relationship should be scaled up to the data users. So we don't want to just sit as two statistical offices, improving the production of statistics, but really getting into the realm of the utilisation of statistics, and that is where we need to bring in other government agencies, based on what ONS and GSS are nurturing. 

  

MF 

There you have it, statistics are important, but it's outcomes that really matter.  

That's it for another episode of ‘Statistically Speaking’, thanks once again for listening.  

You can find out more about our international development work, read case studies and view our ambitious strategy, setting up the ONS’s vision for high quality statistics to improve lives globally, on the ONS website, ONS.gov.uk, and you can subscribe to new episodes of this podcast on Spotify, Apple podcasts and all the other major platforms.  

You can also get more information, or ask us a question, by following @ONSFocus on Twitter. 

I'm Miles Fletcher and our producer at the ONS is Alisha Arthur. Until next time, goodbye 

 

ENDS 

In this episode we chat to members of the ONS Social Survey Collection Division about the importance, and challenges, of getting the general public to take part in crucial surveys that help paint a picture of what life is like across Britain.

 

Transcript 

MILES FLETCHER 

Welcome again to ‘Statistically Speaking’, the official podcast of the UK’s Office for National Statistics. I’m Miles Fletcher.  

Now I don’t know about you - but it seems hardly a moment passes these days when we are not being asked to feed back. How was our service today? Are you satisfied with this product?  Please fill in this short survey. Your responses matter. 

Well, forgive the natural bias, but today we’re talking about surveys that really do matter.  

ONS surveys – some of which are the very largest conducted regularly in the UK – don’t just inform economic and social policy, though they are hugely important to it. The data they gathered also represent a public resource of immense and unique value.  

But persuading people – some unaware, some sceptical and even hostile, others just very busy – to take part in them is a growing challenge for statistical institutions worldwide.      

In this episode then we’ll be discussing how the ONS gathers often personal data from members of the public right up and down the country.  

Taking time out of their day to answer my questions, and to explain why it’s absolutely crucial that you participate in our surveys if you get the opportunity to do so, are Emma Pendre and Beth Ferguson, who head up the ONS’s face-to-face Field Operations;  

and sharing their own personal experiences of life on other people’s doorsteps we have two of the ONS’s top Field Interviewers, Tammy Fullelove and Benjamin Land. 

Welcome to you all. 

Emma, if I come to you first – give us an idea of what exactly the field community in ONS is. Who are you and what do you do?  

EMMA PENDRE 

The Social Survey Collection Division is the largest division in ONS. We primarily collect data from households either online, face to face or by telephone using computer assisted interviewing, and also work at air, sea and rail ports collecting data from passengers. All the data collected is used to produce quite a number of our key ONS publications which help to paint a picture of what life is like in the UK. And these can include things like estimates of employment and unemployment, how we measure inflation, how we measure migration, and a key topic of interest at the moment is the cost of living. So while most of ONS relies on the data that we collect for our outputs and statistical bulletins, the statistics that we particularly generate also support research, policy development and decision making across government and other private sector businesses as well.  

MF 

Now Beth, bringing you in here, when it comes to household surveys, presumably someone's deciding which households are going to be approached to take part. Who makes those decisions and how is it done?   BETH FERGUSON So I'm not going to pretend to understand the clever people in the statistics team who work out how we get the right people to cover a broad spectrum of society. But yes, that's done by the sampling team and they choose a random sample for the surveys.  

MF 

And that's generated presumably from using the electoral roll.  

BF 

It's generated from something called PAF which is the Post Office Address Finder. I'll have to confirm exactly what that stands for. Yes, but essentially, it's a list of addresses across England, Scotland and Wales.  

MF 

And when it comes to the passenger survey, it's a question of stopping what we hope will be a representative random sample of people as they pass through those ports.  

BF 

Yes, it is. Yeah. But at the moment we're currently working on departures and arrivals. So yes, it's a random sample of individuals stopped and asked questions.  

MF 

But to make the data really representative and really valid, of course, we've got to be covering the whole of the country. The country in this case being Great Britain.  How do we ensure that that coverage is working day in day out?  

BF 

That's our role as the kind of management of the face-to-face field interviewers. Different surveys are done over different frequencies. So we've got the Labour Force Survey and the transformed Labour Force Survey which addresses are issued for on a weekly basis and those surveys are delivered on a weekly basis. And then we've also got our other longer, more detailed financial surveys that we’re issued with a quota for on a monthly basis. So our job is to make sure we've got the right people, in the right places, to knock on the right doors, to get hold of those members of the public and, you know, encourage them to agree to complete surveys for us.  

MF 

And luckily for us we’re joined by two of those “right people” here today. Tammy and Benjamin, welcome to our humble podcast. Now you are both at the sharp end of our survey data collection, working as field interviewers.  I'm obviously really interested in what you do day to day, but first off tell us how you got into this line of work. What was the attraction for you Tammy, how did you become a field interviewer?  

TAMMY FULLELOVE 

So prior to working for the ONS - I've never worked in public sector before, I've always worked in the private sector - and I've actually got a finance background. But then after being on maternity leave, having a young family, seeing the job advertised and the flexibility working with people in a very, varied job sort of pulled me to it to apply to be honest. And that was seven years ago, and I can honestly say I enjoy every single day I'm out in the field. It's great.  

MF 

And Benjamin how about you, what was your background?  

BENJAMIN LAND 

Well, I've done a variety of hospitality jobs in the past. I then applied to work on the Census at the start of 2021. And my manager at the time she had worked previously for the ONS on the basket of goods figures, and she recommended it as a really great place to work. It's funny how timing happened I saw a vacancy for a field interviewer, which I applied. And then I started in May 2021. So almost two and a half years ago now.  

MF 

Okay, so you've both got quite a bit of experience already under your belt. I was wondering of both of you, is there such a thing as a typical day for a field interviewer?  

TF 

I can honestly say no, every day is completely different. Depending on the area where you go into, where you may be working, streets apart, houses apart. You never know what door you knock on who can be behind that door, which makes every day completely varied, especially with the studies that you may be interviewing for, that they can be very different with the content. So yeah, two days are never the same.  

BL 

I totally agree with Tammy. It varies. My week has a sort of flow to it. So I tend to get out quite a lot at the start of the week to visit various addresses. If it’s LFS they change every week. On the financial surveys it's monthly so you've got longer to familiarise yourself with the area. We tend to have a team meeting most Tuesday mornings just to check in and see how we're doing. And then obviously interviews are scheduled around respondents’ timetable so that can be any time up to sort of eight, nine o'clock at night and sometimes Saturdays, if that's when they're available.  

MF 

Going out to people's houses on a daily basis, you no doubt encounter a wide variety of people. That must have led to one or two amusing episodes.   TF I've had occasion where people will answer the door in not the most suitable attire, shall we say, for public viewing. I don't know how much further to go into this, but yeah, definitely opening the door in towels which have fallen off and dressing gowns which haven't been completely covered. It definitely happened a couple of times over the past few years.   

MF 

Perhaps that's what they mean by “raw data”.

Beth, if I can come back to you, are there particular surveys which are considered to be especially important for us to be speaking to people in their homes in the way we’ve just been talking about? Ones that perhaps can’t be carried out in other ways. 

BETH FERGUSON 

It’s the more detailed financial surveys. So we've got the Family Resources Survey, the Living Costs and Food Survey, the Survey of Living Conditions, and the Household and Assets survey. They are quite long, more detailed surveys. The living costs and food survey, that requires the respondent to complete an interview, but then they also have to get hold of all their receipts of any expenditure for a two week period and annotate them and hand those over to the interviewer. So it's quite a detailed, involved survey. The Household and Assets survey, again it’s dependent on how many people in the household can, you know, take up to two hours to complete and ask lots of detailed financial questions around savings and pensions and other things. If you're in the home, you can ask them to get the documents, support them to review the documents, make sure that they're actually giving the right information which, if they were to go online and do it themselves, there's no guarantee that they would get the right detail that we're actually looking for.  

MF 

So it's quite an intensive experience really, isn't it compared with simply asking someone to tick the boxes on a webpage? And I guess it very much depends on building a personal rapport with the survey participants? 

BF 

Absolutely. And that's the key. That's the key to a really successful interviewer is that ability to build rapport in a snapshot on the doorstep. You know, before they've had the opportunity to give a polite no, no thank you or sorry, not today. They reckon it is approximately 10 seconds on the doorstep to get that engagement and build that rapport, and then maintain that through what can sometimes be quite a lengthy interview. Keep that friendliness, that rapport going so that the person being interviewed remains engaged and keen to do it.  

MF 

Now Tammy, you’ve already told us about your previous financial background. Do you find that helps you when you're collecting data on economics or topics around money?  

TAMMY FULLELOVE 

Yes, I do. Like Beth’s already mentioned, a couple of our financial studies go into people's income and expenditure. So having that sort of background I feel does help me, especially when they're speaking about what benefits they receive, what sort of things they pay out. It definitely does sort of give me the edge I do feel.   

MF 

That’s great, because it’s no secret is it Beth that the ONS, like other statistical organisations around the world, are finding it increasingly challenging to get people to take part in surveys.  

BETH FERGUSON 

Yeah, absolutely. I think it's got more and more challenging. Pre-pandemic it was getting more challenging, but the shift during and post-pandemic has been quite significant in terms of the number of willing people to do surveys for us. 

MF 

A shift in what direction? 

BF 

Fewer members of the public are willing to actually do surveys for us. Now whether that's because there's less trust in the government or actually, because of the pandemic, everybody's working from home and time is more limited. But no, it's definitely harder to get a response now.  

MF 

What techniques do we use then to try and change people's minds to get them to take part? 

BF 

At the moment we're doing a lot of work, certainly with the face-to-face field community - we're calling it a Respondent Engagement Programme. So looking out for clues and signs from, you know, when you approach the doorstep in the area, identifying the kind of things that may be key to them. Our statistics on things like CPI and RPI and, you know, the change in cost of food - that being constantly in the news gives us, kind of like, a lever to start an open conversation on the doorstep, particularly when we're looking at the financial surveys.  

EMMA PENDRE 

And also Miles. It's worth noting that all the surveys are voluntary, so the offer of incentives such as vouchers in exchange for the time taken to complete a survey will also continue to be significantly influential in maintaining our response rates.  

MF 

Absolutely Emma - offering people a small incentive has actually been proven to work hasn’t it, and I guess in cost terms, it's better to spend some money on that rather than wasting it on chasing people who are never going to take part.  

EP 

Yes, that's right. The vouchers are very significant. They do help maintain our response rates. And again, being in a cost of living crisis at the moment. Our respondents see them as very helpful.  

MF 

But even with incentives, and as Beth has suggested, there’s still a reluctance by some people to be involved in our surveys. Coming to you Tammy and Benjamin, as our people on the front line every day - upon your shoulders falls the responsibility for persuading people in many cases to actually take part. Do you have a standard approach, or do you tailor what you do according to particular circumstances? 

TAMMY FULLELOVE 

We definitely have a doorstep introduction, which has to cover a few different points to obviously make sure respondents are aware of the confidentialness of obviously the answers that that will be providing. But I do believe having a smile as soon as they open the door is the biggest thing - you're obviously trying to get them on board and trying to get them to either go online to complete the study or to make an appointment if they can't do it there and then to do the interview. It definitely has to be tailored I think, compared to who answers the door and obviously what reasoning they would like to help complete the study. Whereas some people as soon as you knock on the door, they've had the letters, they're waiting for you. They really want to help. So yeah, it definitely does depend on who's behind that door and obviously why they would like to help the Office for National Statistics.  

MF 

We live in a suspicious age and some people might think that there's something fishy afoot.  

BENJAMIN LAND 

That's the challenge Miles is people often initially they think it's a scam. I turn up with my badge and they're like, “Oh, you are real”. And taking the time to explain to people once we've done the doorstep introduction that it’s not a scam and it is legitimate, valuable research that we're carrying out and it certainly impacts everyone.    

MF 

I can imagine how tricky it must be to convince people sometimes, but you strike me as someone who isn’t likely to be put off by that. 

 BL Yes, yeah, I love a challenge. There was one lady last summer and every time she was like, “Oh, I'm busy. I've just come back from holiday, can you pop round the next week?” And it got to the point where she's like “I'm decorating my house.” I said, Ma'am, that's fine. I'll come and help you decorate your house if you complete this survey. And she's like, “Oh, you're so persistent.” I managed to get an interview and I was really pleased about that. So there's a little, you know, a little win in the bag.  

MF 

Well done, though I should point out that painting and decorating is not officially one of the ONS’s services for getting people to take part in surveys.  Tammy have you got experiences like that? 

TF 

Yeah, I've never got into painting and decorating, I'm gonna admit that. But it is a great feeling when the first time you knock on the door people don't want to help they're too busy, especially now post-COVID, with the amount of people working at home. So like Benjamin said, you're interrupting a Teams call. You're interrupting them doing some work. So you have to get over that first hurdle. But, you know, making that appointment, and sometimes they will make the appointment but then they either won't answer the phone, or they won't be in when you turn up, which can be frustrating. But yeah, when you actually do complete that study and they do feel like, you know, they have helped and you've gone above and beyond to secure that interview, it is definitely a great feeling. So maybe I should be offering painting and decorating services, maybe that would help.  

BL 

Don’t take my tricks. No, the sense of achievement or, like Tammy says, you do get people that break appointments, you know, due to personal circumstances, and you somehow have to chase people and encourage them, but when you do secure the interview, and you get the data. There's something about when something’s hard won you value it more.  

MF 

Yes, indeed. But how many people have heard of the ONS would you say? 

BL 

A lot of people now, because we were quoted a lot during the COVID statistics, regularly on the news and is quoted... I read the newspaper I appreciate not everyone does. But a lot of the data in newspapers it will state that it's been sourced from ONS.  

MF 

That recognition factor has helped help you on the doorstep. Do people get that the ONS is an impartial organisation operating at arm's length, certainly from ministerial government? 

BL 

No, no, I think we’re often tarnished with the same brush as the TV licencing people that come round, especially in certain areas where I knock on doors. You know that they were met sometimes with hostility, to put it politely.  

MF 

Clearly some persuading to be done in a wider sense there as well. But is that your experience too Tammy?  

TF 

Yeah, they do believe that we are a government body and that we are influenced by a particular minister, or by the government that's in power at the time. If people are very anti-government on the doorstep it does create that hostility as the first sort of part of your introduction.  

MF 

Do you try to talk them around on that?  

TF 

Yeah, exactly like Benjamin said that really, you know, we don't have a minister in control of us. We are separate to the government. Everything is private, confidential. We don't share the information. You know, there's different a few things that you have to try on the doorsteps to try and get that buy-in from the respondents. 

MF 

Those of us who live and breathe statistics, of course, we wouldn't need to be persuaded to the value of taking part, but the challenge is to convince the whole population, or at least a representative sample of the whole population. It seems well removed from everyday life for a lot of people, but how many people do you think “get it” in terms of the value of statistics, you know, particularly economic indicators and high-level population data?  

TF 

I think it's great if you can get to an area. And you know, that statistics, whether it's been from some sort of government funding, have helped in the area. So you can say to someone on the doorstep, well, the reason why this school was built or this doctor surgery, or this park, or some sort of local information, really does that help to sort of say why they are important to provide the information. But on the flip side, speaking to students who will obviously do research looking at the ONS data, they might be using, obviously, in their own work and people who work in sort of the public sector, I think, do understand to a degree how important it is. But then, I think the vast majority and I think Benjamin will help me with this, don't really understand why we’re collecting this or what benefit the information could have to them and where they live.  

BL 

That's right. I think a lot of people they have a global sense of it, but they don't understand the impact it has on their life and I work quite a lot in Bournemouth. And there's a lot there's a big student population as we've got Bournemouth University and the Arts University College, and a lot of the students do actually know or use the ONS data. I was actually at a student house yesterday in Winton and that makes my life much easier if I can link it to their own studies.  

MF 

In covering the whole of the country, of course, that means covering areas, which in statistical language are “hard to reach communities” - that’s the phrase that's used. And frankly, of course, that often means areas of considerable social deprivation.  Emma... How do we target those areas in particular, does that require extra attention or special techniques?  

EMMA PENDRE 

So our vision is to be fully inclusive by design. So that ensures that both the data and our workforce are fully representative of the population that we serve. The pandemic actually opened up opportunities and challenged how we have historically done things in the ONS. So to give a specific example here is around one of our key data sources, which is the Labour Force Survey. Before the pandemic we would write to addresses randomly, selected from our database of all UK households, and invite people to take part, and then knock on their doors to follow up if we didn't hear back from them. During the pandemic, when face to face interviews became impossible. We had to rely on people responding to the letter and taking part in the telephone interview. We saw pretty quickly that this was leading to bias in the responses, with particular demographics, such as the older population being more likely to respond where we were less likely to hear from people who sort of rented their properties. We knew we needed to speed up the work already underway to improve the survey. So fast forward two years and we now have transformed the Labour Force Survey and making it an online first survey which is now supported by telephone collection where needed. We've proven that to make the survey inclusive and reduce bias we also need to be knocking on doors. So for households invited to complete the survey from November 2022 They now might get a visit from a field interviewer who encourages them to complete the survey online or via a telephone interview, and we call this mode of field work “knock to nudge”. 

MF 

So in other words, it's not enough just to send somebody a letter inviting them to take part - that's likely to go unheeded. But a friendly face at the door and a little bit of gentle persuasion, can have a really useful effect.  

EP 

Absolutely. Right.  

MF 

And this is very important, because the ONS has committed with the Inclusive Data Task Force to make a special effort to ensure everybody is represented in official statistics, and field communities have been involved in that work.  

Tammy, you operate in an area that's quite ethnically diverse. How do you bridge barriers in communities where English perhaps is not the first language for a significant number of people?  

TAMMY FULLELOVE 

So in the North West, we do have a number of regions where it's densely populated, very different cultural diversity, I suppose obviously, London would cover the same. And we do rely on interviewers who speak second languages, who can then translate the languages of the people on the doorstep to go through the interview, or even just to help on the actual doorstep to speak to people and advise what the study is about and to make the appointments.   MF 

And Beth... when it comes to choosing field workers I guess it's very important as well that you've got people who are not only representative in the general sense of those communities, but actually have got some understanding, some feel, for the people they're dealing with.  

BETH FERGUSON 

That's part of the skill of a field interviewer. And I guess it comes from the fact that we've got interviewers from all different backgrounds, but it also comes as they learn the role, understanding which areas you know are going to be more challenging, where you're going to have to put a bit more effort in and understanding that actually... as an interviewer you can knock on a number of doors and, you know, you know who's going to be easy, you can get interviews relatively easy from various different sections of society. And you know that's going to be easy, but you also know that if you're going into an area that's more deprived, you're going to have to put more effort in, you know, and for some interviewers it comes immediately and, for others, it's learned over a period of time, where those more challenging areas are, what's actually going to work, what's going to resonate with the people behind the door that, you know, you're going to need to get that interview from to make sure the data is representative of everyone.   MF Now, Emma, let's talk a little about the future of the field community, because obviously we hear so much now about big data and the ability to discover and gather insights from that. A mountainous array of data sources that can now give us rapid, fast data, covering just about every topic you can think of. But, nevertheless, the ONS sees value in continuing to run these very large, and very personal surveys face-to-face and over the phone.  

EMMA PENDRE 

Yeah, social surveys will continue to have an essential role to play in ONS’s future, but also as part of a joined up data acquisition approach as well. I don't feel it's any longer a competition between whether we use surveys or other data sources. We have now come to realise that we actually need to work together and complement each other. So surveys are still fundamental in collecting the data that other sources cannot provide. And whilst new types of data sources are allowing us to more rapidly take stock of what's happening in our society and economy, they can't tell us everything or provide insights on things like personal opinions, attitudes, or exactly how people might be feeling at a given point in time. That will only ever be possible from talking to people.   MF 

And on that note, can I just say that it’s been a pleasure talking to all of you today. 

[OUTRO MUSIC] 

That’s it for another episode of Statistically Speaking.  

Thanks once again for listening, and also thank you for taking part in our surveys. Without you all the incredibly valuable information we get from our surveys – which help to inform better decisions by your local council, for instance – would simply not exist. 

If you haven’t yet had the opportunity to take part, and you get a knock on the door in future from one of our field interviewers, please do answer and take the time to respond.  

And if you happen to be in the Bournemouth area of course, and need some painting and decorating doing, then Benjamin’s your man!  

You can subscribe to new episodes of this podcast on Spotify, Apple podcasts and all the other major platforms. You can also get more information by following the @ONSfocus feed on Twitter.  

Special thanks to producers Steve Milne and Julia Short.  

I’m Miles Fletcher and until next time... goodbye.  

ENDS 

 

In this episode we discuss how the ONS has been working to transform the way we count the population, using new datasets to give more accurate, timely, and detailed measurements. 

On 29 June 2023, the ONS will be launching a public consultation on its proposals for a transformed population and migration statistics system. Understanding user needs will be essential evidence in making its recommendations to Government on the future of population statistics.   

More detail available at: www.ons.gov.uk  

To explain more about the public consultation, and answer your questions, the ONS is holding a series of free events in July 2023:  

• National Statistician’s launch event, London, 4 July 2023. (Online attendance also available)  

• National Statistician’s launch event, Cardiff, 6 July 2023. (Online attendance also available)  

• Launch webinar, 13 July 2023. (Online only) 

You can also watch our transformation journey video, which is also available with British Sign Language (BSL), and in Welsh, with BSL. 

 

TRANSCRIPT 

 

MILES FLETCHER 

Welcome again to ‘Statistically Speaking’, the official podcast of the UK’s Office for National Statistics. I’m Miles Fletcher and this time we're looking at the future of our population statistics. How best to count all of the people, all of the time, and provide the most valuable information on changing characteristics that can drive excellent research and sound public policy. All of that is the subject of a major consultation exercise that's running during the summer of 2023. It's all about the Office for National Statistics proposals to create what's described as a sustainable and future proof system for producing essential statistics on the population.  

Joining me to unpack all that and explain how you can get involved in the consultation process is Jen Woolford, Director of population statistics here at the ONS. And we're joined once again by Pete Benton, Deputy National Statistician.  

Pete in a previous episode, you described how the once in a decade census has been the bedrock of our population statistics for a very long time, but now it looks like some pretty fundamental change could be on the way?   PETE BENTON Well, that's the question. What's the future hold? We've been doing a census for over 200 years now once a decade, and it paints a beautiful, rich picture of our population that's fundamental to planning all of our services that we use: health care, education, transport, they all depend on the number and type of people living in a given area. But the question is, can we get more detail from other data sources every year, and might that mean that we don't need a census in 2031? Because we've got enough and that's the question that we are now talking about.   MF Okay, so before we go into the detail of how we might achieve that, then paint a picture for our listeners. When we talk about population statistics, what are they exactly? And why are they so important and to whom?   PB Well in between a census, we estimate the total population, by age and by sex and we do it nationally and we do it for local authorities. We estimate migration, how many people have moved into the country and how many people have moved out and also how people move around the country because that affects the population at any given area. And of course, we also do surveys that give us top level national level statistics about all kinds of things whether it's the labour market, or our health, things that the census asks and gives us detailed information for small areas, surveys, kind of paint a top level picture in between times.   MF So to date, how have we gone about getting those numbers, and how good has that information been?   PB So the census gives us the baseline once every 10 years. And we take that and we add births, we subtract deaths, we make an estimate of international migration. And we use that to adjust the data and we make an estimate of migration around the country, and that gives us those population estimates and those migration statistics.   MF So to do that you need, or you’d have had to have drawn on something like the census, that universal survey of the whole population.   PB That's right. The census is the benchmark by which we reset the system once a decade. But of course, after nine years, that information is getting more out of date and we do a census again, 10 years on to reset those statistics. And again, give us that rich picture. The question we're looking at now is how much can we get in between times? And how much do we then still need all the detail that a census would give us once a decade?   MF So Jen, the world has moved on in those decades since the census in its present form has been going. You would think there's an opportunity out there to transform how we go about counting the nation. Give us the background to that.   JEN WOOLFORD So we've been looking over decades to bring more and more data together to improve our population statistics. So Pete talked about how we look at the movement of people between censuses both in and out of the country and between different areas. And for some time now, we've been using what we call administrative data to understand those movements in the population. But now we have access to lots more data than we have in the past, and it gives us lots of opportunities to change how we're producing population statistics. So back in 2014, government first set out its ambition for us to build a population and migration Statistics System with administrative data at its heart. In 2018, we published a white paper, which set out our plans for a digital first census in 2021. But also that we should be making a recommendation to government about what the future of Population Statistics looks like, and that that recommendation should be based on a public consultation. And that's the consultation that we are going to be launching at the end of June.   MF The challenge therefore, is to come up with something as least as good if not, preferably better, but without using a census.   JW Absolutely. And people's needs are changing. So whatever we do has to respond to whatever the user needs are of the day. So in the past, where maybe populations didn't change so much at a local level so quickly, then having a census once a decade that gave you that detail, that detail would still be quite relevant 10 years later. But the population is changing so rapidly now that that decade old data can quite quickly become out of date. And an example of where this could be a problem for us and for policymakers is if we look at the COVID pandemic. During the pandemic, we saw really localised outbreaks of COVID infections, and we really wanted to understand what was going on in those areas and what the characteristics of people in those areas was to try and understand what might be leading to those outbreaks. But we didn't have census data, the 2021 census data then, we were having to go back to what those areas look like in 2011. So by transforming what we do, and having more up to date information about those local populations, it would have given us a much better idea of what might have been driving those local outbreaks.   MF And there was another example perhaps during the pandemic when the government was trying to work out what proportion of the population had been vaccinated at local level relying on population statistics that because they were backed up by the census was subject to quite significant margins of error.   JW That's right. So if you want to know what proportion of people in an area have been vaccinated, you need to know how many people are in that area in the first place. And if you're looking at a vaccination rate that's really high say kind of 90% that 10% is what's important, the 10% that aren't vaccinated. Now, you might only have a 5% error in your population estimates. But that could mean that you're thinking you've got 15% of the population to look at rather than the 10%.   MF Pete, we've heard this term admin data (administrative data) already. And in that we're talking about all the information that gets collected whenever someone engages with public services, tax bills, benefits, going to the dentist, that kind of thing. Now, presumably that information has been collected for quite some time. So why is it only in the last few years that we're really starting to see and begin to use the potential of that data?   PETE BENTON It takes time to develop the methods for doing it. So we've put a lot of effort into understanding the data sources and understanding the quality of the statistics that result so that we can be clear what we can and can't do, and that we can show that to the people that use the data to make decisions in order to understand the quality of what they're getting and give us their views of that.   MF Can you think of some examples of administrative data as already being used effectively in official statistics, the sort of things that the ONS produces.   PB Well we've always used them actually, when we produce our population statistics. We estimate the local population using the number of people registered with a GP and how that changes over time. So it’s not new, it's just that we're expanding what we might be able to do here to try and get so much greater benefit every year, to improve decision making every year for all of our public service planning.   MF And the opportunity, as Jen has already suggested, to link that data to understand how different groups, down to really quite small groups and local level and by different characteristics, are being affected by certain issues.   PB That's right. Different datasets tell us different things. So there are datasets that tell us about educational achievement and there are datasets that tell us about household income, for example. And by bringing those together, we can understand the implications of education per outcomes of household earnings so we can really start to tie together the kind of public services that we get and the outcomes that we get as households.   MF Now the possibilities of all this, of course, of being able to bring all this data into one place is a very exciting one from an analytical point of view, but from the point of view of the public and individual citizens at the same time, you could see why some people might be concerned about this, both from an ethical and a secure point of view.   PB Well, when you think about it, this is nothing new for ONS. We've been doing a census for over 200 years and we keep those data safe we always have done, and we also do surveys every year of households on very sensitive topics. Some of them are people's experiences of crime or their health for example, and we do surveys of businesses to understand the economy and produce our statistics about GDP and inflation. Those data are all sensitive, and we keep them all very securely. So in one sense, there's nothing new here. We are good at this. We know how to keep data secure. It's all anonymized. So there is never anything published that identifies an individual and even within ONS, the analysts only get to see anonymous data.    MF And very important to state, is it not, that it's not a question of building up pictures of individuals. It's always from a statistical point of view. It's the numbers we're interested in and not the people.   PB Absolutely! We don't care about Peter Benton or Miles Fletcher, we care about the picture it paints of the nation. It's the statistics that come from it. And we are absolutely strict about confidentiality.   MF Jen, other countries of course are wrestling with this as well and adopting and trying new kinds of systems. What's been the experience internationally?   JEN WOOLFORD So you're right, lots of countries are looking at new and innovative ways to create the population statistics bringing lots of different sources together. We all operate in slightly different contexts. So in Scandinavia, for example, they've been producing population statistics like this for a long time. But those are countries that have population registers, which means their context is very different from ours. And to be absolutely clear here, we're not looking at building a Population Register. We're looking at creating statistics from bringing together different data sources. And there are a number of countries who are in the same position as us. So for example, Australia and New Zealand, and they are looking to try and develop similar systems for producing population statistics as we are and we're working very closely with those countries to share our learning and to share the methods as we're developing them so that we're all learning from each other.   MF So talking about the potential of these new data sources, including all the administrative data, can you give us some examples of what we're not doing that we might be able to do much better in future?   JW There are a number of advantages and improvements we can make for greater use of data. Firstly, in the existing system, we use the census to benchmark our population estimates. So in between censuses, we estimate population change with births and deaths and migration, but we tend to get a bit of a drift in those population estimates. So we use the census then to benchmark it and bring those estimates back in line. With this new system, we're looking at not just estimating the change but also estimating the number of people at a point in time, so that hopefully will reduce that drift that we get in population estimates and mean that over the 10 year period, our estimates are more accurate. The other thing that can happen between censuses is you can get quite a lot of change in local areas and the data we have doesn't reflect that change, because it's based on the previous census. So an example here could be that the conflict in Ukraine has led to a number of Ukrainian refugees moving to England and Wales since we conducted the census. So in some areas, the makeup of the population there will have changed significantly since we conducted the census. And in our existing system, we wouldn't be able to pick that up. With our new system, we'd be able to pick up that localised population change much more quickly than we can at the moment.   MF And presumably that would be of enormous benefit for local authorities, where everyone would be trying to provide services down to local level, because you've got a much more up to date picture of how many people are there, and we saw recently when the census results were published, some local authority areas have experienced big changes in population.   JW Absolutely. The other thing to be aware of with the census is that it was conducted during the pandemic and it was conducted during a period of lockdown. What we saw was that people moved out of some of the metropolitan areas during that period of lockdown, back to whether that's the kind of parental homes for students or for young members of the workforce. So the populations in those metropolitan areas will have changed quite rapidly as the country opens back up and as people move back into those metropolitan centres. The approach that we're taking now should be able to pick up that change much more quickly, not just the numbers of people, but also the characteristics of people who are moving within the UK.   MF And how does this benefit individual citizens? What's this going to mean for the public generally?   JW So better data means better decisions. It means that better planning can be made for things like school places, better planning for public transport, where to put hospitals, where to put sports centres. All of these decisions are based on our data about the population and by having better data, you'll have better decisions.   MF And you’ll be able to target services and be able to target spending as well on a much more short term basis, rather than having to make decisions coming along into the future when circumstances could be changing.   JW Absolutely. Or the decisions might still be long term, but you'll be able to monitor the impact that those decisions are having much more closely than you can at the moment.   MF So Jen, is there anything we won't be able to get from such a system? And we've heard some people suggest, for example, that we wouldn't be able to get that very small level data, the street level data that's so useful from a census, and survey purists point, of the census as a great way of capturing social history.   JW We're always faced with trade offs when we make decisions about things like our methods, or anything in life, and there are likely to be trade offs here. What we've done to date is we've done lots of research that shows that there's bags of potential here with what we can do with administrative data and the understanding of the population we can get from administrative data. There are still outstanding questions for us. So there are some characteristics, for example, people who provide unpaid care, that isn't available from administrative data and we still need to work out how we will provide that level of data. The census gives such a wealth of information about things like ethnicity where we get down to really granular classifications of ethnicity, it may not be possible to do that with administrative data. However, on the flip side, we can produce statistics that we didn't get from the census using administrative data. So on the 2021 census, we didn't collect information about income. But we've published research that shows that we can get down to small area estimates of household income using a combination of administrative data. We've also published research which shows that we can produce the kind of variables that we do get from the census. So we've published research on ethnic group and also on housing stock, types of housing, and we've also managed to get to linking different admin data together so that we can look at income by ethnic group, and housing type by ethnic group. So producing what we call multivariate statistics through linked administrative data. We still have a programme of research to really understand how far we can replicate what we get out of the census. But the consultation that we're about to launch is really about understanding whether what we can demonstrate and deliver with administrative data answers user needs. And if it doesn't answer some of our user needs, what are those needs, and so we can then plan our future research to make sure we're focused on the right things.    MF And of course, it's genealogists - people who love to trace family trees - who find the census data so valuable.   JW Absolutely. And in the existing system census data is archived for 100 years and then made available to genealogists and others to really explore their family history. In the new system we have a wealth of data that we could be using to understand the population and we need to work with genealogists to understand exactly what it is that would be useful for us to archive for future posterity. So although that's not the focus of the consultation, genealogists are very welcome to respond to the consultation and let us know more about their needs, or we'll have future conversations to make sure that we're clear on what the need is here and how we can best answer it.   MF And that's what the consultation is, to a large extent, all about.  

JW 

Absolutely.  

MF 

And it's important to understand that these proposals haven't just been whipped out of thin air, a considerable amount of work has already gone on getting us to this point in time hasn't it. Can you talk through some of the research that's already happened and some of the evidence that has been provided to suggest that a new and transformed system might well be the way forward?   JW Yes, this has been a long programme of work where we have focused on two different types of research. One is around improving our estimates of the population and being able to get to small area population estimates more frequently than we can at the moment. And the other is around the characteristics of the population. So what can we say about ethnicity or employment down to local areas. On the first of those, we've done a lot of work talking to local authorities about the estimates that we've produced and their understanding of our outputs and whether they match with what they see on the ground. We have compared what we get through administrative data to the figures that we got from the 2021 census. So lots of work comparing the outputs and talking to our users about how credible those outputs are. We're also looking at how can we improve our estimates of migration, in particular international migration, and we've been working very closely with the Home Office and the data that they hold to understand more about the flows of people in and out of the country and the reasons for those flows. So people coming as international students, people coming to work, people coming along humanitarian routes, and we've built already lots of improvements into our migration statistics using administrative data and we've got lots of plans going forward for even more improvements that we can deliver there. We also have an expert panel, the methodological assurance review panel, who quality assure our methods. So these are people who are real experts in statistics and methodology, who have looked at the detail of the methods that we're using to produce those outputs and check that those are sensible and the best methods that we could be using.   MF So to sum up then Jen, how far ultimately could this new system take us?  

JW Well, the sky's the limit, really. As more and more data become available, there's more and more we can do, as our methods improve. As our computing power improves, there's more and more we can do to really understand the population, its characteristics, how it moves around. So this is going to be an ongoing programme of work for years to come.   MF So Pete, tell us then about the specifics of the consultation. Who is it for and what do we hope to get out of it?  

PETE BENTON Well, it's for anybody who would like to respond. We in particular, want to hear from people who use the statistics to get their view on the balance between all that detail that the census gives us once a decade compared with the frequency of having more information every year, and we want to understand people's perspectives on those trade offs, but anybody is welcome to respond to it. And of course, this is just the continuation of a conversation that we've been having for years. We're continually talking to the big stakeholders, the big users of our statistics across government, in local government, in the commercial sector to understand their needs for statistics. So this is a culmination of a conversation that's been going on for years.   MF Okay, so when does the consultation start? And how exactly do people go about taking part?  

PB 

Well, it'll be an online consultation. It'll start in June and it will end in October.    MF Okay. So the consultation completes in the autumn. Big question - what happens then?   PB So we will take a good look at all those responses we will understand what people have told us and then 12 weeks later, we will put out our response to that consultation summarising what we've heard. Following this, the National Statistician will make recommendations to government based on all of ONS’ research and the findings of the consultation to put administrative data at the core of a transformed population and social statistics system, and that recommendation will also consider the future of the census arrangements.   MF 

So there you have it, a one in a million opportunity – or more pedantically, one in 59.6 million, given that’s the accurate population of England and Wales according to the last census - to share your views on an incredibly important piece of work.  

Consultation opens on June 29th and runs through to the end of October. If you'd like to find out more about it and all of our transformation plans for population and migration statistics, you can do so by visiting the ONS website: www.ons.gov.uk  Or you can attend one of the free in person and online consultation events that the ONS has organised in July, details for which you can find on this episode's podcast page, as well as online through our social media channels, and the ONS website. 

Thanks to Jen Woolford and Pete Benton for taking us through everything today. And thanks as always to you for listening.  

You can subscribe to new episodes of this podcast on Spotify, Apple podcasts and all the other major platforms. You can also get more information by following the @ONSfocus feed on Twitter.  

I’m Miles Fletcher and from myself, and our producer Steve Milne, thanks for listening.  

ENDS 

In this episode of Statistically Speaking we shine the spotlight on local data and look at how good statistics for small areas make for better targeted policy interventions, and more effective use of valuable public resources.  

 

Transcript 

 

MILES FLETCHER 

Welcome again to Statistically Speaking, the Office for National Statistics podcast. I'm Miles Fletcher and in this episode we're talking about local data for local people - How good statistics for small areas make for better targeted policy interventions, and more effective use of valuable public resources.  

We're going to explore, for example, how new data sources are helping to precisely calibrate economic circumstances and local communities. How we may even be able to calculate the GDP of your street or village. Now many economic forces are of course global. Some of the solutions to issues like competitiveness, productivity and inequality might begin on our doorsteps.  

As ever, we have the cream of ONS expertise here on hand, this time in the shape of Emma Hickman, Deputy Director of the ONS sub national stats division, and Libby Richards, Deputy Director for UK wide coherence and head of an important new initiative called ONS Local, which we'll be hearing about in full. Also joining us is Stephen Jones, Director of Core Cities UK. Its aim is to promote the role of our great cities in creating a stronger fairer economy and society.  

So Emma, to set the scene for us first then please explain precisely if you would, the value of really good local stats.   EMMA HICKMAN So the needs are multiple, really. I think the most important thing is that we are seeing a huge increase in locally targeted policymaking and that’s at a range of different levels across government. So in central government, we see near the department for levelling up Housing and Communities kind of really wanting to think about how do they target policies that are going to help to level up the country but equally what we're also seeing is an increase in devolution which is giving more power to local areas and local policymakers. And so it's really also important that they have the statistics and the data that they need and the evidence that they need to make really, really good decisions for their local areas. And they can do that in a really powerful way because they also have knowledge of their local areas. And then finally, you know, actually for citizen kind of uses of our data and statistics really one of the inclusive data principles that people are able to see themselves in the data and that they feel that the data and the statistics that we're producing as an office represent them. And so having statistics and data available at really geographies that are very meaningful to people is hugely helpful in making sure that as a country, right across the UK that we are kind of reflective of the experiences of really kind of a wide range of people and you know, local economies and end users and understand kind of how they're experiencing that as well.   MF 

I guess one of the fundamental principles here is that it's it's local knowledge. It's all very well and everybody thinks they know that local area, but to understand all local areas, we need comparable statistics and data produced to consistent standards.   EH Yes, absolutely. And that's, I mean, that's one of the key challenges. I think we'll probably kind of come to talk about a little bit later, but you know, absolutely. And that's really about understanding you know, where are the where are the inequalities within regions, as well as between regions? I think we have a lot of information available about, you know, kind of regions, but actually, we also know that some of the inequalities that people really feel are much greater actually within regions and between them and kind of being able to draw that out of data and statistics in a comparable way I think is really important for helping sort of policymakers and decision makers to understand where best to target resources.    MF Stephen, from a policy perspective, describe the demand for local data at the moment, what sorts of policy solutions are policy makers coming up with and how are those best informed by really good data?    STEPHEN JONES I think it covers all branches really of policymaking. I think as Emma was saying, the kind of need for really understanding and having a kind of quantitative basis for what's happening in a place is, is actually absolutely crucial for designing policy, whether that's policy about trying to make the economy grow, whether that's policies aimed at trying to reduce disadvantage and challenge facing individuals, whether that's policy about delivering the most effective and efficient public services in the right places at the right times, all of those things, whether that's done in public or private sector need to be built on a good evidence base, good understanding. I think the other thing I would add to the richness of local data can do you can kind of contextualise and understand, you know, a number on its own doesn't mean a huge amount, but if you know that you are 10% higher or 20% lower than your neighbouring place. Or the city of the same size. It's those kinds of contextual dimensions that really help nuance and finesse your policymaking.   MF And it does come back to that question of trust in data than to make those comparisons in a really reliable and meaningful way. Which I guess is where the ONS, the Office for National Statistics, where we come in. Now Libby tell us about ONS Local. This is an initiative which is all about making sure that that really high quality data is available for the policy makers   LIBBY RICHARDS ONS Local is our advisory service that is staffed by ONS analysts who are based in every nation of the UK and every region of England. And the idea is that we are here to help local policy makers, regional observatories, and lots and lots of different users of sub national data to really understand the enormous offer from ONS in terms of local data. Having said that, it's also very much about those working relationships as well. Stephen’s talked a lot about context and understanding the nuances and so understanding the situations and challenges that are happening locally is absolutely key to ONS Local helping local areas understand that context better.   MF The big ONS surveys of course have long carried, many of them are typically think about the Labour Force Survey over a very long period of time, carried a great wealth and local data that obviously gets lost in the national headlines that these data releases generate. But is it a question of getting better value out of what the ONS is already creating or actually about sourcing new data from different sources?   LR It's a bit of both, very much, in being able to take people through what we already have when understanding their questions, particularly when multiple local areas are asking the same question that's really maximising what ONS already do. However, Emma's side of the house in particular, less so in the regionally and nationally distributed ONS Local is really about developing those new statistics getting into how do we get down to hyper localised sort of 400 to 1200 household building block data that then allow people to build those areas that means something to them. Emma, I don't know if you want to chip in?   EH Yeah, very happy to. There's two strands I think to that Miles. I think there's one which is about, you know, how do we make the most of survey data and kind of new administrative data sources together to enable that level of granularity? And then the second part is actually when we talk about administrative data probably, that might not really mean things to lots of people. That's data that is collected for a different purpose, but collected on a on a very, very routine basis. And there are actually a fair number of new sources of that kind of data that we're able to get into the ONS.   MF That's interesting. Can you give us an example of that?   EH So, I say relatively new. I mean, I think ONS have had this data for quite some time now. But in order to get the level of granularity that we need on Gross Value Added statistics, for example, which is a measure of productivity, we use HMRC’s VAT data for businesses and then we can link that to kind of our survey data and think about how can we then apportion estimates down to the level of geography that we need, knowing that the survey is the place where we've been able to ask the question that we really want to know the answer to and then we can use the other data to model sort of some of the other granularity that we need. The other thing is we've been really successful and using card payments data throughout the pandemic to inform the government's response. And we've recently successfully acquired a really exciting new data source from Visa, it's aggregated, so there's absolutely no way of identifying people in the data, but they've aggregated it at a really granular level of geography for us. So again, it would be in the region of probably hundreds of households, but actually that's granular enough for us to get some really, really good insights into kind of how you know, consumer spending is kind of playing out in the local economy. And there are all sorts of applications for that, that we're really excited to be to be able to start taking forwards now that we've got that data in the office.   MF So just with those three very important data sources, suddenly we're creating right down to that very micro level, as you say, 400 to 1200 households really quite a full picture of local economic activity.   EH And the really exciting thing about that is that people can then build their own geographies as well from that. So you know, traditionally in statistics, we tend to produce data at the level of an authoritative boundary like a local authority, but actually you might really want to know about, I don't know, West Midlands Metro, for example, they extended the line a few years ago, you might really want to know about local economic activity around that and actually, that's not going to be captured in the sort of administrative boundaries and so having the data at that level of granularity really allows people to build a geography that sort of area of interest or importance to them in some way.   MF Creating a GDP of your street or village.  

 

EH 

Indeed. 

 

MF 

Okay, that's the project for now, but it comes across with some pretty significant challenges. It comes back to this problem of comparability doesn't it, and particularly if you're looking across the UK contexts there. We've got different government structures, we've got some devolved areas, we've got areas and we've got big metropolitan authorities as well. How difficult is it to be able to standardise and to make uniform the data right across that rather complex government picture?   EH Incredibly so. To the point where we don't necessarily aim for uniformity. It's very much about how do we make sure that we're able to tell stories that are coherent and consider that UK wide angle when thinking about the nations but also thinking about how do you enable that comparability that's very tricky. And the more and more devolution happens, the more and more difficult that actually can become, particularly when you're looking, for example, at health data where it is a devolved policy area across the four nations. But actually, if you live on the border, let's say between Wales and England, actually, you may well be getting your health care on the opposite side of the border from which you live and therefore you've got to be able to have an opportunity to consider that.   MF There's the issue then of course of samples as well. And the more local you go, of course the less representative your sample is going to be.   EH Absolutely. And that gets particularly tricky. Even at a nation level where we're thinking about Scotland, Wales or Northern Ireland, for example, the opinions and lifestyle survey, actually, it's quite difficult to find out what that looks like for Northern Ireland. And ideally, we'd want to be able to get more granular than the nation level, but sample sizes make that really tricky to still be representative. And so either we'd need to expand the survey to get that level of granularity or we have to actually say the best we can do is this.   MF Yes, because there is only one holy universal survey of course and that is the census and that only happens once every 10 years. I recall when we were running the big COVID infection survey at the height of the pandemic, even a massive data gathering operation like that. We could still only end up getting it down to sub regional level which is what units are for half a million people. So it does show doesn't it how important it is to make the most of that admin data which can be extremely comprehensive sometimes   EH I, you know, completely agree with you there Miles on administrative data and how important it is to be able to kind of think about innovative ways to combine that data with our survey data to get a more granular level of information. I talked a bit earlier about kind of estimates of gross value added and I can say that's just that's a measure of productivity and it feeds into the largest component of GDP and in local areas. What we were able to do there as I mentioned kind of earlier, we took HMRC’s VAT tax data which is collected for all businesses that pay VAT, we were able to link that to a data set that ONS hold called the interdepartmental business register and the information that's held on that is all of the information about business structure, so has a VAT reference in there so we can link it to HMRC data. But the most important information on there for us was actually that where the local units are, so for example, Tescos will have a headquarters somewhere but you probably have a Tesco Express quite close to where you live. And that's one of the local units so tells us where the local units are and their postcodes and it also tells us how many employees work in those local units. And so we can make an assumption like productivity for all employees in the organisation is the same, and then we can look at actually what the productivity for that firm is top level and then divide that by the number of employees to kind of say, well, actually, if all employees are equally productive, this local unit has a productivity sort of measure of this much, and then we can aggregate that back up again to the sort of area so you know, really kind of key to be able to understand those methods, but there are some other challenges as well, but I can probably come back to those.   MF That's fascinating stuff. I mean, you could point to a certain, perhaps a certain enterprise, a certain employer, that is considered to be, you know, fundamental to a local economy. But this way, you can actually really press precisely quantify what that importance is.   EH And I think that's one of the challenges because actually as a as an office, we don't want to be disclosing the productivity of any single firm or any single business because that is personal information. So one of the things that we've had to do in very local areas where there are what we call dominant businesses or dominant organisations who have like most of the productivity for that area, is we've actually, you know, I'm gonna be honest, we've we've sort of masked it a bit. And so we've kind of averaged a few local areas together so that you still have a building block level of data, you still have a building block so you can build a bigger area, but you don't actually have any businesses that are considered dominant within the statistics that we produce. That's taken quite a complex algorithm to be able to achieve that. I won't go into too many details just to say that it is a consideration and the challenge that we've had to really innovate to be able to be able to publish that information.   MF It's important to stress Isn't it that all the usual principles of non-identification and confidentiality apply in this work as much as they do anywhere else across the ONS.   EH Yeah, absolutely.   MF Give me a couple of examples of some specific bits of work that you've been doing then. There's been an analysis of towns and out of town locations particularly and how local employment growth is happening outside of town and city centres.   EH My team kind of over the last sort of couple of years have been doing a whole series of analysis of towns in particular, like I say, that's a geography that people can really relate to, you know, lots of people kind of live in a town or a city. And that's something that's a bit more understandable than maybe a local authority and is a bit closer to them than the region for example. Our recent analysis on towns and out of town locations when we looked at employment growth, I think has some quite important findings actually for transport planning. For example, what we found is that actually employment growth is not happening the most in town centres, it's happening more and faster within two kilometres of the edges of a town of the town boundaries. And so what we think it might be happening is that kind of employment growth is actually happening in industrial parks are situated on that cusp between town and kind of rural areas. And when you're thinking about, you know, how people might travel to work, for example, I think it's really, really important to have those insights so that we're not just planning transport routes, for example, that go into town centres 

MF 

And what other insights have we been generating?  

 

EH 

So another recent piece was a new piece of analysis on the nighttime economy. So I think lots of people will think about the nighttime economy as being predominantly about bars and restaurants and obviously, you know, they will have a really, really big impact on those sort of industries during the pandemic. But in fact, what we find is that actually the nighttime economy in rural areas are surprisingly busy and that's because we also have a nighttime economy that is around health and health care. Nurses, for example, kind of working night shifts and that sort of thing. And then the other kind of aspect to it is sort of warehousing and transport as well. There's often kind of an overnight element to that, too. And again, having that understanding of like how that kind of plays out in different parts of the country is kind of a really, really useful. We originally did it just for London, interestingly, and then we've done this kind of new analysis looking at the whole country, which was really interesting. Other things produced quite recently as well are an expansion of job quality indicators of work across the UK, which is important because if you just look at kind of employment numbers, you're not really getting a sense of, you know, you get a sense of who's employed and who's unemployed in terms of characteristics of people, but what you don't get is like how good is the job quality for those people and actually, job quality is probably quite important for a lot of individuals and in terms of how good they feel about kind of going into work and how productive they are? And all of those those kinds of things,   MF That also forms the understanding doesn't it of why some people have opted out of employment in recent years.   EH Absolutely. And it also can tell us about things like how many people are working part time who want to be working full time for example. Or vice versa, you know, so there's kind of like a measure of underemployment in there. It tells us a little bit about what percentage of people are working on zero hours contracts versus permanent contracts, all those kinds of things, I think are quite, you know, sort of quite important.   MF Some other developments well worth pulling out as well. I think we've been able to produce very interesting picture of comparative housing affordability down to quite local level.   EH Yes, I think our main housing affordability release goes down to local authority level, but we have produced actually a range of housing affordability statistics, the local authority, one that we published recently probably been the most comprehensive, we're also doing a lot of work on the housing data that's collected through the census as well to understand dwellings and their characteristics as well. You know, how many dwellings are occupied and versus non occupied and how that varies by different parts of the country as well. Housing affordability in particular tells us about how people's earnings relate to what they spend on housing, and obviously that has huge impact on again, kind of, you know, people's disposable income at the end of the day. So I think it's certainly an important one.   MF So lots of fresh insights that are coming from the ONS and local statistics, but it's important to point out that a lot of this you could be doing for yourself if you're so inclined, and we've brought forward a tool called and it's much more exciting than the name implies, actually. It's called the Sub National Indicator Explorer tool. Libby, can you explain how that operates? And some of the really interesting insights that you can generate with it.   LR So the Sub National Indicators Explorer is something that we know and have known for a while that users desperately want. So often, if you are trying to understand a particular place, you have to go to lots of different sources to actually find information about one area. So for example, if you want health you have to go to one place. If you want to find out about education, you have to go to another and find your area and then collate that yourself. What the sub national indicators Explorer allows you to do is bring together all of those relevant indicators into one place so you can find your local authority and compare it with say up to three others across more than 40 different metrics ranging from gross median pay, right the way through to healthy life expectancy, and so you have this incredibly useful tool where you go, I want to know everything about place x and you get it all in one place. Our intention is to develop that a little bit further and eventually head into some of the developments that have come out recently around the census where you can build your own maps, build your own areas and flexibly bring different data things together. Alongside that we've also been thinking about how else we might be able to compare other areas and the team have recently done an analysis that clusters local areas together under metrics similar to and including some of the same from the sub national indicators tool and so that explores places that are statistically similar using things like regional growth metrics, and we can see what different parts of the country could potentially learn more from each other. They might be facing similar challenges and therefore getting beyond their local area to kind of join up with other areas across the country and this also gives some really weird potentially interesting insights.   MF Yes, which shows that despite the north south divide, about which we continue to hear a great deal some places in North and South have a great deal in common with each other.   LR Indeed, and actually places for example, in the south may be very different. So Portsmouth down on the south coast can look a lot more like places in the Northeast than possibly other areas on the south coast. Portsmouth is in a cluster of higher connectivity but lower health and well being whereas neighbouring Havant is in a much higher health and wellbeing and moderate educational performance cluster and you can see this all over the place. So for example, Newcastle upon Tyne is actually very similar to the New Forest and Havant and in fact, so is York and Great Yarmouth. And so they're actually disperate across the country, but mostly situated in particular areas. However, if Havant or the New Forest is facing a particular problem, maybe going and having a chat with York might actually be quite helpful depending on the problem.    MF That seems an excellent moment to bring in Stephen Jones as director of Core Cities. Stephen, the local picture, of course, is much more complex than that old cliche about the north south divide. But what work are you doing with the ONS and with others, to produce a really informed picture which policymakers can then act on to deal with these issues of localised deprivation, economic disadvantage and so forth.   SJ Firstly, we're doing a piece of work as Core Cities with the Royal Society of Arts called Urban Futures Commission, looking at the kind of like what's the long term potential and trajectory of our biggest cities in the UK and within that, you know, this is the sort of position of why do UK cities relatively underperform compared to the international peers in the developed world is quite a well established problem that's decades old. What some of the new data available is allowing us to kind of really get a better handle on is, why is that the case what is happening to for example, a fairly recent new release of fixed capital formation, so investment data, at a local authority level split by the different asset classes that the ONS have produced is really helpful to bring an understanding and a kind of richness to basically what both public and private investment we can see that our big cities outside of London have a relatively lower levels of public and private investment, particularly then if you strip out real estate investment. So investment in capital and business intangibles, those things are particularly low. So not all of our core cities, the total investment in Greater Manchester most recently was about 9000 pounds per head, central London, it's 55,000 pounds per head. If you go down to Newcastle I think it's down to 3000 pounds per head. You know, that's a dramatic difference in levels of public and private investment.   MF Does having much more reliable local data, perhaps hold with it the promise that the policy interventions that result from it can be therefore much more effective?   SJ So completely. You know, one of the things that I'm quite excited about in terms of using the local GVA data that Emma was talking about as a new release is there's been a whole host of different policy interventions over the last 10, 20, 30 years trying to kind of create economic activity within zones areas and whatever was saying about the ability to build your own geographies, I think is really has real potential in it. So whether it's the enterprise zones of the Heseltine era or the enterprise zones of the George Osborne era, whether it's free ports policy more recently, whether it's transport led regeneration schemes around new road junctions or new rail stations, whether it's the role of universities, science parks, investment in innovation zones, the government recently announced in the budget just a few weeks ago, the question of investment zones, all of these policies, they are some of the national ones – there's many more when you think locally are attempting to try and create concentrated economic activity within certain locations. One of the main criticisms in a policy sense is that that activity will just get displaced from elsewhere. If the business that is currently located three miles up the road will move to within the zonal boundary to gain sort of benefits and advantages that are being offered there. Well, we'll kind of be able to tell whether that's true or not, by actually looking to see whether the areas nearby have sort of reducing GVA compared to the areas that are growing and I think being able to properly evaluate policy interventions over the last 30 years to really then decide, well, is it worth pursuing policies like the investment zone announcement of recent weeks or actually should we be trying other approaches? I think that that kind of insight is going to be incredibly valuable.   MF Indeed, and perhaps also with data at a much lower level and much more micro local level as well, perhaps much smaller, more precisely targeted interventions might be what's called for.   SJ Exactly and I think that again, picking up some of what Emma was saying earlier, some of this data is a tool for local authorities. This has huge potential sort of exactly where are the jobs located? Are they in the town centre? Are they in the business park on the edge of town? What time of day is that activity happening? Is it shift patterns versus is it concentrated in the sort of 945 when we know these things, whether you're sitting there working out your local plan and working out where you're going to zone, your new employment land where you're working out whether you're going to offer any business rate incentives in a business improvement district when you're sitting there working out and what time of day do you need to have your trading standards officers available, these kinds of planning decisions day to day when you're trying to think about what your refuse collection plans and patterns are those things that local authorities are doing on just managing public services bringing together those different aspects having that sort of insight to know what's happening, when and what's most effective, we'll just make our policies more efficient. And in a world where public finances are constrained, particularly so for local authorities and have been for a while or be able to use the funding that is available more efficiently and the delivery of those services I think is hugely beneficial. The other thing that I'm interested in I think, is an area where we as Core Cities can can work with the ONS and others going forward is how do we make more advantage and take more advantage of the data, administrative data that is held locally? So if you think of an average local authority, they have huge amounts of data about that area. Whether that's through kind of council tax dates on collections, arrears, council tax discounts, whether that's through business rate data, whether that's through library card membership, planning applications, the list goes on. Obviously, for the same reasons, as we've talked about the need for protecting individuals and protecting data confidentiality, some of that data, you know, we'll need to be careful about how do we use but at the moment, it's largely sitting there on databases being under explored. If we can get to a world where we can start matching some of that data with some of the data sources that the ONS are making available, and then matching it with data sources such as Emma was talking about that the private sector can bring to the table like Visa and others. I think it's in bringing those sort of insights together. You can actually really, really develop the rich pictures. I can see Libby you would like to come in, so I might just pause there.   LR Yeah. I was just gonna say Stephen there mentioned about utilising locally held local data alongside national level local data, sort of your ONS data, your government department data, and actually that is one of the things that we're really hoping that ONS Local can help with by having people locally with very good relationships with those individuals in local government, local authorities, regional observatories, actually, if we can pull together their administrative data with what we have at the national level and help with some of that analytical insight because also aware, as Stephen said, local governments are constrained and resources actually, if ONS can help in that analytical insight, then even better that we can help along the way.   MF So Emma, an exciting vision of the future there and the possibility to be really improving local and regional policy interventions. What's coming next?   EH The really big exciting development that I just wanted to mention is the kind of opportunity for collaboration and I think ONS as an organisation are on the cusp of opening up the Integrated Data Service more widely, and actually, we've been working really, really closely with that team over the last couple of years or so to understand what a good data asset would look like for subnational. And to kind of start to make sure that we can do some of the data engineering to make that micro data. So when I talk about micro data, I'm talking like response level information from surveys kind of available in a secure and safe way and also in a way that's easily linkable, so that you can easily pick up something about health and something about quality jobs and link them together in that service and do the analysis that you were talking about. That's one of the most exciting developments. I think that's on the horizon in terms of how we'll be able to collaborate and kind of use and share data more widely, keeping in mind that privacy aspect. So you know, the idea is that all of that data is anonymized before it goes into the service and then things will be in kind of really strictly controlled through it. But there is that opportunity for those wider collaborations. I don't know Libby, whether you wanted to come in a little bit on some of the other kind of future developments as well. 

 

LR 

Yes, so over the last 9 to 10 months we have co-designed the ONS Local service going out across the country, doing round tables, getting people together in the room, putting forward our vision of what ONS Local might look like but very much saying “tell us why we’re wrong, what doesn’t work for you, tell us what we’re missing”. So really building that service with our users, and now we’re really beginning to fly now that we have people across the country. Other bits of new work also on the horizon include new data looking at the effect of place on geographic mobility across towns and cities, so we can follow those trends as people move around the country and can help us build pictures of places, track educational outcomes and workforce trends by area, at a level that we’ve not been able to do in the past. We’ve also talked a lot today about the Gross Value Added (GVA) data, and that obviously focuses on businesses. The next innovation for those sorts of granular statistics is more looking at the households aspect, and therefore allowing more targeted policymaking for those bespoke areas, and understand those hyper-local affects that are so important at the moment, particularly when considering all those devolution aspects.   MF 

Some insight there on the work underway here to ensure people across the UK see themselves in our data. Many thanks to our guests today Emma Hickman, Deputy Director of ons sub national stats division, Libby Richards, Deputy Director for ONS Local and UK wide coherence, and Stephen Jones, Director of Core Cities UK.  

I'm Miles Fletcher and thank you to you for listening. If you've got a question or comment about these ONS podcasts, you can find us on Twitter @ONSfocus. You can also subscribe to new episodes of the podcast on Spotify, Apple podcasts and all other major platforms.  

Many thanks to our producer for this episode at the ONS Alisha Arthur. Until next time, goodbye. 

 

ENDS