Welcome to the Demographics User Group

Representing the needs of commercial users of information

Get alerted to future DUG news stories by entering your email address in this box....

DUG: More News

February 13, 2015

Retail Research and Data Master's Dissertations – 15 projects now advertised

Following up the news on October 20 (below), we are very pleased to report that commercial companies have already come up with 15 proposals for Master’s projects this year.

These include one by the DUG members collectively: “Data Sharing between commercial companies, and with government: what are the current and potential benefits?”

You can see all the project proposals on the Consumer Data Research Centre website: http://cdrc.ac.uk/2014/10/retail-research-masters-dissertations-the-call-for-2015-projects/ 


February 12, 2015

Some tips from Professor Martin Callingham on free mapping (buildings & built-up areas) & postcode files

Talking to Martin earlier this week, he was enthusing about some of the above data (which is available free). Here’s his message, with an accompanying  map of Newport:

1. Building outlines – the outlines of buildings for the whole of the UK

You can get to the building outline file on  http://geogale.github.io/2011OAC/ where there are a series of folders, one named '2011 OAC buildings zip'. The outlines are shown in black on the map.

2. Built up areas file

This is the file giving built-up areas for England and Wales, which are shown in grey on the map. Get this on https://geoportal.statistics.gov.uk/geoportal/ and search for “built-up areas”. It also provides a useful gazetteer.

3. The National Statistic Postcode Lookup

The NSPL file contains every postcode in the UK with grid references to 1 metre updated quarterly and a lot of more fields of higher geography and also the new OAC. It has three postcode fields – the 8 and 7 bit ones and the 'variable length' one (i.e. the one with a single space before the last three characters). This last one has a problem in SPSS (and probably all software) as the first 2600 postcodes are 7 characters long, so the software assumes they all are, and truncates the ones that happen to be 8 characters long. As long as you know about this it is possible tell the software not to be silly. Use https://geoportal.statistics.gov.uk/geoportal/  and search for nspl. The map uses the building outline and built-up areas file. The rest comes from Meridian which is available on the open data site – Ordnance Survey section http://data.gov.uk/publisher/ordnance-survey

 All the files are free. The cartographic files are in shape format


February 2, 2015

Cabinet Office Minister (& Open Data evangelist) Francis Maude to stand down as MP

We’ve just spotted the news that the Cabinet Office Minister Francis Maude is standing down at the General Election: http://www.bbc.co.uk/news/uk-england-sussex-31082819

Setting party politics aside, there can be no doubt that Francis Maude has been a major figure for DUG and our goals over the last five years, notching up 11 mentions on our website.

Having been sceptical about the 2011 Census, he found funding for the ONS’s major assessment “Beyond 2011” for the next Census. Even more importantly, he has relentlessly promoted Open Data, winning the DUG Award in 2011 for “His enthusiastic promotion of the policies of Open Data, and investigating alternatives to another traditional Census”. You can see our citation, plus photos here: http://www.demographicsusergroup.co.uk/resources/2011conf-DUGAward2011-FrancisMaude.pdf

We will be very fortunate if we get a new minister with similar commitment after the election.


February 1, 2015

A summary of DUG’s many activities last year

If you’d like to remind yourself of what DUG members did in 2014, or want to tell a new face what the group is about, you can download the 4-page DUG Highlights here.


January 29, 2015

The Open Data User Group's new report on "The National Information Infrastructure" – to make more government data freely available

We are pleased to report that the Open Data User Group (ODUG) is adding to the pressure for more public data to be made freely available.

Its new report “The National Information Infrastructure (NII) – Why, What and How” states that:

“The UK has been muddling along for years without coherent data describing the fabric of our society' and calls on the government to 'make the NII an immediate priority and to provide a clear path forward for public sector data holders, many of whom are already moving to release more of their data as Open Data, so that their current actions will become part of a coherent bigger-picture where the parts fit together properly for the future……Both the public and the private sector should have access to the NII, and also citizens so that they can hold the government to account.”

Here’s the link: http://data.gov.uk/sites/default/files/library/odugUK_NII_final%20%281%29.pdf

The Annexes include examples of Housing Land (including OS mapping), and Health Data. It’s good to see further pressure for more data to be made open.


January 25, 2015

Ordnance Survey to become a "Government Company"

Matthew Hancock, Minister of State for Business and Enterprise at the Department for Business, Innovation and Skills, has issued a statement announcing a change in Ordnance Survey’s operation model.

You’ll see that OS is to change from a Trading Fund to a Government Company. The statement begins:

“The change is operational in nature, and is aimed at improving Ordnance Survey’s day-to-day efficiency and performance. It will provide the organisation with a more appropriate platform from which to operate, and one which provides greater individual and collective responsibility for performance. Ordnance Survey will remain under 100% public ownership with the data remaining Crown property, with ultimate accountability for the organisation staying with the Department for Business, Innovation and Skills. Further to this change, in the coming weeks I will also be setting out more details on how Ordnance Survey will be building on its existing extensive support for the Government’s Open Data policy and on some senior appointments which will further strengthen the management team……..”.

Ordnance Survey’s business model has led to a series of problems for users generally, and commercial users in particular (see various reports in DUG News), over many years.

Whether this reform will prove to be an improvement, or worsen the situation, remains to be seen. We’ll keep you informed of further news and comments


December 17, 2014

The Open Data Institute’s new "Open Data Roadmap"

The Open Data Institute (ODI) has just launched its Open Data Roadmap for the UK, setting out steps the next UK government can take to harness the benefits of open data for improved policy making and social, environmental and economic benefit.
It contains nine recommendations for the UK to continue to build on its open data successes including:

• an ‘all of government’ data strategy that brings together all aspects of data policy in one place in the Cabinet Office
• the creation of a new Chief Data Officer to develop, advocate and progress all aspects of this strategy
• building data publication into everything the Government Digital Service does,
• empowering trading funds to release more open data, including Companies House, Ordnance survey, Met Office and the Land Registry
• greater investment in data training for government, business and citizens
• a mandate that public procurement contracts require the release of open data, in line with the UK’s commitment under the G8 Open Data Charter to ‘open by default’

Here’s the link: http://theodi.org/roadmap-uk-2015

It’s good to see further pressure for more data to be made open.


December 13, 2014

A prototype for a future ONS website – your comments would be very welcome by 5 Jan 2015

Here’s an extract of a message from ONS, which is seeking users’ comments on a new prototype for its website.

We hope that you might find a few minutes amidst the Christmas rush – or between Christmas and New Year – to have a look.

“This website, http://alpha.ons.gov.uk/#!/  is the culmination of three months development and is based on feedback from hundreds of users of the current website. The considerable engagement we have had from users throughout this project has been invaluable and without them this prototype would never have happened.

The primary goal of this prototype is to test an approach based on the feedback we have documented over the last nine months. Before we start the next stage of development we want to engage a larger group of users to validate or challenge the decisions we have made so far.

The site is limited in its coverage of ONS content and does not aim to provide current statistical information. It should not be seen as an alternative or replacement for the ONS website at this point. However, we hope it provides enough depth of content to allow users to get a real sense of what does and does not work for them. The alpha website will be publicly available until Monday 5 January 2015. The more feedback we receive during this period the better the resulting future ONS website will be.

Feedback can be provided via the link on the prototype itself or by emailing web.comments@ons.gov.uk  . Please remember any detail, no matter how small, will be invaluable. I hope you find the prototype a step in the right direction and we look forward to any feedback you might have.”


December 6, 2014

Passenger statistics for every railway station in Britain: April 2013 to March 2014

Here are some new trainspotting stats from the Office of Rail Regulation:

“Waterloo has retained its position as the busiest station in Great Britain as top stations continue to attract growing numbers of passengers, reveal statistics by the Office of Rail Regulation (ORR).

London Victoria comes second with 81,356,330 entries and exits (an increase of 5.2%) and London Liverpool Street third with 63,004,002 (up 7.8%). The biggest climber this year in the top 10 stations was London Euston which clocked 41,911,706 entries and exits – a 9.4% jump on last year. Eight out of the top ten busiest stations in Britain were located in London, with Birmingham New Street coming in eighth (34,748,984 entries and exits, up 8.3%) and Leeds in tenth place (27,729,453 entries and exits, up 5.8%).”

If this is for you, put on your anorak and bobble hat and open the link below to scroll through estimates of passengers for every station in Britain:


December 3, 2014

Staying-in fashion: Britons spending more on rent and home entertainment – ONS's Family Spending Survey 2014

Here’s the Guardian’s take on ONS’s latest Family Spending Report 2014:

“British households are spending less on eating out, hotels, alcohol and smoking, according to the annual official survey of family budgets, which also reveals how rent, gas and electricity are taking up an ever larger proportion of spending.

Although spending rose last year compared with the previous one, much of the increase was taken up by rent payments as the property boom ate into household expenditure, the Office for National Statistics figures show.

As rising housing costs exposed the consequences of failing to get on the property ladder, the ONS also found disparities between the lowest and highest earners – with the richest 10% spending three times as much on food as the bottom 10%.”

There’s more here: http://www.theguardian.com/money/2014/dec/02/british-households-spend-less-entertainment-bills-rent-survey

And here’s a link to ONS’s report: http://www.ons.gov.uk/ons/rel/family-spending/family-spending/2014-edition/index.html


November 19, 2014

DUG Members’ needs for data from government – our Data Manifesto – latest update

DUG’s Data Manifesto, which was initiated by Mike Whitelegge at M&S, continues to make an impact in Whitehall, and is proving a very useful summary wishlist / reference document. Here is the latest version, which you might find useful if you are asked “What more government data would you like to see made freely available?”


November 18, 2014

Ordnance Survey accused of stifling competition in open data row

Here’s yet more criticism of Ordnance Survey…….

“UK mapping agency Ordnance Survey stands accused of using £800m of government contracts to stifle competition in a row over the release of geographical information as open data. Geo-data firm Getmapping claimed Ordnance Survey received illegal state aid in a complaint it lodged at the European Commission (EC) last Friday (7 November 2014).”

“Getmapping claimed Ordnance Survey used two 10-year government contracts worth £800m to stifle smaller, private geographical intelligence firms. The complaint seeks to make Ordnance Survey curtail its ambitions so private companies can prosper in its stead. Getmapping said the Department for Communities and Local Government (DCLG) let the contracts wrongfully in 2010 and 2011, because it didn't put them to public tender.”

"Ordnance Survey has been granted these contracts, which has made it highly profitable – much more profitable than most industry companies. It is using its state aid to compete with the rest of the partner network. Ordnance Survey is a serial abuser of its partners. And the government knows it."

You can read more here: http://www.computerweekly.com/news/2240234712/Ordnance-Survey-accused-of-stifling-competition-in-open-data-row  

November 4, 2014

UK Statistics Authority publishes a new Statement of Strategy – including the importance of business users

Following the talk by the National Statistician, John Pullinger, at DUG’s Conference http://www.demographicsusergroup.co.uk/24.html  , here is the UK Statistics Authority’s new Statement of Strategy, entitled “Better Statistics, Better Decisions”.

It’s good to see the emphasis on the importance of businesses as users of stats, e.g. “Decision-makers across the spectrum – individuals, businesses and government”.

There are also several mentions of the UKSA seeking to be “helpful” to its customers – which is good to see, setting the expectation. 

October 25, 2014

The DUG Conference “Can sharing data within and between organisations create social and economic benefits?” – and our annual Award

At our annual conference on 8 October, the winner of The Demographics User Group Award 2014 for “Better information from government” was…………

The Office for National Statistics

for its active engagement with private sector users when carrying out the most successful Census in recent decades

You’ll find the citation, ONS’s Press Release, and a photo, together with the conference presentations, at the tab on the left.

October 24, 2014

Supermarket locations, free from Geolytix

Our friends at Geolytix have produced another open (free) dataset…..a list of supermarket locations.

“There is no commercially sensitive information attached to the stores, just the store name, and an address from the retailers store locator, our GeoLytix town and locality names, and an exact rooftop location (in British National Grid and web friendly WGS84 lat/long). Yes, we have probably spent more time directing ourselves around the country in google street view than the google van spent capturing the imagery (and yes, we have manually located each of the 6,872 stores with the aim to get a really good dataset).

Current retailers include Aldi, Asda, Booths, Budgens, Marks and Spencer, Morrisons, Sainsbury’s, Tesco, Waitrose, Wholefoods. Lidl and Co-op are coming soon.”

Here’s the link: http://geolytix.co.uk/blog/?p=238


October 20, 2014

Retail Research and Data Master's Dissertations – call for projects now, to commence in 2015

Many of you will be aware of the opportunity for companies to propose projects for Master’s students that we have supported in the last three years.

New funding from the ESRC is making it possible to run this is the longer term, and to start earlier (ideally this month & next), to maximise the chance of getting the pick of the best students for 2015.

Here’s more detail from Guy Lansley at UCL, who is running the scheme this year.


Please start thinking about possible projects now, and don’t hesitate to get in contact with Guy to sound out your ideas.


October 14, 2014

Book Launch: London: The Information Capital – Friday, 31 October 2014 from 18:30 to 20:30

If you are interested in analysing, understanding, and mapping London, this will be right up your street.
Here below is a message from Dr James Cheshire of UCL about his book, and its launch – a great opportunity to meet up with young researchers who are exploring and visualising free open data for London.

“A lot of the discussion points – data hacking, visualisation, open data, user privacy, new datasources – are topics covered in my new book entitled London: The Information Capital (http://theinformationcapital.com/  ). The general theme throughout the book is that London is world leading in its data availability since it benefits from the likes of the 2011 Census as well as TfL data and large numbers of social media users. We visualise all these things alongside some previously unseen data such as Londoner's dating preferences (from Lovestruck) and what the police helicopters get up to on any given day (thanks to surprisingly successful FOI request).

It will be published on the 30th October and available in all good bookshops (as they say). Its RRP is £25 but the Guardian and Amazon currently have it heavily discounted at the moment (£15 and £13.60 respectively).

I am also hosting a launch event at UCL on the 31st October to which all are welcome. Sign up here: http://bit.ly/1w0bdih  
We hope the book will add to the growing momentum behind open data and to help London lead by example.



October 14, 2014

Census Research User Conference 2014 – 31 October 2014, Royal Statistical Society, London

The UK Data Service is holding a one-day conference to provide users and data creators with an opportunity to meet and discuss developments in census research.

The day will be composed of presentations from census agencies on developments in data and from users who have used the data in their research. The presentations will be based on analysis of the 2011 and earlier census products including aggregate tables, flow data and microdata

The conference will be followed by the Enhancing and Enriching Historic Census Microdata (EEHCM) event to launch new census products.

This is pitched at academic users, but might provide an opportunity to develop new ideas and make contact with census specialists.

You’ll find more details, and can book here:


September 27, 2014

ONS’s new web page – "Get involved with Census Analysis"

ONS has announced a new Census webpage, which includes details of planned and previous user events, a link to the StatsUserNet census discussion forum [good stuff for Census anoraks], and news of tools and what others have done with 2011 Census data. Here’s the link:



August 22, 2014

DUG's 11th Annual Conference: “Can sharing data within and between organisations create social and economic benefits?” – Wednesday 8 October 2014, at the Royal Society

This year our topic is “Can sharing data within and between organisations create social and economic benefits?”, and we will again have more than 100 delegates, enabled by financial support from the Economic and Social Research Council.

The topic has been prompted by discussion amongst DUG members, and meetings with the Cabinet Office, ONS (planning Beyond 2011), and the ESRC (with its big investments). This seems an ideal time to debate an increasingly hot issue. You’ll see on the provisional programme that our speakers include the National Statistician John Pullinger, and a there’s workshop to discuss public acceptability, future opportunities, and challenges and how to overcome them.

As always, the purpose of the conference is to bring together people from DUG's 16 member companies, and also specially invited guests in government and universities, to spread knowledge and stimulate new ideas. This is DUG’s eleventh annual conference, and we are convinced that increased dialogue brings enormous benefits to all parties.
As in previous years, the proceedings will be posted on this website.


July 25, 2014

"Big Data and Location": an AGI event in partnership with DUG – 30 September, IBM South Bank, London

The Association for Geographic Information has been in touch, seeking DUG’s involvement in its conference on “Big Data and Location”, and we are very pleased to partner in this event, which we think will be of real interest to many DUG members involved in geographical analysis. It will feature Mike Whitelegge of M&S as both a speaker and chair, and the programme also includes speakers from IBM, Deloitte, Cloudera, MapR, CapGemini, Esri and Ordnance Survey together with exhibitor stands from a range of suppliers.

Here’s the programme: http://www.geobig5.com/files/6114/0569/9029/ProgrammeLondonv1.pdf

You can book at http://www.geobig5.com/events/big-data/

July 28, 2014

Francis Maude confirms plans for the next Census in 2021 – and DUG is quoted in the Financial Times

The Government’s response to the Public Administration Select Committee’s report (see DUG News April 21) has just been published, and includes a letter from Francis Maude, confirming that the government will back the National Statistician’s recommendation that there should be a census in 2021, conducted at least partly online.

Mr Maude reiterated his view that the census is “outdated” and could be delivered more effectively and more cheaply, but said he accepted the 2021 Census should go ahead, alongside use of existing data and surveys. But he added: “It is the government’s ambition that beyond 2021 the decennial census would not be undertaken, instead more regular and timely administrative data would be used to produce statistics.”

You can see the report here: http://www.publications.parliament.uk/pa/cm201415/cmselect/cmpubadm/601/601.pdf

There’s a good article in the Financial Times (behind its paywall) by Emily Cadman, which includes:

“Keith Dugmore, who organises the Demographic Users Group (DUG), a group of some of the UK’s biggest employers in consumer-facing businesses, including Barclays, Boots, John Lewis and Sainsbury’s, said the conclusion was the right one.

The group had expressed grave concern about the option of moving immediately to solely administrative data, warning that any loss of data of very small geographic areas would be “catastrophic” for business users.

Mr Dugmore said that while “we don’t want to be lumbered with a traditional census forever” there was a “sense of relief” that the traditional census will be carried out again while research is done to see whether administrative sources could provide a true alternative.”

July 25, 2014

"Harnessing Open Data for Business Advantage" – MRS Census Geodemographics Group seminar – 5th November

This seminar will focus on how users have been applying census and other open data, with an emphasis on ‘how to’ case studies and useful lessons learnt.

You’ll see that there is a fine array of speakers, and that there is a discount if you book before 6 October: https://www.mrs.org.uk/event/course/2436

July 22, 2014

New, free, geodemographics – ONS releases the 2011 Output Area Classification (OAC) for the UK

ONS has announced the release of the new version of OAC:

We also have:
• Interactive maps, produced by UCL: http://public.cdrc.ac.uk/

• A new website for the Output Area Classification User Group (OACUG): https://plus.google.com/u/0/communities/111157299976084744069  

July 22, 2014

Companies House to make all of its digital data available free of charge

Whilst not many DUG members use Companies House data about businesses, here’s some encouraging news about further moves to make government data open and free.

“Companies House is to make all of its digital data available free of charge. This will make the UK the first country to establish a truly open register of business information.
As a result, it will be easier for businesses and members of the public to research and scrutinise the activities and ownership of companies and connected individuals. Last year (2013/14), customers searching the Companies House website spent £8.7 million accessing company information on the register.
This is a considerable step forward in improving corporate transparency; a key strand of the G8 declaration at the Lough Erne summit in 2013. It will also open up opportunities for entrepreneurs to come up with innovative ways of using the information……….”.

You can read more at: http://www.companieshouse.gov.uk/freeDataPressNotice/

July 16, 2014

New Geodemographic Classification Systems – a review by Peter Sleight

Peter Sleight, of the Target Marketing Consultancy, continues to provide very useful summaries of the competing geodemographic classifications – Mosaic, Acorn, OAC, and several others.

Here’s his latest:

July 14, 2014

UKSA Independent Assessment of the 2011 Census – DUG's Response

DUG has submitted this response to the UK Statistics Authority.


July 12, 2014

Land Registry privatisation vetoed by Vince Cable

There’s more news in today’s Guardian about Whitehall infighting over the government agencies and their data:

“Plans to privatise the Land Registry have been abandoned by the government after business secretary Vince Cable vetoed the scheme following a series of rows between Tory and Liberal Democrat ministers.
It had been hoped that a sell-off would raise at least £1.2 bn for the Treasury, with ministers deciding on the appropriate structure, such as creating a deal with a joint venture company.
But this week, ministers will tell parliament that the planned move has been suspended indefinitely. The Conservatives had been in favour of a partial sell-off via the creation of a joint venture.”

Read all about it here:

July 11, 2014

“How UK businesses are securing and managing talent to grow in a big data world” – a new report

Here’s news of a report on a hot topic for many DUG members. It’s authored by Nesta (an innovation charity http://www.nesta.org.uk ) in association with the Royal Statistical Society, which invited DUG to be represented on the panel at a launch on Wednesday 9 July:

June 28, 2014

New website "Datashine" – detailed and fast mapping of Census statistics across England & Wales

Following on from the news of “Maps of 2011 Workplace Zones in London” (25 June), here’s another initiative from UCL, this time providing online mapping of a wide range of Census variables at detailed level across the whole of England & Wales.

Here’s the link: http://blog.datashine.org.uk/

Warning: once you start playing, you might find it difficult to stop….

June 25, 2014

Maps of 2011 Workplace Zones in London – See the local differences in hours worked. education, transport, health, etc.

A PhD research student, Robin Edwards, at UCL’s Centre for Advanced Spatial Analysis (CASA) has been very quick – days after the release of the statistics – to publish a series of maps of c.10 variables by 2011 workplace zones in London:


It’s fascinating stuff – and may well lead to you further digging into the data.

June 17, 2014

Output Area Classification (OAC) – Its current use on government surveys, and the launch of OAC 2011

Many of you will be familiar with ONS’s (free) geodemographic Output Area Classification (OAC). If not, you can find out about it at http://areaclassification.org.uk/

We have learned from ONS that the following government surveys are now coded with OAC:

1. Annual Population Survey (APS) (which is based on waves from the Labour Force Survey)
2. Crime Survey for England and Wales (CSEW)
3. Dental Health Survey of Children and Young People (CDH)
4. Household Assets Survey (HAS) – formerly the Wealth and Assets Survey (WAS)
5. Integrated Household Survey (IHS)
6. Living Costs and Food Survey (LCF)

The new 2011-based version of OAC is expected to be launched next month.

June 13, 2014

New file of Census data at Output Area level for the whole UK

We are pleased to report that ONS has now produced a single file of statistics at Output Area level, spanning all four countries of the UK. (Experience in 2001 showed that it is so much better for the Census Offices to do this at source, rather than multiple users all try to sort out the sometimes varying definitions themselves). Here’s the announcement from ONS:

Today, 13 June 2014, ONS has made available the Key and Quick Statistics tables at Output Area (OA) level, for all Output Areas in the UK. This combines information previously published separately by the census office responsible for the conduct of the census in each constituent country of the UK.

The tables are equivalent to the local authority level UK data released in three parts in October / December 2012 and January 2014. Between them, those three releases provided all Key and Quick Statistics that were harmonisable across England and Wales, Scotland, and Northern Ireland.

The tables – 26 Key Statistics and 47 Quick Statistics – are provided now for all Output Areas in the UK, in a simple csv form suitable for additional processing by users. The data is accompanied by supporting information that explains the steps taken to merge categories in some tables to produce a UK harmonised table, and further detailed explanations and definitions of terms used in tables. To aid users in re-aggregating the OA data administrative and other common geographies, a look-up file that relates all OAs to every standard higher level geography in each country is also included.

All of the data is available to download in a single zip file from:



June 7, 2014

2011 Census – New (free) postcode directories & lookups released by ONS

The May 2014 editions of ONS’s Postcode Directory (ONSPD) and National Statistics Postcode Lookup (NSPL) have been released on the ONS Geography portal:



May 23, 2014

Workplace Statistics from the 2011 Census published today

Here’s news from ONS of the publication of counts and detailed classifications of workers (so vital when planning store locations) for Workplace Zones in England and Wales.

The workplace population of an area is defined as “all usual residents aged 16 and above who are in employment and whose workplace is in the area”. People who work mainly at or from home or do not have a fixed place of work are included in the area of their usual residence.

This definition focuses on only those people who are working, and the statistics are for a new “Workplace Zone” geography, which is particularly valuable for targeting very small areas within city centres. The statistics cover a wide range of topics. You can find more information and download the data at:


May 23, 2014

Government surveys on Income, Expenditure, & Wealth – User Conference – 7 July

This free full-day meeting in London, organised by the UK Data Service and ONS, will provide a forum for the exchange of information and views between users and producers of the following surveys:

• Family Resources Survey
• Living Costs and Food Survey
• Wealth and Assets Survey

The programme contains a mixture of papers from data producers and researchers.

To view the programme and book a place please go to http://ukdataservice.ac.uk/news-and-events/eventsitem/?id=3749  


May 14, 2014

John Pullinger appointed as the new National Statistician

We are very pleased to report that John Pullinger is to be the new National Statistician, succeeding Jil Matheson. The UK Statistics Authority’s News Release is here.

John has always emphasised the importance of the users of statistics, being very supportive of DUG over many years, and has been a regular guest at our annual Conference.

April 21, 2014

House of Commons Select Committee Report: "Census should not be scrapped"

Following our earlier news (see March 1 below) about the Public Administration Select Committee undertaking a study on the Future of the Census, it has now reported, urging the Government not to scrap the 2021 Census, but also promoting more use of data from administrative sources:

“Good figures on the people in the country are of fundamental importance to the statistical system, policy makers and society more widely, and the ten-yearly census gives detailed information on small areas. The Report follows the National Statistician’s announcement in March 2014 that she recommends that Government keep the census in 2021, but that it should be conducted largely online, and that the Government should make much greater use of the data which it already holds in order to improve the accuracy of population estimates. The Committee supports the recommendation from the National Statistician, but urges the Office for National Statistics to do much more to make the best use of the data which the Government already collects, for example through the Department for Work and Pensions, HM Revenue and Customs and the Department of Health. The Committee says that the Office for National Statistics’ work on the future of the census has, to date, been limited, and recommends that the Office for National Statistics now sets out a much more ambitious vision for the use of this data to provide rich and valuable population statistics.”

Bernard Jenkin MP, Chair of the Committee, said:

“The Government has previously indicated that the ten-yearly census should be axed, but we believe that it is too soon to decide whether or not to scrap the census. We think there should still be a census in 2021. The Government should make far greater use of the data it already holds in order to improve population estimates. We urge the Government to address public concerns about data sharing and to learn the lessons from the flawed rollout of the “care.data” programme, so that the best use can be made of the information already collected and held by the Government. The Government’s objective of “better, quicker information, more frequently and cheaper” depends upon it.”

You can see the report at: http://www.publications.parliament.uk/pa/cm201314/cmselect/cmpubadm/1090/109002.htm

It’s good to see in the Summary: “Witnesses emphasised, among other things, the great financial benefits to business provided by census data and these have been quantified.” 


April 15, 2014

Big Data – Open/Free data available for use by businesses – Conference organised by the Research Councils & Government

“This unique day aims to remove the barriers for companies to access this open data, revealing how to extract value and insight for the benefit of your company- of all sizes and sectors. You will be equipped with expert guidance on how to take advantage of the data sets on offer, as well as introduced to enabling funding schemes and organisations that can help you to take raw data and make it into something valuable.”

This sounds right up our street.
Time: Tuesday, 06 May 2014, 09:00 - 17:00
Place: Science & Technology Facilities Council, Rutherford Appleton Laboratory, Didcot, Oxford OX11 0QX

More details, & booking: http://biginnovationcentre.com/Events/133/Big-Data-Week-UK-big-data-open-data-available-for-business

April 12, 2014

UK Data Service makes 2011 Census statistics available to all

Here’s some good news from the ESRC’s UK Data Service:

“InFuse provides both academic and non-academic researchers with access to 2001 (England and Wales) and 2011 (UK) Census aggregate statistics under the Open Government Licence. This means there's no need to register or login to extract the data.

InFuse makes it quick and easy for the user to find and download information for places of interest across the UK and it is free of charge.”

You can read all about it at: http://census.ukdataservice.ac.uk/news-and-events/newsitem/?id=3761

This should be an important step in opening up data held by the academic community to wider access. 

March 28, 2014

National Statistician’s recommendation on the future of the Census and population statistics in England and Wales

Many of you will be aware that DUG has been very closely involved with ONS in its assessment of possible options for the next Census “Beyond 2011”.

We also responded to the formal consultation late last year, and we now have news of Jil Matheson’s recommendation to government http://www.ons.gov.uk/ons/about-ons/who-ons-are/programmes-and-projects/beyond-2011/beyond-2011-report-on-autumn-2013-consultation--and-recommendations/index.html
• An online census of all households and communal establishments in England and Wales in 2021 as a modern successor to the traditional, paper-based decennial census, combined with…..
• Increased use of administrative data and surveys in order to enhance the statistics from the 2021 Census and improve annual statistics between censuses.

This is a good result – but only so far, because Treasury has to accept the cost-benefit analysis, and Government has to approve the recommendation.

DUG was quoted in the Financial Times (apologies that you might have to register to view it):
http://www.ft.com/cms/s/0/907c1262-b598-11e3-a1bd-00144feabdc0.html#axzz2xMJK3EMs _____________________________________

March 18, 2014

PASC report on Statistics and Open Data – "The sale of PAF with the Royal Mail was a mistake"

Hard on the heels of the message about BIS’s report on the National Address Gazetteer, the Public Administration Select Committee (PASC) has published its report on Statistics and Open Data, which gives more support to DUG’s cause:

“In a report on statistics and open data published today, the Commons Public Administration Select Committee (PASC) says the benefits – potentially running into billions of pounds for the economy and society as a whole – of harvesting the masses of unused knowledge and information that Government collects outweigh the risks to privacy, and that there must be a “presumption of openness” with public data.”

“Some government data sets are of huge direct value to the economy. The Committee says that the Postcode Address File (PAF) was included in the sale of Royal Mail to boost the share price at flotation: this move took an immediate but narrow view of the value of such datasets. PAF should have been retained as a public data set, as a national asset.”

(We sent our submission to PASC on 5 September, urging it to press for the National Address Gazetteer and the Postcode Address File to be made Open Data).

You can see further news, and also download the report here:
There has also been good coverage in the media, e.g.
Guardian: “MPs and open-data advocates slam postcode selloff” (with KD being quoted) http://www.theguardian.com/technology/2014/mar/17/mps-and-open-data-advocates-slam-postcode-selloff
BBC: http://www.bbc.co.uk/news/business-26605375  
Such comments will help the Cabinet Office’s cause in its continuing battle with BIS and the Treasury.

March 15, 2014

An open (free) national address gazetteer? DUG's letter to BIS

Following the news on February 22, here is DUG’s letter to BIS.
You’ll see that we are continuing to make the case that the National Address Gazetteer should be free open data (but reluctantly see the proposed Freemium model as a step towards the ultimate goal).

March 2, 2014

Passenger statistics for every railway station in Britain: April 2012 – March 2013

Here are some new trainspotting stats from the Office of Rail Regulation:

• London Waterloo remains the most heavily used station in Britain clocking nearly 96 million passenger entries and exits last year – an increase of 1.8m over 2011-12.
• The number of people using stations across Britain in 2012-13 increased by over 82m – a 3.3% rise compared to the year before.
• The list of top 10 stations has remained unchanged since 2011-12 with eight of the 10 stations located in London.
• All the top 10 stations reported a rise in usage with the largest increase at London Euston, which saw an increase of 1.8m entries and exits – due to the increasing demand for London Overground services.

If this is of interest, you might want to put on your anorak and bobble hat before you open the link below to scroll through estimates of passengers for every station in Britain:

March 1, 2014

House of Commons Select Committee on the Future of the Census – DUG representation

The Public Administration Select Committee is undertaking a study on the Future of the Census, and an oral evidence session was held at the House of Commons on Tuesday 25 February.

Keith Dugmore spoke on behalf of DUG, and the following is now on the parliamentary website:
Video: http://www.parliamentlive.tv/Main/Player.aspx?meetingId=14944

Transcript: http://www.parliament.uk/business/committees/committees-a-z/commons-select/public-administration-select-committee/inquiries/parliament-2010/statistics/future-of-the-census/

We also followed up with this submission.

February 22, 2014

An open (free) national address gazetteer? Here's an independent report for government on its feasibility

With our long campaign for a definitive and free National Address Gazetteer in mind, if you are an addressing anorak, please read on…..

The Department of Business, Innovation & Skills has now published an independent report by Katalysis:

It recommends that a basic address product should be free to all users at the point of use under the Open Government Licence while premium versions would still be sold, leaving current production and maintenance roles much as they are today. Whether the government will act on this is not known, but the report is certainly valuable in its summary of the current state of play regarding address files and their ownership, and in setting out future options.

February 11, 2014

Access to data from the Valuation Office Agency

The VOA for England and Wales collects some valuable data, including:

Prices paid on sale or transfer of properties
VOA Council Tax – addresses, characteristics and attributes of properties
VOA Residential Private Rented Sector (PRS) rents

Non domestic rating:
• Rents paid on letting of properties
• Receipts/expenditure and costs of construction
• Property addresses, floor areas, characteristics and attributes

The recently-appointed Director of Information and Analysis gave a really good summary presentation of the current situation, and future plans to make more data available, at an HMRC meeting in October, and this is now public at:


February 9, 2014

Land Registry tipped for part-privatisation – Public Consultation

Several DUG members use information from the Land Registry, which is responsible for keeping and maintaining details of the ownership of land and property in England and Wales, and is a source of data on house prices.

The above disconcerting news appeared in both the Daily Telegraph:
and also the Guardian:

The Government announcement suggests we won’t lose the supply of free data, with reassuring words such as “maximising the reuse of property data for the benefit of the economy”, but if Land Registry data is important to you, you might like to have a close look at the consultation document:

The consultation will close on 20 March 2014. 


February 6, 2014

Scotland's Census 2011 – Further news of outputs

Many of you will be aware that 2011 Census outputs for Scotland have lagged behind those for England & Wales, and Northern Ireland.

We are pleased to report that, since the news (see below, 5 July) about population estimates and household estimates by Postcodes & Output Areas, and geography products, including OA boundaries and Postcode to OA look-up files, various statistics at OA level have now been published.

These are summarised in the latest Outputs Prospectus http://www.scotlandscensus.gov.uk/en/censusresults/outputsprospectus.html  (see page 4), which also includes news of plans for further releases.

January 27, 2014

Retailers: Do you have projects now that Master's students can help you to progress this spring & summer? 

If you work for a UK retail company, this is an opportunity for you.

The ESRC’s “Retail Research & Data” initiative, which involved commercial companies proposing projects for Masters’ students, was a real success in both 2012 and 2013, and Professor Paul Longley & I are pleased to report that the ESRC has decided to support this initiative in the longer term.

We are therefore seeking proposals for new projects as soon as possible.

This presents a great opportunity to get a Master’s student to help you to make progress with:
• Major current issues, such as multi-channel marketing and customer insight, store networks, transport, social media, and many others.
• Projects that you’ve long had in mind, but not had the resource to carry out.
• Working with data – both your own customer data, and also maximising the value to be obtained from government initiatives such as www.data.gov.uk, and the arrival of more 2011 Census statistics (including Daytime and Workplace populations) for the whole UK.

If you have a project in mind, please email Keith Dugmore ASAP (ideally by this Friday 31 January) at dugmore@demographic.co.uk and I’ll send you a Project Proposal Form.

It’s essential to move quickly on this, to publicise projects early and get the choice of the best students.

Our previous experience illustrates the main features of the initiative:
• ESRC’s publicity for the 2013 scheme, including a list of the 11 projects that ran: http://www.esrc.ac.uk/collaboration/business-sector/retail/retail-masters/
• Presentations by two prize winners at the 2013 DUG Conference: http://www.demographicsusergroup.co.uk/23.html
• Presentations by 12 of the 16 students at the 2012 DUG Conference: http://www.demographicsusergroup.co.uk/22.html


January 14, 2014

Happy New Year........... & here's a summary of DUG’s many activities in 2013

Very best wishes for a successful and enjoyable 2014.

Looking back on the past year, we hope that you’ll find this summary of the activities of the Demographics User Group during 2013 both interesting and useful.

You’ll see that it’s been another busy and successful year, with members sharing their experience in numerous ways, pressing government for further progress in the policy of making more data freely available, and also deepening our links with the ESRC, which is increasing the priority given to retail research.


January 9, 2014

Land Registry open data: House price paid data (November 2013) – down to postcode level

The latest alert from Data4nr includes house price stats right down to postcode level:

“The monthly residential property price data, tracks all of the residential property sales in England and Wales that are lodged with the land for registration. The data includes: the full address of the property (Primary addressable object name (PAON)), where available, secondary addressable object name (SAON), street, postcode, locality (if available), town, district, county), the price paid for the property, the date of transfer, the property type (Detached, Semi, Terraced, Flat/Maisonette), whether the property is new build or not, whether the property is freehold or leasehold.”

Here’s the link: http://www.data4nr.net/resources/1674/


January 7, 2014

A message from Martin Callingham and Steven Bond: Keith Dugmore made an MBE

"We are delighted to announce that Keith Dugmore, creator of the Demographics User Group, was made an MBE in the New Year’s Honours List. The citation was:

MBE – Keith Dugmore, Founding Chair Statistics Users Forum – “For services to UK Business”

This is a great personal achievement for Keith who has for many years successfully brokered and nurtured dialogue between various agencies of government, academia and the private sector. The benefits of the greater closeness of business to both government and academia are already apparent and will directly impact on the UK's economy and competitiveness into the future.

Many of you receiving this note will have been directly involved in this process and will know that only Keith could have made this happen. We are sure you will join with us in congratulating him on this richly deserved honour.

Martin & Steve"

(DUG’s Non-Executive Directors)


January 7, 2014

Full UK Census Data Pack from GeoLytix

Following up their earlier news below (18 February last year) it’s good to see that Geolytix are making a UK-wide (NB, yes, including Scotland) Census Data Pack available free of charge:

“Today, the full 200 variable GeoLytix UK Census Open Data pack is available for immediate free download. http://www.geolytix.co.uk/geodata/census

This release includes the Scottish data for all variables. We have also made three additions to the dataset since the last release:
• Added the Household Social Grade Variables (AB/C1/C2/DE)
• Included our own GeoLytix Urban/Rural flag for each OA
• Split the White ethnic group into ‘White British’ and ’White Other’ "

There’s also a brief blog on this release: http://geolytix.co.uk/blog/ 


December 22, 2013

HMRC consultation on sharing and publishing data for public benefit – the Government response

Following our submission (see 30 September below), HMRC has now issued its response:


This is positive towards making HMRC data more easily accessible, and will provide good context for when discussing priorities for the release of more datasets.


November 30, 2013

Retail Research Data – A website to help retailers find free data easily – new release

“The primary purpose of Retail Research Data (RRD) website is to enable insight and store location analysts working in retail organisations to get easy access to free datasets which may be of value to their businesses.”

This new release includes more 2011 Census statistics, and improved mapping. It is part of the ESRC’s Retail Research and Data project.

Here’s the link: http://www.retailresearchdata.com/

November 25, 2013

New report from BIS: "Seizing the Data Opportunity"

Here’s news of a new report from the Department for Business, Innovation and Skills: "Seizing the Data Opportunity".
Universities and Science Minister David Willetts has announced “A new strategy will ensure the UK is well placed to take advantage of the opportunities for using data. Seizing the Data Opportunity sets out plans to create a skilled workforce in data analytics, develop the UK’s data infrastructure that will allow access to high-end computing, and help data to be shared and linked to support ground-breaking research”

The report focuses on some of DUG’s goals, especially to improve the data infrastructure, and to promote training.

You can find the press release, and a link to the report here: http://news.bis.gov.uk/Press-Releases/Strategy-positions-UK-as-world-leader-on-data-science-69543.aspx 

November 22, 2013

ONS Consultation: "The census and future provision of population statistics in England and Wales" – DUG response

Following the earlier news below (September 24), DUG’s response has now been submitted to ONS.


October 31, 2013

2011 Census – ONS's "Workday Population" statistics for Output Areas in England and Wales

ONS has just sent out an alert announcing the above good news.

We’ve been pushing for this as an alternative and supplement to Workplace statistics (which will be for a special Workplace Zone geography, rather than for our regular Output Areas).

Here’s the link: http://www.ons.gov.uk/ons/rel/census/2011-census/workday-population-statistics-for-output-areas-in-england-and-wales--part-1-/index.html

12 tables provide information on some of the key characteristics of the workday population of England and Wales such as population size, general health, religion and ethnic group at Output Area level.
Similar tables for Workplace Zones are expected in Q1 next year. 

October 24, 2013

The DUG Conference “Understanding millions of mobile consumers”

Our Awards: (1) "ESRC wins top accolade from UK businesses"; and (2) Professor Martin Callingham

At our annual conference on 9 October, DUG made two awards:

(1) The Economic and Social Research Council, “for its increasing focus on retail research, and developing training in quantitative skills”
The ESRC produced a fine Press Release. This year we also made a DUG Personal Award to……….

(2) Professor Martin Callingham, formerly of Whitbread, and now Birkbeck College London, “in recognition of his role in encouraging the formation of DUG in 1998, and his immense contribution since, including training, playing with data, and providing an endless stream of new ideas”.

Both awards are hugely deserved.

You can see the Conference presentations at the tab on the left. 


September 30, 2013

HMRC consultation on sharing and publishing data for public benefit – DUG's response

Following the announcement of HMRC’s consultation (see July 29, below), you’ll find DUG’s response here.


September 27, 2013

ONS's Census News – including Postcode products, and Origin / Destination stats

ONS's latest News Alert includes:

• Migration stats at Output Area level, available from the Nomis website: https://www.nomisweb.co.uk

• Updated Postcode to OA look-up table:

• Origin / Destination (including commuting) statistics – are due in Q1 next year. 


September 25, 2013

ESRC Retail Research & Data – A website to help retailers find free data easily – Test Release – Your comments are very welcome

“The primary purpose of Retail Research Data (RRD) website is to enable insight and store location analysts working in retail organisations to get easy access to free datasets which may be of value to their businesses.”

Here’s the link: http://www.retailresearchdata.com/

This is part of the ESRC’s Retail Research and Data project. The first release was made available earlier this year. This new test release now includes information from the 2011 Census, the first release of the new Output Area Classification (OAC), and improved mapping.

It would be really helpful to get your feedback to dugmore@demographic.co.uk before we publicise the site more widely. 


September 24, 2013

ONS Consultation: "The census and future provision of population statistics in England and Wales"

As part of its investigation of alternatives to another traditional Census "Beyond 2011", ONS is now consulting on what it sees as two possible approaches:

• a census once a decade - similar to the 2011 Census but primarily online; or
• a census using existing administrative data and compulsory annual surveys.

Both approaches would provide annual statistics about the size of the population, nationally and for local authorities. A census using existing data and annual surveys would provide statistics about population characteristics every year. An online census would provide more detailed statistics but only once a decade.

The consultation runs until 13th December 2013, and ONS has arranged a number of events.
You can find the consultation document and a link to the online questionnaire here:


September 5, 2013

Public Administration Select Committee (PASC): Study into Open Data – DUG's submission

Following the earlier message below (July 27), the key themes of DUG’s submission are to:

• Recognise the importance of government open data to business
• Welcome the acceleration in progress in recent years
• Alert PASC to the fact that users outside the public sector (such as businesses and charities) are not able to enjoy all the free access arrangements which have been made for public sector users
• Urge PASC to press for the National Address Gazetteer and the Postcode Address File to be made open data


August 31, 2013

Job vacancies

Several new job vacancies in DUG member companies can be seen at the tab to the left.


August 25, 2013

The Guardian Data Blog, and Data Journalism

Lee Madden, former member of DUG whilst at Barclays, and now at Ernst & Young, is a great fan of the Guardian Data Blog, and has posted news on the DUG Network, on LinkedIn (it appears that you’ll need to sign in, rather than providing a direct link here).

“The Guardian’s data blog is a great source of both commentary on data and datasets. They also promote courses on data visualisation and have an active interest in data journalism. I'd recommend checking it out – I particularly enjoy it when they provide data commentary on stories that are in the public eye, e.g. mapping of the London riots or analysis on the A-Level and GCSE results.”

This also leads to “Data journalism at the Guardian: what is it and how do we do it?” Illuminating stuff. 


August 20, 2013

Tracking a decade of changing Britain using the Census and other Open Data – MRS Census Geodemographics Group seminar – Wednesday 6th November, 2013

Here’s news of a seminar “Tracking a decade of changing Britain: How businesses are benefiting from early use of Census and other Open Data”

You’ll see that there is a fine array of speakers, including VARs, and that there is a discount if you book before 6 October. https://www.mrs.org.uk/event/course/1946   


August 11, 2013

DUG's10th Annual Conference: “Understanding millions of mobile consumers” – Wednesday 9 October 2013, The Royal Society

This year’s DUG Conference at the Royal Society is on the topic “Understanding millions of mobile consumers”, and we will again have >100 delegates, enabled by financial support from the Economic and Social Research Council.

The topic has been prompted by discussion amongst DUG members seeking new insights into the drivers of consumer behaviour in our "always connected" world", and the need for better information about footfall, workplace populations, new shopping behaviour enabled by the growth in the use of smart phones, and the necessity to analyse massive volumes of data. The provisional programme is here.

In the morning session our expert speakers will explore these issues. The afternoon focuses on the ESRC’s support for retail research, including a showcase for some of the research projects specified by retailers for Masters’ students: http://www.esrc.ac.uk/collaboration/business-sector/retail/retail-masters/  and also has two Workshops:
• Research Issues: “What’s rattling your cage?” to identify the top priorities
• Skills Training for Insight analysts, which reflects members’ growing concern about difficulties of recruiting numerate staff, and the need for training

As always, the purpose of the conference is to bring together people from DUG's member companies) and also specially invited guests in government and universities, to spread knowledge and stimulate new ideas. This is DUG’s tenth annual conference, and we are convinced that increased dialogue brings enormous benefits to all parties.

As in previous years, the proceedings will be posted on this website. 


August 1, 2013

2011 Census data at Output Area level in E&W published – ethnicity, identity, language and religion

The ONS has now announced the publication of 19 tables for Output Areas in England and Wales on ethnicity, identity, language and religion – some of the variables that are of most interest to many DUG members.

You can download them from NOMIS: https://www.nomisweb.co.uk  


August 1, 2013

The Open Data User Group (ODUG) is seeking new members – might you consider applying? (Closing date: 23 August 2013)

Many of you will be aware of ODUG, an independent advisory group to government, which highlights public sector data with potential to bring social and economic benefit, and works with the Government to promote its release as open data. This is very much in support of DUG’s own long-established objectives.

ODUG is now seeking new members:


July 29, 2013

HMRC consultation on sharing and publishing data for public benefit (Closing date: 24 September 2013)

HMRC has launched this consultation https://www.gov.uk/government/consultations/sharing-and-publishing-data-for-public-benefit

“This consultation invites views on proposals for HMRC to increase the scope for the department to share non-identifying information (that is, information that does not relate to identifiable individuals or legal entities), and on proposed safeguards. Options cover general and aggregate data as well as anonymised data sets.

HMRC also seeks views on the potential benefits, costs / risks and necessary safeguards for proposals to share VAT registration data, either publicly or under controlled conditions for specified purposes, for example, credit rating. Views are welcome on the principles underlying these proposals and on the suggested approaches to implementation.”

DUG has long had its eyes on HMRC data (such as Incomes), and data held by the Valuation Office Agency (which reports to HMRC) such as Council Tax, and will respond to the consultation. 

July 27, 2013

Public Administration Select Committee (PASC): Study into Open Data (Responses due by 3 September)

PASC’s latest enquiry is focusing on the role of Open Data, what the government’s aims should be, what other data could be made available and how, as well as reviewing the data currently available and finding examples of best practice. It also asks which datasets are the most important and who uses them. Here’s the link:



July 22, 2013

Council Tax statistics from the Valuation Office - new classifications

Several DUG members already make use of statistics on Dwelling Stock by Council Tax Band, which are published at Output Area level for England and Wales: 


There are now some extra stats which go only down to Lower Super Output Area level, but which give more detail about dwellings. “These statistics were calculated from domestic property data extracted from the Valuation Office Agency’s administrative database on 1 August 2012. They provide a series of tables on the number of dwellings by Council Tax Band, property type (e.g. house, flat, bungalow, etc) and number of bedrooms and by various recognised statistical geographies, spanning England and Wales”. They also include “build period”:



July 5, 2013

Scotland's Census 2011 – Further news of outputs – Some OA-level data in August

http://www.scotlandscensus.gov.uk/en/news/articles/outputsprospectus.html the latest timetable for Census outputs for Scotland, includes, on 15 August: 

• Population estimates and household estimates by Postcodes, Output Areas and Datazones
• Geography products, including Area (OA) boundaries (shapefiles), Postcode to OA and OA to other geographies look-up files

So we’ll then have some stats for all four countries of the UK, but you’ll also see that Scotland’s Key and Quick Statistics (for Output Areas) are scheduled “from [NB] late September 2013”.

June 28, 2013

2011 Census – ONS, England and Wales – Postcode Headcounts, Key Stats for Built-Up Areas – published

These were published today, & here’s the link: http://www.ons.gov.uk/ons/guide-method/census/2011/census-data/index.html


June 18, 2013

ONS Census Analysis – Free Workshops – Manchester, 18th July, & London, 24th July 2013

For those of you already up to your necks in, or limbering up for, analysis 2011 Census, here’s the chance of some therapy being provided free by ONS: http://www.ons.gov.uk/ons/guide-method/census/analysis/summer-workshops/index.html

“Census Analysis workshops will take place in Manchester and London in July. This is your opportunity find the latest information on the 2011 Census outputs progress, what's coming next and to discuss the census analysis articles published so far date and the planned work for the future. The link provides all the information as well as an email address to secure your place: Census.analysis.inbox@ons.gsi.gov.uk

June 15, 2013

Shakespeare Report – Reviewing Open Data – The Government's Response

Following the publication of Stephan Shakespeare’s report on Open Data last month (see below), the Government has just published its response:

It is positive, and includes:

“Building on the recommendations of the Shakespeare Review, the government will include details of a new national Information Infrastructure in our revised Open Government Partnership National Action Plan due to published in October.

Working with data users, we will identify the data we hold that should be part of that new National Information Infrastructure. This data will be available for release to support economic growth and the developers and entrepreneurs who use open data to build information-led businesses – helping us compete in the global race.”

This could be another step towards the National Address Gazetteer being made freely available, but Royal Mail won’t easily give up its rights to sell PAF:

“The Postcode Address File (PAF) is a fully integrated part of Royal Mail’s business and will continue to remain so. In recognition of the continued importance of this dataset to innovation and growth, the Government has been working closely with Royal Mail to improve the licensing regime and drive greater take-up of the data. The Government is now announcing that, from the beginning of July, Royal Mail will provide the PAF for free to independent* small charitable organisations and free to independent* micro-businesses for one year. Royal Mail will also be increasing the number of free online address look-ups on its website from 15 to 50 per day. In addition, Royal Mail will consult in July on a radical simplification of the licensing regime for all customers based on a permissive licence with minimal restrictions.”
* Independent – not associated or affiliated with any existing Solutions Provider

June 7, 2013

ONS's new Open Geography Portal – for downloading geographic data such as postcode directories, boundaries, names and codes for ONS's area statistics

Here’s news of ONS’s Open Geography Portal “which allows users to discover, view and download geographical reference data to support National Statistics”
The link https://geoportal.statistics.gov.uk/geoportal/catalog/main/home.page  looks unexciting, but by roaming around (e.g. Browse) you can find some useful stuff.

June 5, 2013

Public Administration Select Committee – Communicating and publishing statistics – Report now published

Recalling DUG’s submission on 13 October last year, the House of Commons PASC report on Communicating Statistics has now been published.

The recommendations are not revolutionary, but add further pressure in the right direction, including:

• The UK Statistics Authority should work proactively to bring together and clearly present key statistics, from various sources, around common themes or events, such as elections and referendums, as well as broader topics such as the labour market and economic trends.
• The ONS website must be improved. Government statisticians should work much more closely with different kinds of users in order to present statistics in ways which meet their different needs.
• The Committee says the Statistics Authority should find more creative ways of communicating statistics, for example, through interactive guides. This should be in addition to the publication of more raw data in machine-readable format for experts who want the full results, not just the edited highlights presented in releases for a mass audience.
• In addition to the many routinely-produced statistics, government statisticians produce thousands of pieces of data on demand, known as “ad hoc statistics”. This is positive, but more of this kind of data should be published proactively, rather than simply in reaction to requests.

You can find more here: http://www.parliament.uk/business/committees/committees-a-z/commons-select/public-administration-select-committee/inquiries/parliament-2010/statistics/

May 19, 2013

2011 Census results for England and Wales – the latest news from ONS

ONS has just published the latest edition of its Census Newsletter. It includes information on data releases, the commissioned tables service, and an update on plans to improve access to data via the ONS website. Here's the link:

May 16, 2013

Shakespeare Report published – Reviewing Open Data to explore growth opportunities – further positive news

Following the announcement in October (see below), the Shakespeare Review of Public Sector Information was launched at a packed meeting yesterday.

Its recommendations include:
• The government should produce and take forward a clear, predictable, accountable ‘National Data Strategy’ which encompasses PSI in its entirety.
• A National Data Strategy for publishing PSI should include a twin-track policy for data release, which recognises that the perfect should not be the enemy of the good: a simultaneous 'publish early even if imperfect’ imperative AND a commitment to a 'high quality core'. This twin-track policy will maximise the benefit within practical constraints. It will reduce the excuses for poor or slow delivery; it says 'get it all out and then improve'.
• “One would be hard-pressed to find any expert who, asked to create new structures for core reference data from scratch, would advocate the current Trading Fund model”. The overarching aim of the Trading Funds [such as Ordnance survey] should be to deliver maximum economic value from public data assets they provide and support, by working to open up the markets their data serves. …..This means they should work towards opening up all raw data components, under the Open Government Licence (OGL) for use and re-use.

And also:
• We should develop a model of a 'mixed economy' of public data so that everyone can benefit from some forms of two-way sharing between the public and the commercial sectors.

These recommendations very much follow the opinions that we expressed when Stephan Shakespeare was a guest at our meeting in September last year, and subsequent representations by DUG members, and will give another hefty shove toward opening up more government data.

Here are some links:
Shakespeare Review Report: https://www.gov.uk/government/publications/shakespeare-review-of-public-sector-information
Deloitte Market assessment: https://www.gov.uk/government/publications/public-sector-information-market-assessment
Ministerial quotes from Matthew Hancock and Francis Maude have also been published on the Gov.UK website:
The Guardian’s Public Leaders’ Network blog: http://www.guardian.co.uk/public-leaders-network/2013/may/15/stephan-shakespeare-publicsector-open-data  


May 13, 2013

The Census: Now and in the Future – SRA Summer Event – Afternoon, 26 June 2013

On 26 June the Social Research Association’s summer event “The Census: Now and in the Future” will examine this issue from a variety of expert perspectives.

“Chaired by Mark Easton of BBC Home Affairs, this promises to be a lively and engaging afternoon”, with presentations from:
• “How is the Census faring” Ian Cope, Director, Population and Demography, ONS
• “What’s needed locally?” Ludi Simpson, University of Manchester, president of the British Society for Population Studies
• “Can the use of ‘Big Data’ eliminate the need for yet another traditional Census in 2021?” Keith Dugmore, Director, Demographic Decisions Ltd, and Honorary Professor, UCL
• Discussant: Simon Briscoe, author and journalist (and advisor to Bernard Jenkin, MP)

The event will be held at the Local Government Association in Westminster, and the cost is £65. You can find a flyer and booking details here: http://the-sra.org.uk/event-registration/?ee=74 

May 12, 2013

Land Registry to release free historical property data 

Here’s some good news from the Land Registry:

“We are pleased to announce that we will release a number of free datasets as part of our commitment to the Open Data agenda.

The following datasets will be available to download from our website under Open Government Licence:
• 30 May 2013 – historical House Price Index (HPI) tables. These will be in CSV format and from July they will be in machine readable format as linked data. The tables form part of the HPI calculation and contain up-to-date indices, average house prices and sales volumes dating back to 1995 for all England and Wales.
• 28 June 2013 – historical Price Paid Data (PPD) in CSV format. This will be available as linked data from July. Historical PPD will contain records of the price paid for every residential property sold at full market value lodged for registration between 1 January 2009 and 31 January 2012 in England and Wales.”

There’s news of further datasets at: http://www.landregistry.gov.uk/announcements/2013/land-registry-to-release-free-historical-property-data

And you can see commentary from Geolytix at: http://geolytix.co.uk/blog/?p=175


April 20, 2013

"Everyone's postcodes to be privatised in Royal Mail flotation, despite objections from Sir Tim Berners-Lee" – Daily Telegraph

A report in yesterday’s Daily Telegraph starts:

“A national database containing the postcodes of everyone in Britain is to be privatised along with Royal Mail, despite direct "very forceful" protests to David Cameron from Sir Tim Berners-Lee, the founder of the world wide web.”

and includes:

“Cabinet Office minister Francis Maude has been privately battling with Business minister Michael Fallon over whether to include the database, worth between £500million and £900million, in Royal Mail's flotation, which is expected as early as October.”  (You might have spotted that both ministers are past winners of our annual Award).

Here’s the link to the full article:

Yes, a further battle in the address wars. DUG will continue to support the case for national addressing to be free.



April 8, 2013

Electoral Register opt-out rates for Local Authorities (ranging between 8% and 80%)

Callcredit Information Group has today published its '2013 Regional Electoral Opt-Out Report'. The message starts:

“Of the 46m consumers eligible to vote within the UK, 18m have chosen to opt-out and have therefore have made themselves unavailable for marketing activity and wider benefits it can bring. The over-all opt-out rate for 2013 is 39.8%, a 0.2% decrease from 2012.

There remains a large regional variation with the lowest opt-out rate of 8.03% and the highest of 80.25%, therefore marketers will need to continue to utilise additional data sources in order to verify their own customer data.”

You can read more at: http://www.callcredit.co.uk/press-office/news/2013/04/2013-electoral-register-opt-out-rates-begin-to-slow


April 4, 2013

Passenger statistics for every railway station in Britain: April 2011 – March 2012

“Waterloo has retained its position as the busiest station in Great Britain as top stations continue to attract growing numbers of passengers, reveal statistics published today by the Office of Rail Regulation (ORR).”

If this is of interest, you might want to put on your anorak and bobble hat before you open the link http://www.rail-reg.gov.uk/server/show/nav.1529 to scroll through estimates of passengers for every station in Britain.



March 26, 2013

2011 Census release 2.4 – Including Social Grade at OA level for England and Wales

ONS has announced further news about the release of more statistics, including:

• Approximated social grade (household reference persons aged 16 to 64) – at Output Area level (downloadable from NOMIS www.nomisweb.co.uk/ )

• Also, the Commissioned Tables service has been launched much earlier than planned. This could be useful if you want more detailed categories for topics such as birthplace, ethnicity, religion, etc. 


March 17, 2013

ODUG's response to the OFCOM review of PAF: "Don't sell our postcodes"; and our message to Michael Fallon MP

Coinciding with Heather Savory, Chair of the Open Data User Group, speaking at the DUG meeting on 15 March, here’s news of her blog. ODUG (including Bob Barr) has just made an excellent response to OFCOM about the future of the Postcode Address File:

Also, DUG has written about “the need for free use by citizens and the private sector, as well as the public sector” to Michael Fallon, who is now the Minister for Business and Enterprise, having been Chair of the Treasury Sub Committee, when it recommended the creation of a national address file. You may remember that he won our award at the DUG Conference in 2008 http://www.demographicsusergroup.co.uk/resources/2008conf-DUG_AWARD_2008.pdf

Let’s hope that this mounting pressure finally achieves a breakthrough in government policy, and the arrival of free addressing. 


March 5, 2013

DUG 2011 Census Workshop session – Presentations

DUG’s Census Workshop at the Royal Society on 27 February went well, and here are the three presentations:

• The value of small area Census data products to commercial companies (Annette Dellevoet)

• 2011 Census – Statistics (Emma White, Head of Policy and Analysis, 2011 Census Outputs at ONS)

• 2011 Census – Geography (Ian Coady, ONS Geography) 

February 24, 2013

Beyond 2011 – DUG’s minimum needs for Census-type statistics: our views, now sent to ONS

With many thanks to the members who contributed, here is a summary of DUG’s needs for future Census-type statistics, which has now been sent to ONS.

The ONS Beyond 2011 team is continuing to develop the business case to the Treasury.


February 24, 2013

The Shakespeare Review – & the possibility of more free Open Data – an invitation to respond to this YouGov Survey

Stephan Shakespeare, the Chair of the government’s Data Strategy Board, who was a guest at DUG’s September meeting, is reviewing public sector information. Here’s the message from the DSB:

“The Shakespeare review of Public Sector Information led by Stephan Shakespeare will examine ways in which the public sector can improve access to its data and information to stimulate economic growth, promote efficiencies, and bring wider benefits to society.

At the end of the review, recommendations will be made to Ministers on how to widen access to public sector information and how to exploit new and innovative opportunities for open data including issues around data protection and copyright.

Stephan is keen to gather views from as many types of users, re-users and potential users of public sector information as possible and invites you to complete an online survey https://start.yougov.com/refer/vvCYQdvVkq6JVB  . The survey will close on Friday 15th March 2013.

Your contribution to this survey is important and will help shape the future of the UK’s public sector information and open data policy.”

Do please complete it if you have 10 minutes – the greater the number of replies will increase the probability of more data (such as address files and mapping) being made freely available.


February 20, 2013

2011 Census Key and Quick Statistics for England and Wales – now other geographies, including postal sectors

Following the news on 30 January about 2011 Census data down at Output Area level for England and Wales, here’s news of the same statistics for other geographies, including postal sectors.

“ONS has now published 2011 Census Key Statistics for national parks, and Key and Quick Statistics for health areas, postcode sectors and National Assembly for Wales constituencies and Assembly for Wales electoral regions.”

The Key Statistics for national parks are only available via the ONS website at: http://www.ons.gov.uk/ons/guide-method/census/2011/index.html

The data for health areas, postcode sectors and National Assembly for Wales constituencies and Assembly for Wales electoral regions will be available via the NOMIS website at:


February 18, 2013

2011 Census Pack free from Geolytix

Here’s some good news from Geolytix. It starts:

“Today GeoLytix are releasing harmonised Output Area Census Data covering the c.200 core variables as a single easy-to-use flat file. This data is being released with no charge under the same Open Government License that the censuses themselves use. There are no additional attribution requirements, you can use this data in exactly the same way you can use raw census data. We’ve also written a swish User Guide, prepared meta-data tables and even included a single consolidated look-up file to higher geographies and proportional look-ups to the 2001 Output Areas.”

You can read all about it at: http://geolytix.co.uk/blog/ 


February 15, 2013

ESRC: Master's projects proposed by retailers – news of the first 10

Following the news on 12 January (below) there are already 10 retail projects announced on the ESRC’s website:


February 14, 2013

"Mapping the 2011 Census: how one man produced a library for all" – a report in The Guardian

Here’s news from the Guardian:

“Alex Singleton downloaded every single census dataset for every local authority in England - and then produced a free library of downloadable PDFs. Find out what he did…….

Singleton has basically produced a detailed mapping report - as a PDF and vectored images - on every one of the local authorities of England & Wales. He automated the process and has provided the code for readers to correct and do something with. In each report there are 391 pages, each with a map. That means, for the 354 local authorities in England & Wales, he has produced 127,466 maps.” [Goodness…..]


February 6, 2013

Further Benefits of an Open National Address Dataset – Paper by the Open Data User Group

It’s good to see the Open Data User Group is piling on the pressure for a free national address file, with an excellent paper about the benefits:



January 30, 2013

New stats after 10 years: 2011 Census Key and Quick Statistics for Output Areas published today

Here’s news from ONS of fresh Census data down at Output Area level for England and Wales:

There’s more information about Bulk Data in csv format:

And also further geography products for electronic download:

January 22, 2013

ESRC – Retail Research Data – A new website to help retailers find free Open Data easily

As well as the Masters’ projects for retail companies, another element of the ESRC’s Retail Research & Data project is a new website:
“The primary purpose of Retail Research Data (RRD) website is to enable insight and store location analysts working in retail organisations to get easy access to free datasets which may be of value to their businesses.”

The website has 6 types of data:
1. Small area geodemographic classifications - such as the Output Area Classification (OAC)
2. Small area statistics – particularly from the 2001 Census, on many topics
3. Lists of addresses, with postcodes, and places
4. Postcode directories – for linking postcoded addresses (including customer records) to Output Areas and other geographical areas, or to grid references for mapping
5. Map data – digital boundaries (such as Output Areas), and also background mapping
6. Sample surveys – giving lots of detail about particular topics

RRD also includes a mapping facility to view the Classifications and Small Area Statistics for the United Kingdom and its constituent countries.

Here’s the link: http://www.retailresearchdata.org

January 12, 2013

Do you have projects now that Master's students can help you to progress this spring & summer?

If you work for a UK retail company, this is an opportunity for you. The ESRC’s “Retail Research & Data” initiative, which involved commercial companies proposing projects for Masters’ students, was a great success last year, and the ESRC has decided to repeat it in 2013. We are therefore seeking proposals for new projects as soon as possible.

This presents a great opportunity to get a Master’s student to help you to make progress with:
• Major current issues, such as multi-channel marketing and customer insight, regional variations in the economy, store networks, transport, and social media.
• Projects that you’ve long had in mind, but not had the resource to carry out
• Working with data – both your own customer data, and also maximising the value to be obtained from government initiatives such as www.data.gov.uk, and the perfectly-timed arrival of the 2011 Census

If you have a project in mind, please email dugmore@demographic.co.uk ASAP for a Project Proposal Form. It’s essential to move quickly on this, to publicise projects early and get the choice of the best students.

Last year’s experience illustrates the main features of the initiative:
• The list of the 16 projects specified by 10 retailers: http://www.esrc.ac.uk/funding-and-guidance/collaboration/working-with-business/dissertation-opportunities.aspx
• ESRC’s publicity and explanation of the features to students and academic supervisors: http://www.esrc.ac.uk/funding-and-guidance/collaboration/working-with-business/retail-masters-student-projects.aspx
• Presentations by 12 of the 16 students at the Demographics User Group Conference: http://www.demographicsusergroup.co.uk/resources/ESRC+-+Masters+Student+presentations+-+Final.pdf

January 5, 2013

Members’ needs for data from government – our updated Manifesto

DUG’s Data Manifesto, which was initiated by Mike Whitelegge at M&S, has made an impact in Whitehall, and is proving a very useful summary wishlist / reference document.

Here is the latest version, which you might find useful if you are challenged with “What more government data would you like to see made freely available?” 


January 1, 2013

Happy New Year........... & here's a summary of our many activities in 2012

Very best wishes for a successful and enjoyable 2013.

Looking back on the past year, we hope that you’ll find this summary of the activities of the Demographics User Group during 2012 both interesting and useful.

As you’ll see, it’s been another busy and successful year, with members sharing their experience in numerous ways, pressing government for further progress in the policy of making more data freely available, and also a major initiative with the ESRC to provide Masters’ students with relevant research projects.


December 18, 2012

First 2011 Census results for Scotland

After all the activity by ONS for England and Wales, we now have the first 2011 Census statistics for Scotland.

The following link leads to some headline results, and also summarises the timetable for future releases (with Release 2 – Key and Quick statistics tables; Census Profiles; Population and Household estimates by postcode and Geography products – not becoming available until “from summer 2013”): http://www.scotlandscensus.gov.uk/en/censusresults/

And here’s the BBC’s “11 things we learned from the Scottish 2011 census”: http://www.bbc.co.uk/news/uk-scotland-20754751


December 13, 2012

New funding to accelerate benefits of open data – £7.5m Data Strategy Board Breakthrough Fund; a new £850,000 Open Data Immersion Programme; & easier OS data

The Data Strategy Board team reports:

“The Government has announced three new complementary Open Data investments
• A new £7.5m Data Strategy Board Breakthrough Fund to which public sector bodies can apply;
• A new £850,000 Open Data Immersion Programme to which companies can apply; and
• Upgrades to the format in which Ordnance Survey data can be downloaded to make it easier, more accurate and more flexible for companies, government, charities and individuals to use.”

All of the measures announced are expected to be in place by early April 2013. You can see more detail at: http://news.bis.gov.uk/Press-Releases/New-funding-to-accelerate-benefits-of-open-data-684c1.aspx

It’s another step in the right direction.

December 12, 2012

2011 Census – Key Statistics at Local Authority level now published

You will no doubt have seen some of the extensive media coverage about the latest release of Census statistics. Here’s ONS’s announcement:

“ONS has now published the 2011 Census key statistics at local authority level on its website at http://www.ons.gov.uk/ons/guide-method/census/2011/index.html

The release is supported by data visualisation, statistical bulletins, with commentary and short stories on international migration, ethnic group and identity, religion, and (in Wales) Welsh language. The release is also supported by a technical report comparing the estimates of economic activity to the regular labour market statistics ONS produces. Frequently asked questions will be available on the ONS website to help you answer census related queries.”

And here’s some mapping of the results:

You’ll see that these stats are at Local Authority level. The next releases will be:
• 30 January 2013: all of the Key and Quick Statistics tables at Output Area, ward, parish and parliamentary constituency levels.
• By the end of February 2013: Key and Quick Statistics for the remaining geographies, (including postal sectors) together with postcode estimates of population by *gender*, and the number of households.

December 6, 2012

"Transport costs are eating into family budgets, the ONS says" – Living Costs and Food Survey

The Guardian reports:

“British households have cut back on clothes and furniture spending to cope with a sharp rise in petrol bills, according to the government's annual Family Spending survey. The Office for National Statistics revealed that households on average spent a record £483.60 a week in 2011, up £10 on the year before – but much of the increase was gobbled up by a 9% rise in spending on petrol. The figures also reveal that spending is falling fastest among the poorest, and suggest the "squeezed middle" may be a statistical myth.”

You can read more at: http://www.guardian.co.uk/money/2012/dec/04/household-spending-record-high-rising-petrol-prices  
And here’s the link to the survey report: http://www.ons.gov.uk/ons/rel/family-spending/family-spending/family-spending-2012-edition/index.html


December 5, 2012

Top 10% of households are 850 times wealthier than the bottom – ONS

The Guardian has reported the latest results from ONS’s Wealth and Assets Survey:
& here’s a link to the ONS website:


November 23, 2012

2011 Census latest release – first estimates (Population & Households) at Output Area level for England and Wales

ONS has just sent an alert announcing the publication of population and household estimates at Output Area and ward level for England and Wales. The release is available on the ONS website at: http://www.ons.gov.uk/ons/guide-method/census/2011/index.html

You’ll see that it’s region-by-region, and in Excel format (DUG has been pressing for csv for future releases, and ONS has promised it). This follows the earlier publication of geography products such as OA boundaries (see DUG News 31 October)

ONS’s announcement continues:

“The first phase of the second release is planned to commence on 11 December with a full package of key statistics and a selection of quick statistics tables, for England and Wales. This will be supported by statistical bulletins for England and Wales, for Wales (in English and Welsh), with commentary and short stories on international migration, ethnic group and religion, and Welsh language. The release will be accompanied by data visualisations and by a fact pack product that provides quick reference 2001-comparability information for each local authority. Table layouts will be published prior to the release.

On 30 January 2013, we will issue all of the Key and Quick Statistics tables at output area, ward, parish and parliamentary constituency geographies. These outputs will also be accompanied by a statistical bulletin and census analysis. By the end of February 2013, we will issue the Key and Quick Statistics for the remaining geographies such as Welsh Government devolved constituencies, National Parks and health areas together with postcode estimates by [*gender*] and for the number of households. We will announce the precise date in February as soon as we can and at least four weeks prior to that publication.”


November 23, 2012

Open Data chief says postcodes should be free

Just in case you haven’t spotted this in “Civil Service World”, there’s more support for the PAF being made free…….

“The UK’s postcode address database should be released for free use by private companies, the chief executive of the new Open Data Institute (ODI) has told CSW”.

Speaking in an interview with CSW, Gavin Starks argued that the UK’s Postcode Address File (PAF) is “such a fundamental building block for pretty much every business that needs an address that it’s kind of crazy that we don’t have that as a properly open file.”

You can read more here: http://network.civilservicelive.com/news/csw/read/711763/open-data-chief-says-postcodes-should-be-free  
& here’s a bit more about the ODI’s recently-appointed chief, Gavin Starks: http://www.theodi.org/odi-top-jobs


November 21, 2012

GeoLytix release full Postal Boundary Set as Open Data

Here’s news of a really encouraging step forward with more (free) Open Data. The announcement reads:

“Today, GeoLytix are pleased to release our full set of Autumn 2012 GB postal sector boundaries under an Open Data license. Postal boundaries are a critical component of our countries spatial infrastructure. They are used in everything from defining media boundaries and store delivery areas to web map searching, analysing customer data and producing a wide range of maps for the public.
You are free to use and exploit these data in any way you see fit with the single requirement that you attribute GeoLytix, the Ordnance Survey, the National Statistical Agencies and the Royal Mail as the copyright holders. Postal Geography constantly changes and this Open Data version is frozen. Updates and support are available under a paid-for GeoLytix license.”

The story behind their creation is at: http://geolytix.co.uk/blog/ . You can access the product documentation including the license, an FAQ and the user guide together with the full data here http://www.geolytix.co.uk/data-and-software/postal-boundaries  (no registration required)


November 12, 2012

APPSI paper on a National Information Framework for Public Sector Information and Open Data – Call for comments

Government’s Advisory Panel on Public Sector Information (APPSI) has just published this paper, which supports the case for a strategic approach to getting better access to government information such as address files, property information, and map data:


APPSI’s views should help create more momentum for Open Data. You’ll see some more background below. If you have comments on the ideas and proposals, please email them to secretariat@appsi.gsi.gov.uk


November 10, 2012

George Osborne champions science, & The Data Revolution is first on his list

Here’s some encouraging news from a meeting yesterday at the Royal Society.

George Osborne, Chancellor of the Exchequer, spoke of both his belief in the value of science as a driver of the UK economy and his commitment to science funding into the future. He identified eight areas that the government believes that Britain is a world leader.

The first on his list was “The Data Revolution and energy-efficient computing”. His comments included:
“We have some of the world’s best and most complete data-sets in healthcare, demographics, environmental change and food.”

“We have also created the open Data Institute in the East of London in Tech City, to bring together study of all the data from our whole environment. And as a government we are making more and more of that data available. Business will invest more as they see us invest more in computational infrastructure to capture and analyse data flows released by the open data revolution.”

It’s good to see data at the top of the agenda. You can see the full detail at: http://royalsociety.org/news/2012/osborne-at-royal-society/


November 1, 2012

Open Data User Group calls for address files to be made free of charge

It’s great that the Open Data User Group has taken up the cause to make address files to be made free of charge. Here are the headlines:

• ODUG calls for the publication of the Postcode Address File (PAF) as open data under the Open Government Licence
• Charging for all Postcode Address File (PAF) licencing should end
• The Ordnance Survey AddressBase Plus and the National Street Gazetteer (NAG) should be released as open data in conjunction with the Postcode Address File (PAF)

……and you can see more at: http://data.gov.uk/odug 

Many thanks indeed to Sarah Hitchcock, and to other members of DUG who have helped make the case.


October 31, 2012

2011 Census Geography products now available (in advance of statistics next month)

Here’s news from ONS about the geography products that they have just published in advance of the Output Area level statistics that start to flow (with counts of population and households) on 23 November:


You’ll see from the links that they comprise:
• 2011 Census boundary files (.shp) for OAs, LSOAs and MSOAs.
• 2001 Census boundary files (.shp) for OAs, LSOAs and MSOAs.
• 2001 to 2011 Census geography OAs, LSOAs and MSOAs lookup files
• 2011 OA to higher area lookup files
• Postcodes to OAs, LSOAs and MSOAs
• 2011 Census median population weighted centroid files (.shp) for OA, LSOA and MSOAs datasets

We learn that only 2.6 per cent of 2001 OAs have changed as a result of the 2011 Census, with the average population in an OA increasing from 297 in 2001 to 309 in 2011.


October 24, 2012

The DUG Conference “Retail Issues, Big Data, and Research”

and the winner of our 2012 Award for “Better information from government” is ................................The Department for Work and Pensions

The DUG Conference was held on 10 October at the Royal Society, and you can see the programme, presentations, and a list of attendees at the tab on the left.
The Department of Work and Pensions, represented by its Director of Information David Frazer, was present to collect the DUG Award to the Department “for proactively collecting, re-using, and publishing more and better data”.

A DWP Press Release http://www.dwp.gov.uk/newsroom/press-releases/2012/oct-2012/dwp105-12.shtml  includes:
Minister for Welfare Reform Lord Freud said:

 “The Government has a responsibility to be open with data, so we can be more accountable to the public as well as provide the public and private sectors with the tools to plan and improve services.
“Better data also gives companies the information they need to find opportunities for growth, which benefits us all.
“I want to thank the DWP teams working to improve our data and for allowing greater public access of our information resources and to congratulate them on winning this award.”

Our citation reads:
“Commercial companies, when making decisions about targeting investment towards particular types of customers and local markets, welcome the availability of government statistics across the United Kingdom. The Census of Population is one major source, but statistics derived from government departments’ administrative files present a growing opportunity.
The Department for Work and Pensions has led the way, using the information that it collects from the UK’s citizens to create anonymous counts of various populations for very small areas, and making these easily accessible. It is also encouraging the wider use of government surveys, with the Wealth and Assets Survey benefiting from the addition of a neighbourhood classification.
Members of the Demographics User Group (DUG) strongly welcome these developments, and the future plans outlined in the DWP Open Data Strategy, which will enable better understanding of customers and increase the efficiency and growth of businesses”


October 24, 2012

Government review of open data to explore growth opportunities – Chaired by Stephan Shakespeare

Here’s news of a new government review which “aims to improve access to data and help promote economic growth by examining the market for public sector information.”

It’s to be led by Stephan Shakespeare, Chair of the Data Strategy Board, who was guest speaker at DUG’s September meeting.

At the end of the review, recommendations will be made to Ministers on how to widen access and consider new and innovative opportunities for open data. In particular, the review will look at the following areas:
• the current use of public sector information within government, the private sector and by the general public;
• the opportunities for innovation and developing new data services with public sector information;
• the constraints on developing new services with public sector information and the role that current funding models play;
• the impact of competitiveness on existing and new businesses entering the market, as well as the way in which organisations secure access to data; and
• an audit of current and past studies, as well as taking stock of the current use and re-use of public sector information.

There’s more information at: http://news.bis.gov.uk/Press-Releases/Review-of-open-data-to-explore-growth-opportunities-68203.aspx 


October 23, 2012

2011 Census stats on second address / second homes

ONS has just published stats from the 2011 Census on the number of people with second addresses in local authorities in England and Wales.

The BBC has run the story http://www.bbc.co.uk/news/uk-20026773 together with mapping http://www.bbc.co.uk/news/uk-19759650  

The data can be accessed here: http://www.ons.gov.uk/ons/guide-method/census/2011/index.html

The release includes two separate tables which provide, down to local authority level:

 • the number of people who spend more than 30 days a year at a second address in a local authority where they do not usually live, and
• the number of people usually resident in each local authority who had a second address elsewhere


October 13, 2012

Public Administration Select Committee – Study 4: Communicating and publishing statistics – Submission on behalf of the Demographics User Group

Following our response to the first of House of Commons Public Administration Select Committee’s studies on statistics and their use in Government (see August 4), another topic is “Communicating and publishing statistics”:


You can see DUG’s submission here. The key themes are summarised as the need to:

• Put more emphasis on “the last 100 yards”, making it easier to find and use data.
• Shift some resources from statistical production to marketing (identifying needs and meeting them) within the various parts of the Government Statistical Service.


October 13, 2012

ONS's Beyond 2011 Project – papers, & free seminars

Following the news about ONS’s Beyond 2011 consultation, looking at alternatives to another Census (see September 26), here’s the latest:

1) News of research work being undertaken, & the evaluation of options: http://www.ons.gov.uk/ons/about-ons/what-we-do/programmes---projects/beyond-2011/index.html

2) Seminars. Here’s a message from ONS:

“You may be interested to know that we will be running a series of seminars during November as part of our ongoing engagement with our stakeholders.

Seminars are planned for the following dates and locations and will run from 10.30 -15.45:
• 26 November 2012 – Manchester
• 28 November 2012 – London – Royal Statistical Society

During the day we plan to cover developments in the Programme, including our investigations into the options, research progress and how we are taking forward the business case and quantification of benefits. There will be the opportunity to participate in group work and a panel session in the afternoon. If you wish to attend one of these seminars please email beyond2011@ons.gov.uk  stating the date and location you would like to attend and the organisation, if any, you will be representing. Please note that there are limited places at these events and places are allocated on a first come first served basis.” 


September 26, 2012

Will there be another traditional Census? News of two government reports

Here’s news of two separate government investigations into whether there should be another traditional Census in 2021.

1. House of Commons Science and Technology Committee:

“MPs have warned that a decision to discontinue the census would have a negative impact on the Government's social science analysis, as well as social science research in the UK, and that alternative systems for collecting population data may not prove to be any cheaper.” You can see the committee’s report at:

2. ONS’s Beyond 2011 User Requirements Consultation

This report provides the results of the first Beyond 2011 public consultation which ran from October 2011 to January 2012 (you can see DUG’s submission below on 17 January 2012). A total of 266 organisations and individuals provided responses to the consultation and 207 people attended the supporting workshops.

The consultation focused on two broad areas:
• the current and future requirements for population and socio-demographic statistics on different topics; and,
• the trade-off between accuracy, geography and the frequency at which the statistics are produced.
You’ll find the report at: http://www.ons.gov.uk/ons/about-ons/what-we-do/programmes---projects/beyond-2011/news-and-events/reports-and-publications/index.htm

An ONS statement also includes:
“In October 2012 we plan to release a whole range of papers on this research work undertaken over the past 6 months. These will explain much more about the approach we are taking to evaluate the options and what the next steps are. We will be reporting much more regularly from now on with further releases later in 2012 and early in 2013”. This will give us the opportunity to learn much more. 


September 7, 2012

ONS's 2011 Census Newsletter – September 2012

Here is the latest from ONS about the release of 2011 Census statistics, including news of the first results, details of the next Census data release - planned for between November 2012 and February 2013 and available via the NeSS website – and a summary of the Census Statistical Disclosure Control methods applied to protect the confidentiality of individuals and entities such as businesses.


August 30 , 2012

Marks & Spencer customer poll suggests better times ahead – Guardian report quotes Steve Bond

Here’s news from yesterday’s Guardian:

“Marks & Spencer said on Tuesday that the latest poll of confidence among its shoppers had delivered its most positive score in more than two years, providing some good news for its embattled chief executive, Marc Bolland. The high street chain is regarded as a barometer of consumer spending and its authoritative monthly poll involves interviews with 4,000 customers.

M&S's director of customer insight and loyalty, Steve Bond, said the latest reading for the four weeks to 17 August was 12 points higher than a year ago. Optimism was riding high, he said, thanks to the feelgood effect of events such as the Olympics, although he cautioned that the poll, launched by Bolland in May 2010, had only ever been conducted in a downturn.

The "most subdued group" in the marketplace remained families battling higher food and fuel prices, added Bond”.

You can read more at: http://www.guardian.co.uk/business/2012/aug/28/marks-and-spencer-increased-consumer-confidence 


August 21 , 2012

ONS: Glen Watson appointed as Director General

Following the retirement of Steve Penneck, it’s good to see that Glen Watson has been appointed as the new Director General of ONS.

Glen is currently Director of Census at ONS, having previously been appointed Director of Neighbourhood Statistics in 2004 (when he spoke at the first DUG conference).

Here’s ONS’s announcement: http://www.ons.gov.uk/ons/media-centre/statements/new-director-general/index.html?format=print 


August 15 , 2012

ONS/BIS Meeting on "The Changing Shape of UK Business" – Tuesday 6 November 2012

For those of you interested in stats about businesses, here’s advance news of an all-day event, coordinated jointly by the ONS and the Department for Business, Innovation and Skills, whose objectives are to:

• “share what we know about the structure, content and performance of UK business;
• raise awareness of the importance and coverage of our statistics;
• hear from users about the range of uses to which our statistics are put; and
• discuss future plans and priorities with a range of users.”

The event is free of charge, but please note that numbers will be limited. You’ll find more detail at:


August  8 , 2012

What is a data scientist worth? Read all about it in the Guardian

A DUG member has spotted this nugget:


It’s clear that if you are a “Data Scientist” you can earn more than if you are a “Data Analyst”…….


August 4, 2012

Public Administration Select Committee – Statistics and Registration Service Act 2007 – Submission on behalf of the Demographics User Group

Following the session with Bernard Jenkin, MP, Chair of the House of Commons PASC, at the DUG meeting in March, and our letter of 1 May (see below), the committee is launching a programme of work on statistics and their use in Government. It is conducting its scrutiny by means of ten short “studies”:

The first of these is on the Statistics and Registration Service Act 2007. DUG has made a submission, with the key themes being the need to:
• Understand and meet the needs of users outside government
• Divert some resources from production into improving access to statistics, and hence increasing use, and maximising value: “the last 100 yards”
• Make much more use of administrative files to create statistics at the lowest Output Area level
• Put more emphasis on speed of delivery, rather than marginal gains in quality 


August 2, 2012

Public Accounts Committee publishes report on Transparency /Open Data

The House of Commons Public Accounts Committee has just published a report 'Implementing the Transparency Agenda'. The Rt Hon Margaret Hodge MP, Chair of the Committee of Public Accounts, said:

"This Committee fully supports the principle of greater openness and its potential to strengthen accountability and drive improvements in public services. But the Government has a lot more work to do before that potential is realised.

It is simply not good enough to dump large quantities of raw data into the public domain. It must be accessible, relevant and easy for us all to understand. Otherwise the public cannot use it to make comparisons and exercise choice, which is the key objective of the transparency agenda”.

There’s more comment, plus a link to the full report, here:


July 27, 2012

Republic of Ireland – 2011 Census results – Part 2

Following the first publication summarising the results of Ireland’s 2011 Census (see 3 April below), here’s the second, which examines social and economic topics such as employment, occupations, education and skills and health related issues:

Again, you’ll see that the report has lots of eye-catching maps & charts. 


July 26, 2012

DUG Members’ needs for data from government – our Manifesto (Updated July 2012)

DUG’s Data Manifesto, which was initiated by Mike Whitelegge at M&S, has made an impact in Whitehall, and is proving a very useful summary wishlist / reference document.

Here is the current version, which you might find useful if you are challenged with “What more data would you like to see made freely available?” 


July 26, 2012

First meeting of the Open Data User Group (ODUG) – Match Report

The Cabinet Office’s Open Data User Group (ODUG) – see the earlier news on 10 July – has held its first meeting.

Heather Savory, the Chair, has written a blog http://www.guardian.co.uk/public-leaders-network/blog/2012/jul/18/open-data-group-clear-agenda and Sarah Hitchcock of Sainsbury’s tells me that the group got off to a good start.

The note of the meeting and slides are on data.gov.uk at: http://data.gov.uk/library/minutes-odug-10072012 and: http://data.gov.uk/library/slides-odug-10072012 


July 23, 2012

2011 Census – ONS's plans for the 2nd release (November – February), including lots of stats at Output Area level

ONS has now published its plans (subject to any redesign as a result of the application of statistical disclosure control) for its Second Release, which gives 36 tables of Key Statistics, and 71 tables of Quick Statistics, at Output Area level.

The second release will also include postcode estimates (numbers of people by gender in each postcode), and estimates of occupied households in each postcode will be produced.

You can see the detailed tables here:

“ONS consulted extensively with census users about requirements for the output of census statistics and consequently the vast majority of 2001 Census outputs will be repeated in 2011. This is for comparability and continuity purposes, and takes account of user satisfaction with the design of the 2001 statistics.

The statistics in the second release will correspond to the 2001 Census key statistics tables produced for univariate information (counts and as percentages), univariate tables and headcounts. A look-up file is available to download. The look-up file will be developed to be relevant to every release of 2011 Census statistics.” 


July 16, 2012

2011 Census – First results for England and Wales – Largest population increase shown by any Census since 1801

The ONS has just published the first (Local Authority level) results from the 2011 Census for England and Wales: www.ons.gov.uk/census  

There’s some good summary coverage on the BBC website, which starts:

“The population of England and Wales has reached 56.1 million, up by 3.7 million in a decade, the 2011 Census shows.
It is the largest growth shown by any census since they began in 1801.
The population in England was 53 million on 27 March 2011 when the census was taken, about 400,000 more than official estimates. In Wales the population was 3.1 million.
Around 55% of the 3.7m increase was attributed to net migration.
The census showed that the population for England and Wales was greater than official estimates rolled forward from 2001 - by just over 500,000.”

You can read more at: http://www.bbc.co.uk/news/uk-18838540  


July 14, 2012

Data Strategy Board (DSB) – Department for Business Innovation & Skills lists its members

Following hard on the heels of the announcement of the membership of the Open Data User Group (see 10 July), the Department for Business Innovation & Skills has news of the new Data Strategy Board: http://www.bis.gov.uk/transparency/data-strategy-board

If you follow the link to the note of the meeting on 11 July, you’ll see who has been appointed. The three places reserved for “data users outside the public sector” have gone to:

Chris Yiu, Head of Digital Government Unit, Policy Exchange
Bill Roberts, CEO, Swirrl IT Ltd
Steve Thomas, MD Strategic Development, Experian Plc

It is striking that (unlike Sainsbury’s representation on ODUG), the Department for Business has not included any representative of the huge existing community of business end-users. Rather, we have a Think Tank, and two Value-Added Resellers / Publishers – indeed, you might have spotted that Experian has seats on both ODUG and the DSB. 


July 10, 2012

Open Data User Group (ODUG) – Cabinet Office announces members – including Sarah Hitchcock of Sainsbury's

Following the earlier news (June 17) about the launch of ODUG, it’s really good to report that Sarah Hitchcock, Head of Network & Planning at Sainsbury’s, has been appointed as one of the 12 members.

Here’s the Cabinet Office’s announcement:

You’ll see that other members of the Group also include Bob Barr (Manchester Geomatics), and Mick Cory (Chair, UK Location User Group), plus a representative from Experian.

ODUG will add to the pressure that we’ve been applying since 1998 for more Open / Free Data. 


June 29, 2012

The Government's new Open Data White Paper: Unleashing the Potential

The Government has just published its “Open Data White Paper: Unleashing the Potential”

“……..which sets out how we’re putting data and transparency at the heart of government and public services. We’re making it easier to access public data; easier for data publishers to release data in standardised, open formats; and engraining a ‘presumption to publish’ unless specific reasons (such as privacy or national security) can be clearly articulated. From the Prime Minister down, central Government is committed to making Open Data an effective engine of economic growth, social wellbeing, political accountability and public service improvement.”

At the launch event yesterday (our DUG Award winner) Francis Maude gave an evangelical speech – the White Paper has a big ambition, and there’s no doubt that he means every word of it.

The link http://www.cabinetoffice.gov.uk/resource-library/open-data-white-paper-unleashing-potential leads to both the White Paper, and a document which gives each Department’s plans for releasing more data. 


June 28, 2012

Job vacancies

Several new job vacancies in DUG member companies can be seen at the tab to the left. 


June 25, 2012

2 reports: Deloitte's on Open Data, and the Royal Society on Open Science

The momentum in favour of open data continues to grow, and this is illustrated by two new reports:

1. Deloitte’s “Open Data – driving growth, ingenuity and innovation”

“Welcome to our first white paper in a series of research reports and white papers from Deloitte Analytics on the topic of ‘competing on insights’.
Open data – Driving growth, ingenuity and innovation has been researched and written in collaboration with Professor Nigel Shadbolt, head of the Web and Internet Science Group at the University of Southampton and co-director of the Government’s new Open Data Institute.
This publication presents Deloitte’s vision for open data – a subject that has, until now, been overshadowed unhelpfully by considerations of ‘big data’. Open data is about much more than improving government transparency and it is now emerging from its silent revolution……….”
You can see more at: http://www.deloitte.com/view/en_GB/uk/market-insights/deloitte-analytics/de0d882044ad7310VgnVCM3000001c56f00aRCRD.htm  

2. The Royal Society’s “Science as an open enterprise”

This report “aims to identify the principles, opportunities and problems of sharing and disclosing scientific information and asks how scientific information should be managed to support innovative and productive research that reflects public values.” The blockbuster 105-page report can be downloaded at:
Alongside this project, Research Councils UK in partnership with JISC, the Royal Society and Sciencewise-ERC, commissioned a public dialogue exercise on open data, aiming to engage the public with issues around open data to potentially inform future policies in this area. The report of this exercise is can be downloaded from the same link. 


June 22, 2012

Scotland – 2011 Census – News of plans for outputs, & timing

Following the news from ONS about its plans for outputs for England & Wales, and their timing (see news on April 3), National Records for Scotland have now released their 2011 Census Output Prospectus and set out a timetable for a phased release beginning with the first outputs in December this year. The dataset release phases are similar in content to those in England & Wales but will follow six months behind those planned by ONS.

You can see the full detail at: http://www.gro-scotland.gov.uk/files2/the-census/progress-reports/census-2011-outputs-prospectus.pdf 


June 20, 2012

ONS and UCL work on an updated 2011 Output Area Classification

Following DUG’s response to the consultation on plans for a new UK Output Area Classification (see news on 23 March), ONS has now published a summary of the responses: http://www.ons.gov.uk/ons/guide-method/geography/products/area-classifications/ns-area-classifications/new-uk-output-area-classification/index.html

You’ll see the conclusions & next steps which, taken in the context of DUG’s views, look very encouraging. 


June 18, 2012

ONS: "First results from the 2011 Census will be released on Monday 16th July"

Hot news from ONS………

These stats will be populations by age and [gender], and occupied households estimates for England and for Wales (just at Local Authority level).

Key and Quick statistics, including many simple counts at Output Area level (NB), plus postcode populations, are due to follow on between November 2012 – February 2013. 


June 18, 2012

New statistics from DECC on fuel poverty for small areas

Here’s a link to new data on fuel poverty in England from the Department of Energy and Climate Change: http://www.decc.gov.uk/en/content/cms/statistics/fuelpov_stats/regional/regional.aspx

You’ll see that you can request data right down to Output Area level.


June 17, 2012

Open Data User Group (ODUG) – Cabinet Office Introductory Event 14 June 2012

The Cabinet Office held an event last Thursday (14 June) to introduce the Open Data User Group (ODUG). The event attracted a big crowd, and you can see the presentation slides here

ODUG will add to the pressure for more Open / Free Data that we’ve been applying since 1998. 


June 8, 2012

ESRC's Retail Research & Data – 17 Master's projects now underway

The ESRC is seeking to increase its engagement with the retail sector (http://www.esrc.ac.uk/funding-and-guidance/collaboration/working-with-business/  ) and an element of this is the Retail Research and Data project that Professor Paul Longley of UCL and Keith Dugmore are running.

We are really pleased to report that 17 Master’s projects which have been specified by retail companies (most of which are DUG members) now have students working on them. You’ll see it’s a fascinating list of topics: http://www.esrc.ac.uk/funding-and-guidance/collaboration/working-with-business/retail-masters-student-projects.aspx

We are hoping that the ESRC will be asking retailers to specify more projects towards the end of this year. 


June 1, 2012

News from ONS about 2011 Census Geography Products – Boundaries and Lookup Files

ONS has just published its 2011 Census Geography Prospectus. This contains details of what and when geographic products (digital boundaries and lookup tables) will be released. When available, these will be available to download from the ONS website or supplied on hard media from ONS Customer Services.

The first products will be available in October 2012 (subject to the Census outputs for Output Areas being published on time in November) and the second will follow in January 2013.

There’s a summary here:
and you can download the Prospectus itself, and also a FAQ document, e.g:

Q: Will the output area (OA)/super output area (SOA) and workplace zone (WZ) boundaries be free, including for commercial re-use?
A: The Office for National Statistics (ONS) has submitted a request that these boundaries will be available as Open Data, under Open Government Licence (OGL) terms and conditions. This would make them available for free including for commercial re-use. We are awaiting the decision from this process, expected at the end of June 2012.

& “We don’t anticipate that more than 5% of the OAs/SOAs overall across England and Wales will change”.

The anoraks amongst us will also be pleased to see that there’s the option to have OA boundaries clipped to the coastline. 


June 1, 2012

Job vacancies

Several new job vacancies in DUG member companies can be seen at the tab to the left.


June 1, 2012

Information Commissioner's Office consults on new anonymisation code of practice

“The Information Commissioner’s Office (ICO) has begun a public consultation on a new anonymisation code of practice. The code will provide guidance on how information can be successfully anonymised and how to assess the risks of identification. The ICO has also launched a tendering process to establish a network of experts to share best practice around the release of data in an anonymised form. Anonymisation techniques can convert personal data into a form so that individuals are no longer identifiable. The consultation will be relevant to any organisation that wants to release anonymised data, for example under the government’s open data agenda.”

See more at: http://www.ico.gov.uk/news/latest_news/2012/ico-consults-on-new-anonymisation-code-of-practice-31052012.aspx 


May 23, 2012

News of the Open Data Institute from Nigel Shadbolt & Tim Berners Lee

The Open Data Institute will “study the best ways to exploit the growing amounts of data being made public by the government, & train people to use the data in business and advise on what kinds of information should be released.”

The ODI was first announced in the Autumn Statement and now its business plan has been given the go-ahead by the government. A total of £10m of public funds have been pledged over five years to get the ODI going, but the cash is contingent on a similar sum being pledged by businesses.”

You can find out more at: http://epsiplatform.eu/content/shadbolt-tbl-outline-open-data-institute-uk 


May 23, 2012

Chair of the Open Data User Group (ODUG) appointed – Heather Savory

Following up the news on 19 March about ODUG:

“The Minister for the Cabinet Office, Francis Maude, has appointed Heather Savory as the chair of the Open Data User Group (ODUG). Heather has a background in high-technology businesses and strong strategic and governance experience of central government, entrepreneurial businesses and consulting.”

You can read more at: http://www.cabinetoffice.gov.uk/news/chair-open-data-user-group-appointed 


May 15, 2012

Generating Value from the 2011 Census – MRS Census Geodemographics Group seminar – afternoon, 19th September 2012

Here’s news of a seminar “Generating Value from the 2011 Census – How business can benefit from the new free Census data” which will follow hard on the heels of the first release of 2011 Census stats (scheduled for July).

You’ll see that the speakers include the ONS, and that the price is very reasonable (& even lower if you book before 30 June). 


May 15, 2012

ONS 2011 Census – News of plans for outputs, & timing........... detailed comments from the GLA/SASPAC team

Following ONS’s publication of its plans for Census Outputs, the Greater London Authority / SASPAC team has pulled together some detailed comments.

The issues raised include postcode headcounts, urban & settlement areas, flows data, geographical products, and workplace geographies – all techie stuff, but they might come back & bite us in future unless they are discussed with ONS soon. 


May 5, 2012

Retail Sector growth April 2012 – presentation by the British Retail Consortium & the Association of Convenience Stores

The Department of Business Innovation & Skills held a Retail Growth Strategy workshop on 17 April 2012.
Here is a presentation by the BRC and ACS on the shape of, opportunities for, and challenges to the UK retail sector – you might find some of the stats & graphics useful. 


May 4, 2012

Job vacancies

Several new job vacancies in DUG member companies can be seen at the tab to the left.


May 1, 2012

Open Data – Report by the National Audit Office: "Implementing Transparency

The National Audit Office’s newly published report “recognises the strength of the strategic case for greater transparency, and highlights progress across government in fulfilling most of its initial commitments. However, government needs a better understanding of costs, benefits and use to assess whether transparency is meeting its objectives of increasing accountability, supporting service improvement and stimulating economic growth……”

Some key facts:

• £16bn (NB!) is the Government’s estimate of the value of public sector information to the UK economy in 2011
• 7,865 data sets were linked to the www.data.gov.uk website in December 2011
• 47m estimated number of visits made to the police crime map website between February and December 2011

You can find more here: http://www.nao.org.uk/publications/1012/implementing_transparency.aspx  


May 1, 2012

DUG’s letter to Bernard Jenkin MP: the private sector’s need for government statistics and data – potential issues for Inquiries

Following DUG members meeting Bernard Jenkin, MP, Chair of the House of Commons Public Administration Select Committee, here is a follow-up letter.

It highlights 3 areas for potential inquiries by the committee:

1. The government’s Trading Fund business model severely limits the use that can be made of existing data, and needs fundamental reform. (This is referring to Ordnance Survey, Valuation Office Agency, etc.).
2. The use of government administrative files to create new statistics needs to be accelerated. (e.g. HMRC data on income).
3. Government suppliers of data need to put more emphasis on “the last 100 yards”, making it easier for new users, particularly small businesses, to find and use data. 


April 30, 2012

UCL's Centre for Advanced Spatial Analysis – Conference highlights using scoop.it

UCL's Centre for Advanced Spatial Analysis held its Smart Cities conference on 20 April. Here’s a report:

& media coverage included stories on:
• Crunch commuter data to track changing communities
• Urban analysis specialists launch the City Dashboard, a hub for real-time data on major UK cities
• How data could help fix London's transport woes
• Meticulous map shows every London bus journey

You can see more at: http://www.scoop.it/t/spatial-analysis 


April 30, 2012

Scotland's 2011 Census – Update on Statistical Outputs

Following the news from ONS about its plans for outputs for England & Wales, and their timing (see news on April 3), National Records of Scotland has published the first in a series of quarterly progress notes designed to keep users up to date on the statistical outputs from the 2011 Census:

The headlines are:
• Table outlines will be published very shortly together with a note of any significant changes made since they were first published in June 2011
• First 2011 Census results are planned for late 2012 (NB: delayed from the original July date) – a full output prospectus will be published around the end of May
• A lot of effort is being put into producing compatible statistics across the whole of the UK wherever possible 


April 12, 2012

Government seeking a Chair of the Data Strategy Board

Following up the news of 19 March about the government’s plans for open data, it is now seeking a Chair for its new Data Strategy Board:

“Outstanding Chair to deliver better access to public data
The Government has taken significant steps towards extending access to public sector information. The new Data Strategy Board (DSB) will advise Ministers on what public sector data should be released, initially from the four Trading Funds that make up the Public Data Group (PDG): the Met Office, Ordnance Survey, Companies House and the Land Registry. The data from these organisations and others across the public sector has the potential to unlock growth and create new opportunities for high-value businesses of all sizes across the UK.
.……………. To help realise our ambitions we are now seeking an experienced Chair to lead DSB in its formative years……..” 

April 11, 2012

Centrica joins the Demographics User Group

Great news that Centrica has announced its joining DUG. Here’s the press release:

“Centrica is the latest company to join the Demographics User Group (DUG), whose membership already includes major companies across a range of sectors.

Centrica is a top 30 FTSE 100 company with energy businesses in the UK and North America. Upstream it sources, generates, processes, trades and stores energy. Downstream, it supplies gas and electricity to millions of homes and businesses and offers a distinctive range of home energy solutions and low carbon products and services.

Centrica’s Customer Insight team sits within British Gas’s Commercial Marketing directorate. Their activity spans a range of disciplines from Analytics & Modelling to Market Research and it strives to help the business make great commercial decisions in its role at the heart of the business.

Senior Analyst, Harry Harrison stated "We are passionate about putting the customer at the heart of our activity and the more we understand them, the better the chances of us being able to achieve this aim. We are very pleased to be joining DUG and we look forward to sharing our own experiences and learning from the experience of others across different industry sectors to facilitate this aim." 


April 3, 2012

Republic of Ireland – 2011 Census results

“This report is the first of two publications summarising the results of the 2011 census. This first one, This is Ireland - Part 1, looks at overall population change by county; it also examines age, marriage, households and families as well as including first results on nationality, foreign languages, the Irish language, religion and housing……..”

You’ll see that the report has lots of eye-catching maps & charts:

April 3, 2012

ONS 2011 Census - News of plans for outputs, & timing

ONS has now published its Outputs Prospectus, which sets out the release plans for the 2011 Census statistics in England & Wales:

You’ll find the detail at the tab on the left: “Release plans for 2011 Census statistics”. There are four releases
First release July 2012: Age and sex, and occupied households estimates for England and for Wales (just at Local Authority level)
Second release November 2012 – February 2013: Key and Quick statistics (including many simple counts at Output Area level (NB), plus postcode populations)
Third release March 2013 – June 2013: Local Characteristics (including more detailed OA tables)
Fourth release July 2013 – October 2013: Detailed Characteristics, Detailed Theme and Armed Forces
“Subsequent releases” will include Workplace (& possibly Daytime) population.

There’s a tab on the right “Prospectus tables and schedule” to see a spreadsheet which gives lots of detail.

Scotland and Northern Ireland will be releasing their own prospectuses later this month.

March 30, 2012

Mary Portas High Street plans get government go ahead

“The government has announced it has accepted "virtually all" 28 recommendations from Mary Portas in its bid to rejuvenate UK High Streets. These will include creating dedicated "town teams" to manage High Streets and making parking more affordable. It has also unveiled additional measures, including funding schemes and cutting back on red tape……..”.

You can read more at: http://www.bbc.co.uk/news/business-17554053

& the official source: http://www.communities.gov.uk/news/corporate/2120114   

March 29, 2012

Ofcom publishes more detailed Broadband Availability data for England

DUG’s Data Manifesto (see February 6 below) includes the wish for data on Broadband coverage, & here is some news:

“Following a request from the UK governments Department for Culture, Media and Sport (DCMS), Ofcom, the communications regulator, has today published some of its “early” 2011 Infrastructure Report broadband data at a “more detailed geographical level“. It’s for England, and at Local Authority level. Here’s the link:

And the Ofcom (July 2011) report is here:

March 23, 2012

Output Area Classification (OAC) for 2011 – User Consultation – DUG's Response

ONS and University College London (UCL) have been consulting on plans for a new UK Output Area Classification:


The original classification was created for ONS by Dr Daniel Vickers at the University of Leeds using 2001 Census data (it won DUG’s first Award in 2005). Our response to ONS’s consultation is here

March 19, 2012

"Open Data – resist Sir Humphrey’s last stand" – article by Michael Cross

Michael Cross, founder of the Guardian’s “Free Our Data” campaign (& winner of our Award in 2006), is continuing the battle:

“You wait all year for a think tank to report on how government should embrace Big Data, then two do at once. Over the past week, both Demos and the Policy Exchange have published studies on the opportunities created by David Cameron’s pledge to create the world’s most transparent government.
One urges the government to get on with it; to create its promised “right to data” and let the private sector and concerned citizens do their best with it.
The other counsels caution - the transparency revolution will happen only when government departments are equipped with new IT systems to collect and disseminate accurate data, and civil servants and the general public educated to make sense of it.
Wonk-watchers will guess which philosophy emerged from which think tank. Anyone who has worked with government will guess which approach the civil service and wider public sector will find more congenial……..”

Read on at: http://www.thecommentator.com/article/988/open_data_resist_sir_humphrey_s_last_stand


March 19, 2012

Government response to Consultation on Public Data Corporation; also Terms of Reference for Data Strategy Board, Public Data Group, and the Open Data User Group (ODUG)

The Government has now published its response to ‘A Consultation on Data Policy for a Public Data Corporation'. The views that we expressed on 11 October (see below) were shared by many of the 412 respondents.

 The Government has also published the terms of reference for the new Data Strategy Board (DSB) and the Public Data Group (PDG); “This marks a significant step towards embedding a culture of transparency and better access to Government data”.

You can see details of both at:

The Cabinet Office is also setting up Open Data User Group (ODUG). “This Group will give data users and re-users an opportunity to influence the release of Government data”: http://www.cabinetoffice.gov.uk/content/open-data-user-group  

All this will require continuing effort on our part, but it’s good to see that the tide is still flowing strongly in the direction of more open / free data.

March 14, 2012

More free data, signposted by Data4nr

The latest update from http://www.data4nr.net/introduction/  includes business stats for small areas; various health indictors; and road distances to food shops, GPs, Post Offices, etc.


March 14, 2012

2011 Census Evaluation

Here’s a link to information about ONS’s evaluation programme in general which also links to a download about internet completion take-up:


And another relates more to the 2011 Census campaign itself, including an insight into how ONS selected the languages for the translation leaflets:



March 7, 2012

Francis Maude speech to Information Commissioner's conference

Francis Maude gave a speech to the Information Commissioner’s conference yesterday, 6 March 2012. The draft text is on the Cabinet Office web site at:

He spoke on: Transparency; Data Sharing; Fraud, Debt and Error; Identity Assurance; Census; and Privacy. In particular:

Data Sharing:
“In May we will publish proposals that will make data sharing easier – and, in particular, we will revisit the recommendations of the Walport-Thomas Review that would make it easier for legitimate requests for data sharing to be agreed with a view to considering their implementation.”

“And the census is another area where I want to bust the myths around the complexities of data sharing.
I find it incredible that we haven’t fundamentally changed the way we approach the census since it began in 1801.
We are currently supporting the work by the UK Statistics Authority to look at census alternatives that will be cheaper and easier for the customer without compromising the privacy of the information. Technology has a critical part to play in enabling this.”

“I want us to reach a position where the citizen is driving the disclosure of their personal data wherever possible – with effective mechanisms so they can truly decide what they want to consent to and be able to change their mind.”


March 3, 2012

ONS's 2011 Census Newsletter – February 2012

Here is ONS’s latest Census Newsletter. You’ll see that it includes items on Outputs, Analysis, Web Data Access (including an unexpected mention of the Retail Sales Index) an application programming interface, and also some Roadshows in April:

 “ONS will hold a series of roadshows in April 2012 to keep stakeholders informed of progress to deliver the 2011 Census outputs and the planned web data access channels. There will be a demonstration of the planned tools and how these will help users to access and exploit ONS data. There will also be a workshop session to seek your feedback about the opportunities and challenges presented by the planned web dissemination tools. Further information about these events and how you can register, will be issued in March 2012.

Locations & Dates:
• Manchester 17 April
• York 18 April
• London 19 April
• Birmingham 25 April
• Newport 26 April
• London 27 April”


March 2, 2012

Press Coverage of ONS's "Measuring Well-Being"

Wednesday’s papers gave extensive coverage to ONS’s efforts to measure well-being, for example:

“This must be a heartening week for those wallowing in self-pity. Suddenly everyone seems to care how happy we are. Just as a campaign was launched yesterday to dissuade urban elderly people from retiring to the countryside, lest among other horrors they find themselves with neighbours resembling a cross between Nick Griffin and Margo Leadbetter, the Office for National Statistics also published a comprehensive report on who the happiest people in Britain are, and where they live”

Guardian: http://www.guardian.co.uk/commentisfree/2012/feb/29/ons-happy-scots-northern-irish

& the BBC: http://www.bbc.co.uk/news/uk-17192780

You can find ONS’s detailed report as a pdf at:


February 18, 2012

"The Changing face of poverty" – The Daily Mail publicises Charles Booth's famous map of London

A DUG member (who wishes to remain anonymous) tipped me off about an article in the Daily Mail:

“The changing face of poverty: Updated maps of London's poorest areas show epidemic of 'junk food' diabetes in same streets where Victorians died of malnutrition”:

It features Charles Booth’s famous map, which has been the regular Award at our annual Conferences
If you are interested in Booth’s maps, there’s more at the LSE’s Online Archive: http://booth.lse.ac.uk/


February 6, 2012

DUG’s Data Manifesto

Here is a manifesto of DUG members’ needs for data from government, which has been prepared for the UK Location Programme.


February 4, 2012

Tim Kelsey appointed as the new Executive Director for Transparency and Open Data

The Cabinet Office has announced the appointment of Tim Kelsey as the new Executive Director for Transparency and Open Data within its Efficiency and Reform Group (ERG).

Tim gave the Keynote address “Open Data: Making it Real” at our Conference in October: http://www.demographicsusergroup.co.uk/resources/Tim+Kelsey+DUG+2011.pdf  


February 4, 2012

Job Vacancy – Boots

The latest job advertisement – Boots in Nottingham seeking a Portfolio Analyst – can be seen at the tab to the left.


February 4, 2012

National Statistical Offices: Independent, Identical, Simultaneous Actions Thousands of Miles Apart – a view from the USA

The Director of the US Census Bureau has written about initiatives that are common across Anglophone statistical offices around the world:

1. The volume of data generated outside the government statistical systems is increasing much faster than the volume of data collected by the statistical systems; almost all of these data are digitized in electronic files.
2. As this occurs, the leaders expect that relative cost, timeliness, and effectiveness of traditional survey and census approaches of the agencies may become less attractive.
3. Blending together multiple available data sources (administrative and other records) with traditional surveys and censuses (using paper, internet, telephone, face-to-face interviewing) to create high quality, timely statistics that tell a coherent story of economic, social and environmental progress must become a major focus of central government statistical agencies.
4. This requires efficient record linkage capabilities, the building of master universe frames that act as core infrastructure to the blending of data sources, and the use of modern statistical modeling to combine data sources with highest accuracy.
5. Agencies will need to develop the analytical and communication capabilities to distil insights from more integrated views of the world and impart a stronger systems view across government and private sector information.
6. There are growing demands from researchers and policy-related organizations to analyze the micro-data collected by the agencies, to extract more information from the data.”

You can see the full article at: http://blogs.census.gov/directorsblog/2012/02/national-statistical-offices-independent-identical-simultaneous-actions-thousands-of-miles-apart.html


February 2, 2012

Land Registry releases data free of charge

Following the mention of Land Registry Data in the Autumn Statement (see December 6 below), here’s some more news:

“The UK Land Registry as of 30 January is giving access to their monthly transaction data free of charge. This is to be followed by the release of 'Price Paid' data in March 2012”.

There’s more detail at: http://epsiplatform.eu/content/uk-land-registry-releases-data-free-charge  


February 2, 2012

Cabinet Office Open Data consultation – summary of responses

Following our submission to the Open Data consultation (see October 11 below), the Cabinet Office has now published a summary of responses:

“There was a very positive response to the consultation. Launched in August, there were nearly 500 responses submitted before its closure in October.

The consultation set out a series of questions aimed at informing Government’s strategy to embed transparency and open data as core operating principles of the public services. Many of you expressed your support for the principles of open data, your confidence in its ability to bring about a positive change and the benefits it can deliver, including enhancing the performance and delivery of public services and fostering economic growth.”

In short, encouraging stuff.

You can read the detail at:


January 29, 2012

Living Costs and Food Survey – Users' meeting – Tuesday 20 March 2012

Various DUG members are interested in the ONS’s Living Costs and Food Survey (formerly the Expenditure & Food Survey), which is now coded with ONS’s OAC geodemographic classification.

Here’s news of a meeting:
“…….organised by the Economic and Social Data Service, which will provide a forum for data users and producers to meet and discuss new developments and exchange information about the Living Costs and Food Survey. The programme contains a mixture of papers from data producers and researchers. The meeting is free to attend and lunch is provided.

To view the programme and book a place please go to www.ccsr.ac.uk/esds/events/2012-03-20/  ”


January 24 , 2012

Eurostat's new vision for European Statistics

The National Statistician organised a meeting on 18 January which featured Walter Radermacher, Chief Statistician of the EU, talking about “A new vision for European Statistics” (here) to a large audience from ONS and departments such as DWP, HMRC, etc.

Keith Dugmore was invited to give the user view on the panel session, & had a chance to:

• Warn of the danger that EU officials might get an inside track on priorities in the same way as Government Departments do in the UK.
• Welcome the focus on making more use of admin data, better dissemination, and the importance of political support for Open Data in changing culture.
• Make the case for more stats at small area level, plus the necessary boundaries & mapping being freely available.


January 17 , 2012

Mapping of Urban Sprawl using open data from www.data.gov.uk

Blair Freebairn is exploring Open Data, generating new urban sprawl boundaries.

Here is a presentation which defines the current problem, shows what he’s done & why it’s better, and gives some examples.

If you would like more information, email blair.freebairn@geolytix.co.uk  


January 17 , 2012

ONS's "Beyond 2011" User Requirements Consultation – DUG’s Response

Following the earlier news item (October 27) here is DUG’s response to ONS.


January 11 , 2012

Sub-regional public and private sector employment in the UK

Here’s a new article on the rarely-discussed topic of analysing public and private sector employment in different sub-regions of the UK.
“The level of public sector employment in the UK economy has increasingly become a policy issue over recent years. To help inform the debate, ONS produces quarterly public sector employment data that measure the total level of public sector employment at the national and regional level. At present, however, there have been relatively few data releases or articles examining the sub-regional distribution of public sector employment. Furthermore, this distribution is of interest to policy makers both nationally and locally. This article seeks to fill this gap by providing an overview of some of the data available from ONS that can be used to examine the sub-regional distribution of public sector employment and also the sub-regional distribution of private sector employment.”


January 8, 2012

Letter to the Rt. Hon. Francis Maude MP - "The National Address Gazetteer: being run in the interests of Ordnance Survey, to the detriment of citizens and the private sector"

You can see this letter from DUG to Francis Maude here

In particular:
“...........For anyone outside the public sector, the prices for GB coverage and 100+ users range between £130,000 p.a. for the basic version to £189,000 p.a. for the full product. Even a single user would have to pay £24,000 p.a. These prices are prohibitive for almost all potential customers, and appear designed to protect OS’s existing policy of setting high prices for a small captive market.”


January 7, 2012

ONS is seeking users' involvement in designing its website for delivering 2011 Census outputs

ONS is currently developing its website to provide access to 2011 Census outputs, which are planned to start in July. It is seeking users’ views during January & February.

If you would like to learn more, please contact julie.peters@ons.gsi.gov.uk before the end of January.


January 6, 2012

RSS Open Meeting "Measuring Well-Being" - 8th March 2012, 5PM- c.7PM

Several DUG members have expressed interest in this topic: here’s a chance to catch up on developments at a 2-hour meeting after work – see the details here


January 5, 2012

Job Vacancies – Sainsbury's (closing date 22 January 2012)

Here's the latest job advertisement, with Sainsbury’s seeking Senior Location Analysts:

Earlier Job Vacancy announcements can also be seen at the tab to the left.


January 2, 2012

Happy New Year........... & here's a summary of our all activities in 2011

Looking back on the past year, you’ll find a summary of DUG’s activities during 2011 here

As you’ll see, it’s been another busy and successful year, with members sharing their experience in numerous ways, supporting the ONS in its case for what proved to be a successful Census, and the government both investigating the alternatives to another traditional Census, and continuing to promote the policies of Open Data: these initiatives led to Francis Maude, Minister for the Cabinet Office, receiving this year’s DUG Award at the Royal Society in October.


For earlier news 2009-2011, please see the Archive tab