Welcome to the Demographics User Group

Representing the needs of commercial users of information

Get alerted to future DUG news stories by entering your email address in this box....

DUG: More News

JULY 27, 2016

Cabinet Office's ethics framework for data science

The Cabinet Office has recently issued an ethics framework for data science which can be found here


JULY 27, 2016

The Geography of Creativity in the UK

 The above report can be found here


JULY 27, 2016

Transforming the Census - to 2021 and beyond - University of Winchester 28-29th June: Presentations are now available

 Copies of the presentations given at the conference are now available from the ONS webpage https://www.ons.gov.uk/census/censustransformationprogramme/newsandevents under the 'Recent events' heading.


JULY 15, 2016

ONS Approved Researcher Consultation - access now available to commercial companies

Some of you may remember that more than a year ago we made the case that commercial companies should have the same right as government and academics to access ONS’s individual record microdata.

We are now pleased to report that ONS has announced new data release policies: https://www.ons.gov.uk/news/news/onspublishesprinciplesofhowitsdataiscollectedusedandprotected

These include arrangements for Approved Researchers, features of which are that a) the project should serve “the public good”, and b) results should be published.


JULY 13, 2016

ONS Forrester Analytic Engine Review

 The above review can be found here


JUNE 10, 2016

All scientific papers to be free by 2020 under EU proposals

 “All publicly funded scientific papers published in Europe could be made free to access by 2020, under a “life-changing” reform ordered by the European Union’s science chief, Carlos Moedas.

The Competitiveness Council, a gathering of ministers of science, innovation, trade and industry, agreed on the target following a two-day meeting in Brussels last week.

The move means publications of the results of research supported by public and public-private funds would be freely available to and reusable by anyone. It could affect the paid-for subscription model used by many scientific journals, and undermine the common practice of releasing reports under embargo.”

You can read more athttps://www.theguardian.com/science/2016/may/28/eu-ministers-2020-target-free-access-scientific-papers?CMP=share_btn_tw


JUNE 5, 2016

ONS Customer Insights Survey

The results of the recent ONS Customer Insights Survey, in which Keith Dugmore was one of the interviewees, can be found here


May 25, 2016

ONS Geography Products and Services Review

Below is the ONS Geography Products and Services Review. The DUG response can also be seen below on the 8th March. Alternatively you can download the document directly by following this link


Today, we have published our response to the recent ONS Geography Products and Services Review. We would like to thank you once again for taking the time to respond to the ONS Geography Review.
We are pleased to see so many varied and interesting uses of our products and their application in research, analysis and service delivery across national and local government, academia and the private sector. The feedback we received is essential to ensuring ONS Geography can best provide users with efficient, refreshed and/or new geographic services. It has now placed us in a better position to target our research and identify how to improve our products.
You can read the full report here: https://geoportal.statistics.gov.uk/Docs/2016_ONS_Geography_Review_Report.pdf
If you want to know more, please contact ONS Geography Customer Services: ons.geography@ons.gsi.gov.uk or call 01329 444971.
We would like to hear your feedback about our Open Geography portal, please use the feedback tab that can be found on all pages to let us know what you think.
We are always interested to hear about your use of our data.  If you have time please let us know.
Customer Services
ONS Geography
(01329) 44 4971
We are available daily 08.30 – 4pm
Contact us at ons.geography@ons.gsi.gov.uk
Access Open Geography products at https://geoportal.statistics.gov.uk/geoportal
Follow us on Twitter @ONSgeography


May 16, 2016

New report: ‘Annual assessment of ONS’s progress towards an Administrative Data Census post-2021’

Today we publish our first annual assessment of our ability to replicate census-type information by using administrative data, and to move to an Administrative Data Census post-2021: 'Annual assessment of ONS’s progress towards an Administrative Data Census post-2021’ 
In March 2014 the National Statistician recommended that the 2021 Census should be predominantly online, and make more use of administrative data and surveys to enhance census statistics. This recommendation was endorsed by the Government’s formal response, with the ambition that ‘censuses after 2021 be conducted using other sources of data… [subject to] sufficiently validating the perceived feasibility of that approach’.
This new progress report uses a set of high-level criteria to assess our current progress and where we expect the assessment to be by 2023. The following need to be in place for ONS to move to an Administrative Data Census:
1. rapid access to existing and new data sources
2. ability to link data efficiently and accurately
3. ability to produce statistical outputs that meet priority information needs of users
4. acceptability to key stakeholders
5. value for money 
Also released today is our report: 'Summary of feedback on the 2015 Administrative Data Research Outputs


March 13, 2016

Independent review of UK economic statistics – Sir Charlie Bean's Final Report

Following news of the interim report in December (see December 11 below), Sir Charlie Bean's final report “Independent review of UK economic statistics” has just been published (all 259 pages):


Here’s the BBC’s coverage: http://www.bbc.co.uk/news/amp/35786538

The Royal Statistical Society has also been quick to comment, with Mike Hughes, Chair of the RSS National Statistics Advisory Group saying:

“The Bean Review is a substantial and important piece of work. It shows how the UK can have the data we need to drive our economy. We now look to the Chancellor to allocate the resources in next week’s budget to make this happen.

In particular the RSS believes the following recommendations are crucial to take into account, in order to resource ONS appropriately to address key challenges in our national economy. 

  • Removal of obstacles so more public administrative data can be accessed for statistical purposes, through changing the associated legal framework, whilst ensuring ethical safeguards are in place. We welcome that this should be accompanied by an independent person or body to adjudicate on any contentious ethical issues, and that access to data from the private sector should also be explored.

  • The establishment of a new centre of excellence for the analysis of emerging issues in the modern economy. ONS needs to build up its capability to make the most of existing and new data sources, and this should be facilitated through collaboration with academia, the private and public sectors, and internationally.

  • Recruitment of a cadre of data scientists to widen the ONS’ skills base, ONS needs to leverage its value as custodian of much of the data, to provide more of a service in the understanding of statistics. Support is needed for innovation and the skills base.”

You can see the full RSS Press Release here: http://www.statslife.org.uk/files/2016-03-11-RSS-CALLS-ON-CHANCELLOR-TO-COMMIT-TO-INVESTMENT-IN-DATA-INFRASTRUCTURE.pdf


March 8, 2016

ONS Geography Products and Services Consultation – DUG's response

You can see our response to ONS here.


March 2, 2016

Cabinet Office consultation on Data Sharing

After much preparation (you may remember the Cabinet Office presentation at the 2014 DUG Conference), the government has announced a consultation on new data sharing proposals:

“Proportionate, secure and well-governed information sharing between public authorities can improve the lives of citizens. It can also support decisions on the economy which allow businesses to flourish, and improve the efficiency and effectiveness of the public sector. The government aims to do more to unlock the power of data.

This consultation looks at enabling information sharing between public authorities to improve the lives of citizens and support decisions on the economy and society. The proposals fall into 3 categories:

  • Improving public services

  • Addressing fraud and debt


  • Allowing use of data for research and official statistics

    • giving the Office for National Statistics access to detailed administrative government data to improve their statistics

    • using de-identified data in secure facilities to carry out research for public benefit”


Here’s the link: https://www.gov.uk/government/consultations/better-use-of-data-in-government

The closing date is 22 April.


February 27, 2016

New ONS website

ONS has announced improvements to its much-criticised website http://www.ons.gov.uk/ and will welcome users’ feedback.


February 19, 2016

Goodbye, fish and chips: National Food Survey data reveals changing trends in British dining

The subject above says it all – as in the Guardian: http://www.theguardian.com/lifeandstyle/2016/feb/18/goodbye-fish-chips-national-food-survey-changing-trends-british-dining

And also the BBC: “10 ways the UK's eating habits have changed”: http://www.bbc.co.uk/news/magazine-35595530

Here’s the official link: https://www.gov.uk/government/news/new-data-reveals-the-nations-changing-tastes  (although it’s a bit ambiguous about the time period involved).


February 17, 2016

House of Commons Science & Technology Select Committee report: "The big data dilemma"

Following on from the call for evidence (DUG worked closely with the Consumer Data Research Centre in its submission sent on 3 September – see below), the House of Commons Science & Technology Select Committee has now published its report, which highlights the potential value big data has for the UK, and the action needed to use big data ethically and to its full potential.

There are many good recommendations which chime with DUG’s strategic priorities of Best Skills, Best Data, and Best Tools. These include:

  • A Government commitment to a substantial role in improving data skills, to ensure that we have the necessary big data skills base that the UK needs.

  • Opening government departmental data and breaking down data silos, so more datasets can be delivered to others in Government and to external users.

  • A right of access to data for the ONS, and developing a framework for auditing the quality of data within Government departments. 

And also:

  • A Council of Data Ethics within the Alan Turing Institute as a means of addressing ethical issues around big data and to ensure good practice around big data generally.

There’s more detail at:



February 13, 2016

New book: "Geodemographics for Marketers", by Dr. Barry Leventhal

Some of you might have already spotted the above news, and also know Dr. Barry Leventhal, who set up and has run the Market Research Society’s Census and Geodemographics Group for many years.

Barry has now captured his vast knowledge in a brilliant new book, which has the subtitle “Using Location Analysis for Research and Marketing”.

To quote Professor Martin Callingham:

“This book is a comprehensive account of the development, use, and future of geodemographics. …….. Barry has to be congratulated on putting together this excellent book and having persuaded so many other leading experts to contribute. It is written with great clarity, and answers the questions that would be raised in a reader’s mind before they occur.”

You can find the details here:



February 13, 2016

Big Bang Data – Exhibition at Somerset House

Mike Whitelegge (M&S) reports:

“If anyone finds themselves in London with a spare hour before the 20th March, I highly recommend a visit to the Big Bang Data exhibition at Somerset House.  Definitely worth a look for all my fellow data geeks!”

Here are the details: http://bigbangdata.somersethouse.org.uk/


February 8, 2016

Consultation on Changes to ONS Products – DUG response

DUG’s response to ONS is attached here.


February 7, 2016

Daily Mail interview with the National Statistician, John Pullinger

No, you didn’t imagine it………. John Pullinger has been interviewed by The Daily Mail’s “This is Money”.

And it’s a good article, giving him the opportunity to outline his radical vision for ONS, with the headlines “John Pullinger says 'radical change is coming' ” and “ONS must be more confident in engaging in political debate”, and

“Pullinger plans to make a major step into the digital sphere. The revolution includes collecting prices direct from grocery store websites, and making Britain a world leader by carrying out its ten-yearly Census online.”

You can find the full interview at:



January 24, 2016

A summary of DUG’s many activities last year

If you’d like to remind yourself of what DUG members did in 2015, or want to tell a new face what the group is about, you can see the 4-page DUG Highlights here.


January 14, 2016

"Can Big Data Replace the Census?" – MRS Census Geodemographics Group seminar – PM, Monday 7th March, 2016

There has been much talk about dropping the traditional population census and obtaining equivalent estimates from big data and administrative databases. To what extent is this achievable? How much is just hype?

This seminar will explore big data sources that are available now, or coming imminently, and what they can provide for geodemographics users. The main focus will be on spatially referenced data that can be applied down to small area level.

The seminar will bring together expert speakers from all sides of the information supply industry (you might spot some familiar faces), and will be chaired by Dr Suzy Moat, Associate Professor of Behavioural Science at Warwick Business School.

There is a discount if you book before 7 February, and here are more details:



January 8, 2016

How cities grow: the age of  houses– more fast mapping, similar to Datashine

The Guardian has picked up on visualisation of the different patterns of growth experienced in cities and towns across England and Wales shown by Oliver O’Brien for UCL’s Consumer Data Research Centre, using new data from the Valuation Office Agency: http://www.theguardian.com/cities/gallery/2016/jan/01/how-cities-grow-the-age-of-houses-mapped

There’s a link to the CDRC website for you to explore for yourself.


December 21, 2015

"Almost half of Britain's private wealth owned by top 10% of households" – ONS's latest Wealth and Assets Survey

Here’s the Guardian’s report on ONS’s latest Wealth and Assets Survey. It begins:

“Britain’s richest households have pulled further ahead of the rest of the population as house prices have accelerated, with the top 10% now owning almost half of the country’s £11.1tn total private wealth.

The Office for National Statistics (ONS) said the average household was worth £225,100 in 2012-14, when it carried out its latest survey of the country’s assets.

Since the previous survey two years earlier, the top tenth of households had seen a 21% increase in their wealth, including property and shares. That was three times as fast as the increase over the same period for the poorest half of households, who saw their wealth rise by 7%.”

You can find the full report at: http://www.theguardian.com/money/2015/dec/18/britain-private-wealth-owned-by-top-10-of-households

There’s a meaty paper here: http://www.ons.gov.uk/ons/dcp171776_428683.pdf

& here’s a link to the ONS website: http://www.ons.gov.uk/ons/rel/was/wealth-in-great-britain-wave-4/2012-2014/index.html


December 17, 2015

Public Data for the Private Sector – Important new report by the Longevity Science Panel

We are pleased to report that we have another ally in making the case that private sector analysts should not suffer discrimination when seeking access to public datasets.

Dame Karen Dunnell (the former National Statistician) chairs the Longevity Science Panel, which has produced a new report. The Press Release includes:

“A new report by the Longevity Science Panel calls for improvement in the provision of publicly-held data to enable both public and private sectors to deliver products and services to meet growing needs of an ageing UK population; especially in the areas of pensions, release of housing equity, housing and social care…….

The Panel holds the view that information collected from the public in the course of its use of public sector services must be available, with the right safeguards, to all who need to plan and provide services to develop our society and economy.”

Here’s the full report: http://www.longevitypanel.co.uk/_files/LSP_DATA2015_FINAL.pdf


December 16, 2015

Trainspotting stats – "London stations continue to dominate 'top of the stops' charts" 2014-15

The latest estimates of station usage published by the Office of Rail and Road (ORR) show that London Waterloo remains the most used station in Britain clocking over 99 million passenger entries and exits last year.

To view the full station usage dataset – it’s fascinating stuff – and accompanying report, see: http://orr.gov.uk/statistics/published-stats/station-usage-estimates

The report for estimates of station usage across Britain for the period April 2014 – March 2015 also shows that:

  • The total number of entries and exits at stations across Britain in 2014-15 increased by over 119m – a 4.5% rise compared to the year before.

  • There has only been one change to the list of top 10 stations most used stations this year, Stratford Station entered the list with over 30 million passenger entries and exits – a 17.4% jump compared to 2013-14.

  • Nine of the top 10 stations were located in London. Birmingham New Street was only station located outside London.


December 11, 2015

Independent review of UK statistics – Sir Charlie Bean's Interim Report

Following the announcement in August of a new official review of ONS’s economic statistics (see below, 6 August), Sir Charlie Bean’s Interim Report has now been published: https://www.gov.uk/government/publications/independent-review-of-uk-economic-statistics-interim-report

The Press Release starts:

“All data collected and held by public sector bodies should be opened up to the Office for National Statistics (ONS) to transform the production of economic statistics, according to a new report published today.

Professor Sir Charlie Bean is leading the Independent Review of UK Economic Statistics into how to future-proof ONS’s statistics so that they capture Britain’s fast-changing and modern economy. Today’s Interim Report sets out strategic recommendations, with a number of specific supporting actions, to help achieve his vision of cutting-edge economic statistics.

……..In order to fully exploit these new sources of data, including potentially also from the private sector, the report recommends ONS invest in technology and staff, including a cadre of data scientists. To ensure ONS moves ever closer to the cutting edge, the report recommends that it should constantly be on the lookout for new data sources and techniques, learning from businesses and statisticians everywhere…………...”

Three DUG member companies – Co-op, Sainsbury’s, and John Lewis – gave their views to the Treasury team.

The report is potentially significant, both in getting better data from ONS, and also helping ONS to create better data – for example, its recent initiative to obtain price and transaction data to better understand sales channels and market sizes. Recommended Action 5 reads: “Exploit new methods of collecting data and explore the scope for using information gathered by private sector entities in the production of economic statistics, nowcasting and one-off studies of emerging measurement issues”.

The final report is due in April.


December 10, 2015

A nation of tea drinkers? UK families' spending habits are changing – ONS's Family Spending Survey 2015

The Guardian’s reports on the ONS’s latest Family Spending Report:

“The continuing pressure on household budgets is revealed by official figures which show that families across the UK are spending less than they were at the start of the century.

The typical household in Britain spent £531.30 a week in 2014, according to the annual family spending survey from the Office for National Statistics, £7 higher than the year before. But the figure remains lower than the inflation-adjusted £542.50 average in 2001 and significantly below the peak of 2006.

The figures are at odds with GDP data that shows economic output has surpassed its previous peak levels. Households have been squeezed by steep increases in rent, electricity and gas payments over the past decade and the figures indicate they have failed to recover the spending levels of the past.”

There’s lots more fascinating detail here: http://www.theguardian.com/money/2015/dec/08/uk-families-spending-less-than-2001-ons-annual-survey

And here are two links to the ONS website:

http://www.neighbourhood.statistics.gov.uk/HTMLDocs/dvc203/treemap_fin.html (graphics)

http://www.ons.gov.uk/ons/rel/family-spending/family-spending/2015-edition/index.html (the report)


November 30, 2015

2021 Census – ONS's summary of the results from the Topic consultation

Following our response to this consultation (see below 28 August – you may remember our comments criticising its method), here below is news from ONS:

“The Census Transformation Programme ran a public consultation between June and August 2015 inviting responses to our initial view on content for the 2021 Census. Today we have published an overview of the received responses.  The 'You said' report is available in English and Welsh from the ONS website: http://www.ons.gov.uk/ons/guide-method/census/2021-census/consultations/index.html  

The report provides a brief topic by topic summary of responses and includes summary tables about the numbers of data users, agreement with ONS’s initial view, and the impact on users if data were not collected.

We are in the process of working through a full evaluation of all the 1,095 received consultation responses using evaluation criteria published as part of ’The 2021 Census – Initial view on content for England and Wales’.  We plan to publish a full response to the consultation in 2016, updating ONS’s view on the information that will be collected, what will not be collected, and where further work is required.”

ONS’s summary reports 1,095 responses to the consultation, of which 592 were identified as being from genealogists. Of the 279 responses from organisations, only 10 were from commercial bodies.

Turning to the Appendices, responses from individuals and organisations have simply been added together, giving no weighting to large well-informed user groups; and it also appears that a maximum of 249 individuals or organisations responded to the detailed questions about uses, geography, UK-wide, etc. This would appear to confirm our earlier view: “There is a danger that this (method) will have reduced the number of potential responses, and biased them towards specialist users in particular sectors. We urge ONS to adopt a more customer-friendly approach to all Census users with future consultations.”

Barry Leventhal (Chair of the MRS’s Census & Geodemographics Group) has added this comment:

“I fully agree with your comments, and I think that ONS should be urged to analyse the results by sector and take account of the importance of each sector in terms of census value.  It’s not long since ONS estimated value of census for each sector when making their case to the Treasury, so they should have this information.  On this basis, the consultation responses look highly skewed away from the valuable sectors – the 10 responses from commercial bodies ought to carry a lot more weight than the 592 from genealogists.”


November 4, 2015

UK's first "Women in Data" Summit on 26th November at UCL

Helen Dickens at GSK alerted us to this above event.

The agenda includes:

- Success stories: how women have planned, secured and maintained some of the top jobs in industry.

- Addressing the skills gap: staying ahead of the curve in skill acquisition.

- Key industry developments: gain insight on why the industry has shifted and what to expect over the next 12 months.

You can find our more at: http://www.datatech.org.uk/women-in-data.aspx


October 31, 2015

ONS's latest Population Projections "UK population expected to rise by almost 10 million in 25 years"

You might have spotted headlines last week such as:

“UK population expected to rise by almost 10 million in 25 years” (The Guardian)


“UK population to top 70 million in 12 years” (BBC)


“Migration will cause UK population to explode by almost 10 MILLION over the next 25 years” (Daily Express)


These (which treat projections of trends as forecasts or predictions) referred to an ONS announcement:

“2014-based National Population Projections have been published on the ONS website this morning. The release contains the detailed projections and underlying assumptions of population change for both the Principal Projection and selected variant projections produced. It also includes supporting information summarising the main results, describing the quality aspects of the projections, and explaining the methods used to produce them. Further variant projections will be released on 26 November 2015. See: http://www.ons.gov.uk/ons/rel/npp/national-population-projections/2014-based-projections/index.html

2014-based Subnational Population Projections for areas in England, constrained to this new set of national projections, will be published in May/June next year.”


October 25, 2015

DUG Conference 2015 – “How can data analyses best be communicated to decision-makers? Visualising – Distilling – Influencing”, and the DUG Awards

The latest annual DUG Conference at the Royal Society was a great success, with 112 attendees from commerce, government, and academia. We also made two awards:

The DUG Award for “Better information” went to GeoLytix “For making some Open Data more easily accessible to users, and distributing it freely”.

And this year we also made a DUG Personal Award to Stewart Robbins “In recognition of his contribution to DUG, especially in making time to create and deliver his unique training course “Developing as an Analyst” to 172 DUG members since 2007”

You’ll find the conference content at the tab on the left.


October 23, 2015

ONS Small Area Income Estimates published

Here below is a message from ONS about its new estimates of “Total weekly household income by middle layer super output area (MSOA), England and Wales, 2011/12”

Whilst this falls short of an ideal (income distributions + Output Area level + UK-wide + recent), it might be of value to some of you.


“The publication contains estimates of average weekly household income for middle layer super output areas in England and Wales for four different measures of income. The release includes the full data, presented with 95% confidence intervals along with a statistical bulletin describing the geographic trends of income, and a special chapter looking at the income estimates in relation to housing affordability, as an example of potential uses of the income estimates and how they can be used in conjunction with other small area data.”

“For those who are new to the income estimates, there is a brief user guide available explaining how the statistics are produced and their uses and limitations. For those who would like more detail about the models used to produce the estimates, there is a technical report which describes the background, model parameters and diagnostic tests carried out to ensure the quality of the estimates.”


October 21, 2015

Financial Times 19 October 2015: "Forget the big data hype — people want to know what it all means"

Following her attendance at the DUG Conference, the FT’s Emily Cadman has written about Big Data, and gives us a mention:

http://www.ft.com/cms/s/0/900312ae-7577-11e5-a95a-27d368e1ddf7.html#axzz3pC2IbNZc (but you may need to register, to hurdle the FT’s pay wall).


October 21, 2015

The CDRC Masters Research Dissertation Programme 2016 – a call for your projects now

Many of you will be aware of the opportunity for companies to propose projects for Master’s students that we have supported in the last four years.

Funding of the Consumer Data Research Centre by the ESRC is making it possible to run this is the longer term, and to start earlier (ideally this month and next), to maximise the chance of getting the pick of the best students for 2016.

Here’s more detail from Guy Lansley at UCL, who is again running the scheme this year.


Please start thinking about possible projects now, and don’t hesitate to get in contact with Guy to sound out your ideas.


October 14, 2015

UK Statistics Authority – Independent Assessment of population estimate and projections statistics

Those of you who regularly use government mid-year estimates or population projections for the 4 countries of the UK might find these new assessments useful:


As part of this assessment, UKSA has also published 'Population Estimates and Projections: A comparison across the UK', which is published alongside each of the Assessment reports, and can be found here:



October 1, 2015

Geodemographic classification of Workplace Zones in England & Wales (COWZ-EW) – now freely available

A provisional Classification of Workplace Zones for England and Wales (COWZ-EW) has been released by University of Southampton and is available from http://cowz.geodata.soton.ac.uk/

Users can also explore COWZ-EW visually using the wonderful DataShine mapping interface: http://cowz-ew.datashine.org.uk/

COWZ-EW is a geodemographic classification of the characteristics of Workplace Zones (WZs) based on data from the 2011 Census of England and Wales. It classifies WZs based on their similarity in terms of a range of census variables. COWZ-EW was produced by the University of Southampton in collaboration with the Office for National Statistics (ONS) and is freely and openly available under an Open Government Licence.


September 8, 2015

Workplaces – Numbers of workers by their residential neighbourhood type – Workplace Zones by OAC

As you might have guessed, this refers to new information created from the Census, which takes Workplace Zones, and which Output Areas the workers live in / come from. This enables us to see, for example, the types of neighbourhoods that workers at Canary Wharf live in.

We are really grateful to Chris Gale at the University of Southampton who has appended his 2011 Census Output Area Classification, and created tables with WZs as rows, and OAC (Super Group, Group, and Sub-Group levels) as columns.

We would also like to thank Professor Martin Callingham, who has plunged into the data (it’s 468 MB), and advises you to look for the last three Lookup files, which give the simple flow counts for groups 1,2,3, etc…. (rather than the earlier ranked files). Here’s the link: http://tiny.cc/OAC-WZ-Flows


September 3, 2015

Science & Technology Select Committee Inquiry: Big Data – CDRC (& DUG) Submission

Here attached is a submission by UCL’s Consumer Data Research Centre – developed in discussion with DUG members – to this House of Commons Inquiry on Big Data. The background to the inquiry can be seen here: http://www.parliament.uk/business/committees/committees-a-z/commons-select/science-and-technology-committee/inquiries/parliament-2015/big-data/


August 28, 2015

ONS public consultation on the 2021 Census in England and Wales – DUG Views sent to ONS

DUG was defeated by the immense detail of ONS’s online questionnaire (see June 16 news below), but we have:

1. Given our support to the submission by the Market Research Society, and MRS Census and Geodemographics Group.

2. Sent a note giving DUG’s summary view, which supports the MRS’s general approach, and amplifies some specific points.


August 18, 2015

DUG’s 12th Annual Conference: “How can data analyses best be communicated to decision-makers?" – Wednesday 7 October 2015, The Royal Society

This year our topic is “How can data analyses best be communicated to decision-makers? Visualising – Distilling – Influencing”, and we will again have more than 100 invited guests, enabled by financial support from the ESRC’s Consumer Data Research Centre.

The topic has been prompted by discussion amongst DUG members about recent developments in visualisation software, allied with our longstanding focus on how analysts can best influence decision-makers. You’ll see on the provisional programme that we have a great set of speakers, with the day including the latest news of research by the CDRC, Masters’ projects, and a workshop session to discuss the best ways of making an impact.

As always, the purpose of the conference is to bring together people from DUG's 15 member companies and also specially invited guests in government and universities, to spread knowledge and stimulate new ideas. This is DUG’s 12th annual conference, and we are convinced that increased dialogue brings enormous benefits to all parties.

As in previous years, the proceedings will be posted on this website,


August 13, 2015

Census Applications: Using the UK's population census data – 16 & 17 July 2015, University of Manchester – presentations now on the web

Following up the announcement on May 17 below, the presentations given at this 2-day event are now on the web at: http://census.ukdataservice.ac.uk/news-and-events/eventsitem/?id=4074

You might find it easiest to start by looking at the Programme, then spotting the presentations that are of interest, e.g.

  • Oliver O'Brien: DataShine: Automated Thematic Mapping of 2011 Census Quick Statistics 

  • Samantha Cockings, David Martin and Andrew Harfoot: Creating a classification of workplace zones 

  • David Martin, Samantha Cockings and Alan Smith:  Building 24-hour population models with 2011 Census data


August 6, 2015

Review of ONS – "Shake-up aims to bring official statistics into internet era"

The front page lead story in yesterday’s Financial Times broke news of a new official review of the UK’s economic statistics:

“Sir Charlie Bean, former deputy governor of the Bank of England, will on Wednesday put the nation’s number crunchers on notice that the status quo is untenable, arguing that the internet revolution has rendered Britain’s official statistics out of date. Speaking ahead of Wednesday’s launch of his official review into the state of the nation’s economic numbers, he told the Financial Times the current framework for the national accounts “was developed in the aftermath of the Great Depression”. While Big Data, rapid innovations in technology, and the rise of the sharing economy have transformed industries around the globe, official statistics have soldiered along largely unchanged, he argued. “As an economy develops, the traditional ways of thinking about it cease to be so relevant,” he said, flagging concerns that official numbers may be incomplete in an internet age where consumers, for example, book their holidays online, bypassing travel companies.”

Here are the lead story and an interview.

It seems inevitable that this will further fuel ONS’s interest in data collected by companies, which we are already discussing with them.


July 19, 2015

Reports by Universities UK and Nesta: "Securing the skills for the data revolution"

Following our recent contact with Universities UK and Nesta, they have just published two reports:

Universities UK’s report looks at the extent to which universities are producing graduates with the right data analysis skills.

Nesta’s policy briefing “Analytic Britain” has a series of recommendations that includes: embedding data analysis in subjects across the school curriculum, boosting the business skills of university graduates with statistics and programming skills, and developing innovative training solutions to keep the skills of the analytic workforce fresh in the face of rapid technological change.


The reports were launched at a meeting on 13 July, where the panel included Jenny Warrilow (Analytics leader, Boots), who made a powerful case for the needs of commercial companies.

The recommendations include a plan to “set up a cross–cutting taskforce around data analytics to identify good practices for education and skills provision, and spur collaboration across industry”. We hope to get DUG involved in this.


July 18, 2015

ONS Classification of Local Authorities, based on the 2011 Census

Which other places are very similar to………?

Here are the answers:

“ONS has released an updated Area Classification for Local Authorities. This classification uses 2011 Census data to identify clusters (groups) of local authority districts across the UK which have common key characteristics amongst their residents and households. Clustering aims to allocate local authorities to groups in such a way that local authorities within a group are as similar as possible, but the groups are as different as possible from each other.

There are 3 levels to this classification, a Supergroup level, which groups local authorities according to 8 different clusters, from which 15 different Groups were then identified, and then a further 29 Subgroups were created.

There are a number of documents to support this release, including maps, a methodology note, a description of each of the clusters in a Pen Portraits document, a Radial Plots document which provides a visual representation of the characteristics of different clusters according to 59 Census variables. There is also a Corresponding Local Authority District listing which identifies for each local authority, the 5 most similar local authorities – sometimes referred to as 'statistical neighbours'.”

The documentation for this release can be found on the ONS website at:



July 6, 2015

GB Road Traffic Counts

With thanks to James Nolan (formerly a member of a DUG company, and now with Knight Frank), here’s news from the Department for Transport, which collects traffic data to produce statistics on the level of traffic on roads in Great Britain.

“This website provides the underlying street-level data for download. The datasets on this site have been updated to include DfT traffic counts up to and including 2014. The 2015 data is due to be available and added to this site in June 2016.”

Here’s the link: http://data.gov.uk/dataset/gb-road-traffic-counts which leads to further links if you want to dig more deeply.


July 5, 2015

DataShine Scotland – Interactive Mapping of Scotland's Census

Following their earlier work on England and Wales (see June 28 2014 below), James Cheshire and Oliver O’Brien at UCL have now developed “DataShine Scotland”', a visualisation tool that maps the various Key and Quick Statistics from Scotland’s Census 2011.

In addition, on “DataShine Scotland Commute” you can access visuals showing travel-to-work-flows between each Council Area, split out by direction and mode of transport.

To find out more, visit the DataShine blog: http://blog.datashine.org.uk/2015/07/datashine-scotland/


June 25, 2015

Q&A on EU data protection reform

If you want to catch up on EU data protection reform, here’s news from the European Parliament:

“The growing globalisation of data flows, via social networks, cloud computing, search engines, location-based services, etc, increases the risk that people can lose control of their own data. Civil liberties MEPs adopted their position on a major overhaul of current EU data protection rules on 21 October 2013.

This overhaul seeks to put people in control of their personal data, build trust in social media and online shopping, and upgrade the protection of data processed by the police and judicial authorities.

EU member countries approved their negotiating position on the new EU data protection regulation on 15 June 2015. This means that Parliament, the Council and the Commission can now start the first round of three-way talks aimed at striking a final agreement on the regulation. These talks will begin on Wednesday, 24 June 2015. This general regulation on personal data processing in the EU will replace the current patchwork of national laws with a single set of rules, which should make it easier for companies to operate across the EU while at the same time strengthening citizens' rights.”

You can read a lot more here:



June 18, 2015

Population 24/7 methods – new Open Access paper from the University of Southampton

We’re really pleased to report news from Professor Dave Martin and Dr. Sam Cockings at the University of Southampton:

“You'll recall that for some time we have been working at Southampton on our "Population 24/7" research agenda to develop new methods for producing time-specific population estimates – indeed, it was presented it to DUG members some time ago. We are delighted to be able to report that a new Open Access paper (i.e. you don't need any university-based journal subscriptions to get it) by ourselves and Samuel Leung, has just been published in the Annals of the Association of American Geographers and you can get hold of it directly at:


Please feel free to share this with any DUG members likely to have an ongoing interest in time-space population research”.


June 16, 2015

ONS public consultation on the 2021 Census in England and Wales

ONS is now (in 2015) seeking to understand users’ needs for the topics for the 2021 Census:


A PDF of the "Initial View on Content for England and Wales", which runs to 105 pages, is available at http://www.ons.gov.uk/ons/guide-method/census/2021-census/consultations/index.html

You can respond directly online. DUG is aiming to submit a collective view. The closing date is 27 August.


June 14, 2015

The open data failures of the coalition and how it can be put right.....

Here’s an article by Simon Briscoe (who was on the panel at our conference last year), which starts:

“Which parts of the public sector are battling to open up data, and which are resisting it in the UK? There are so many moving parts it’s hard to say. To try and answer this, I produced a couple of open data maps to show how busy the coalition government had been with public sector open data…..”

You can read more at:



June 13, 2015

Open Data Challenge – News of the winners

The Open Data Challenges run by Nesta and the Open Data Institute aim is to encourage people to use (free) Open Data to build solutions to social problems: http://www.nesta.org.uk/blog/open-data-challenge-series-handbook-our-story-so-far

You can download the Handbook, which has projects on topics such as education, housing, and energy.


May 30, 2015

Professor Dave Martin receives top honour from the Royal Geographical Society

We are delighted to report that David Martin, Professor of Geography at the University of Southampton, is to be awarded the prestigious Back Award (2015) by the Royal Geographical Society.

As many of you will know, Dave has been a great ally of DUG over many years, especially on the Census, and has also spoken at several of our events.

“Professor Martin will receive the honour in recognition of outstanding research which has led to a new approach for compiling and analysing official statistics – in particular for the UK Census. His work to demonstrate the importance of small area data contributed to a government decision to retain the census for 2021”.

You can read more here:



May 27, 2015

Family Finance Surveys User Conference – 1 July, 2015 at UCL

The annual Family Finance Surveys User Conference, organised by the UK Data Service, is a full-day conference and is free to attend.

The conference will enable users to hear updates from the data producers on the main surveys, including the Family Resources Survey, Living Costs and Food Survey, and Wealth and Assets Survey.

The programme will contain a mixture of papers from data producers and researchers, including presentations based on analysis of the UK family finance surveys.

For more information and to book a place please go to:


The programme will be available soon.


May 20, 2015

Data Protection Legislation and the EU – a section in today's Financial Times

If you are trying to keep up with the above, today’s Financial Times has a (paper) pull-out section entitled “The Connected Business”.

The first article is “Data rules weave a tangled global web”, and another is “EU toughens its data protection regime”. There’s also a map of “Data protection laws around the globe”.

It’s online at:

http://www.ft.com/reports/the-connected-business (but you may need to register, to hurdle the pay wall).


May 19, 2015

Locating Free Data on the web – Visually

Former DUG member Lee Madden, who is now with EY, has spotted this, and posted it on our LinkedIn “Demographics User Group Network”:

“While surfing the net this morning I came across a rather useful resource on a data blog. The Blog - called Visually - is one that I normally use to get data visualisation ideas, but they have also provided links to a number of sites where you can access free data sets. There are a couple of "usual suspects" there (e.g. Data.Gov), but there are others highlighted too that I'd not heard of including Socrata (governmental data with data viz tools) and a link to the most viewed info-graphic on the web: ‘What are the odds’.”






May 17, 2015

Census Applications: Using the UK's population census data – 16 & 17 July 2015, University of Manchester

Here’s news of an event being organised by the UK Data Service at the University of Manchester. You’ll see that topics include “Creating a classification of workplace zones”, “Building 24-hour population models with 2011 Census data”.

“Now that nearly all census outputs are out and being used the UK Data Service are hosting a two-day conference to celebrate the UK censuses.

The conference will include three plenary sessions:

  • The value of the Census

  • Making use of the Census

  • Challenges for the future

General conference places will cost £100 per person to include drinks, lunch and conference dinner. Student places are available at £50 per person.

Residential accommodation should be booked separately. A Google map of hotels within walking distance of the conference venue is available.”

To view the draft programme and book a place please go to:



May 12, 2015

Francis Maude out as minister for the Cabinet Office as Matt Hancock takes the reins

Keeping an eye on the Cabinet Office, and whether the new minister will continue Francis Maude’s crusade for Open Data, we have some early news:



April 9, 2015

The Internet of Things – and the HAT project at the University of Surrey

Our contacts with academia have led to news of a project concerned with the Internet of Things. To quote Wikipedia, this is “the network of physical objects or "things" embedded with electronics, software, sensors and connectivity to enable it to achieve greater value and service by exchanging data with the manufacturer, operator and/or other connected devices”.

Roger Maull, Professor of Management Systems at the University of Surrey Business School, is running a project which is seeking to understand behaviour and motivations, including in the Retail arena. One example is that he is tagging bathroom products in people’s homes to understand usage.

There’s more about the project at http://hubofallthings.com/ 


March 27, 2015

Ordnance Survey – more Open (free) Data – Open Map, Gazetteer, Property Reference Numbers (??), and more

In what some might see as an Orwellian press release (given its longstanding resistance), Ordnance Survey announced in February that “it is leading the world in releasing open data”.

Here’s the link: https://www.gov.uk/government/news/britain-leads-the-way-as-ordnance-survey-helps-to-drive-economic-growth-and-digital-innovation-through-open-data

Getting to the meat of the announcement, it is good to see that the following datasets were promised for delivery by the end of March:

  • OS OpenMap: “a new street level dataset designed to be the most detailed open data mapping product available for Great Britain. It is an ideal backdrop for integrating, visualising and analysing other open datasets and is designed to underpin a new generation of desktop, web and mobile applications”
  • A new and improved Gazetteer (the most up-to-date place names, road names, road numbers and postcodes for use in location searches for both mobile and online uses)
  • Unique Property Reference Numbers (UPRNs), “which uniquely and definitively identify every addressable location in the country. This will enable users to improve the way in which they share and link information, attaching it to accurate address data and individual properties”

There’s also a new ‘presumption to publish’ process is being implemented “that will allow users to release more of their own data in an open format and will significantly increase the usability and value of public sector data built on Ordnance Survey maps” – this could be significant for mapping from Land Registry, for example.

Following this news, we now have some links:

Open Map “Released in Beta, this is the most detailed street-level open data vector mapping product available”:


Gazetteer of place names “with an accuracy of 1 metre”: http://www.ordnancesurvey.co.uk/about/news/2015/four-new-os-open-data-products.html

Unique Property Reference Numbers (UPRNs): https://www.ordnancesurvey.co.uk/about/news/2015/uprn-release-sharing-location-data.html

BUT…….. “Supporting the local government transparency and government open data agendas, Ordnance Survey are enabling AddressBase internal business use customers to release Unique Property Reference Numbers (UPRNs) on a royalty free and open basis.” So, it appears that this is only free to those organisations which already have AddressBase (the public sector, covered by the Public Sector Mapping Agreement, and a tiny number of other customers).


March 21, 2015

ONS Approved Researcher Consultation – DUG's views

ONS launched a consultation in February on access to secure microdata (individual records) in its Virtual Microdata Laboratories.

We felt that DUG should express a view for two reasons:

  • To support the general principle of better access / more use of ONS files
  • To make the case that commercial companies should be treated the same as public sectors and academic users (which is not the case at the moment).

DUG’s response is here.

It’s good to know that several other user groups (including some in the public sector, and academics) are making similar points.


March 19, 2015

ONS – New Director General for Data Capability – Heather Savory

We are really pleased to learn that Heather Savory, currently Chair of the Open Data User Group (ODUG), has been appointed as Director General for Data Capability at the ONS, and will take up post in early April 2015.

As many of you will know, Heather has been a great ally of DUG, and we anticipate that she will have users’ needs as her top priority when joining ONS. Here's the press release:



February 26, 2015

Familiar regional growth differences re-emerge – a report from Capital Economic

For those of you interested in how the economy is doing in different parts of the country, here is an assessment from Capital Economics:

“While the UK’s economic recovery appears to have regained momentum in early 2015, not all regions have benefited equally. The South, particularly London, is once again growing at a much faster rate than the northern and devolved regions. Indeed, January’s PMIs suggest that the Scottish and Northern Irish economies may have lapsed back into stagnation. Admittedly, a spell of unusually bad weather in Scotland reportedly set its economy back temporarily. Even so, the Scottish economy is also likely to be struggling with the sharp fall in oil prices, which is already taking its toll on investment and employment in the North Sea. And looking ahead, we expect the devolved regions of the UK to continue to underperform as their dependence on public spending and employment holds them back.”

If you'd like to find out more about Capital Economics, their website is: www.capitaleconomics.com


February 13, 2015

Retail Research and Data Master's Dissertations – 15 projects now advertised

Following up the news on October 20 (below), we are very pleased to report that commercial companies have already come up with 15 proposals for Master’s projects this year.

These include one by the DUG members collectively: “Data Sharing between commercial companies, and with government: what are the current and potential benefits?”

You can see all the project proposals on the Consumer Data Research Centre website: http://cdrc.ac.uk/2014/10/retail-research-masters-dissertations-the-call-for-2015-projects/ 


February 12, 2015

Some tips from Professor Martin Callingham on free mapping (buildings & built-up areas) & postcode files

Talking to Martin earlier this week, he was enthusing about some of the above data (which is available free). Here’s his message, with an accompanying  map of Newport:

1. Building outlines – the outlines of buildings for the whole of the UK

You can get to the building outline file on  http://geogale.github.io/2011OAC/ where there are a series of folders, one named '2011 OAC buildings zip'. The outlines are shown in black on the map.

2. Built up areas file

This is the file giving built-up areas for England and Wales, which are shown in grey on the map. Get this on https://geoportal.statistics.gov.uk/geoportal/ and search for “built-up areas”. It also provides a useful gazetteer.

3. The National Statistic Postcode Lookup

The NSPL file contains every postcode in the UK with grid references to 1 metre updated quarterly and a lot of more fields of higher geography and also the new OAC. It has three postcode fields – the 8 and 7 bit ones and the 'variable length' one (i.e. the one with a single space before the last three characters). This last one has a problem in SPSS (and probably all software) as the first 2600 postcodes are 7 characters long, so the software assumes they all are, and truncates the ones that happen to be 8 characters long. As long as you know about this it is possible tell the software not to be silly. Use https://geoportal.statistics.gov.uk/geoportal/  and search for nspl. The map uses the building outline and built-up areas file. The rest comes from Meridian which is available on the open data site – Ordnance Survey section http://data.gov.uk/publisher/ordnance-survey

 All the files are free. The cartographic files are in shape format


February 2, 2015

Cabinet Office Minister (& Open Data evangelist) Francis Maude to stand down as MP

We’ve just spotted the news that the Cabinet Office Minister Francis Maude is standing down at the General Election: http://www.bbc.co.uk/news/uk-england-sussex-31082819

Setting party politics aside, there can be no doubt that Francis Maude has been a major figure for DUG and our goals over the last five years, notching up 11 mentions on our website.

Having been sceptical about the 2011 Census, he found funding for the ONS’s major assessment “Beyond 2011” for the next Census. Even more importantly, he has relentlessly promoted Open Data, winning the DUG Award in 2011 for “His enthusiastic promotion of the policies of Open Data, and investigating alternatives to another traditional Census”. You can see our citation, plus photos here: http://www.demographicsusergroup.co.uk/resources/2011conf-DUGAward2011-FrancisMaude.pdf

We will be very fortunate if we get a new minister with similar commitment after the election.


February 1, 2015

A summary of DUG’s many activities last year

If you’d like to remind yourself of what DUG members did in 2014, or want to tell a new face what the group is about, you can download the 4-page DUG Highlights here.


January 29, 2015

The Open Data User Group's new report on "The National Information Infrastructure" – to make more government data freely available

We are pleased to report that the Open Data User Group (ODUG) is adding to the pressure for more public data to be made freely available.

Its new report “The National Information Infrastructure (NII) – Why, What and How” states that:

“The UK has been muddling along for years without coherent data describing the fabric of our society' and calls on the government to 'make the NII an immediate priority and to provide a clear path forward for public sector data holders, many of whom are already moving to release more of their data as Open Data, so that their current actions will become part of a coherent bigger-picture where the parts fit together properly for the future……Both the public and the private sector should have access to the NII, and also citizens so that they can hold the government to account.”

Here’s the link: http://data.gov.uk/sites/default/files/library/odugUK_NII_final%20%281%29.pdf

The Annexes include examples of Housing Land (including OS mapping), and Health Data. It’s good to see further pressure for more data to be made open.


January 25, 2015

Ordnance Survey to become a "Government Company"

Matthew Hancock, Minister of State for Business and Enterprise at the Department for Business, Innovation and Skills, has issued a statement announcing a change in Ordnance Survey’s operation model.

You’ll see that OS is to change from a Trading Fund to a Government Company. The statement begins:

“The change is operational in nature, and is aimed at improving Ordnance Survey’s day-to-day efficiency and performance. It will provide the organisation with a more appropriate platform from which to operate, and one which provides greater individual and collective responsibility for performance. Ordnance Survey will remain under 100% public ownership with the data remaining Crown property, with ultimate accountability for the organisation staying with the Department for Business, Innovation and Skills. Further to this change, in the coming weeks I will also be setting out more details on how Ordnance Survey will be building on its existing extensive support for the Government’s Open Data policy and on some senior appointments which will further strengthen the management team……..”.

Ordnance Survey’s business model has led to a series of problems for users generally, and commercial users in particular (see various reports in DUG News), over many years.

Whether this reform will prove to be an improvement, or worsen the situation, remains to be seen. We’ll keep you informed of further news and comments


December 17, 2014

The Open Data Institute’s new "Open Data Roadmap"

The Open Data Institute (ODI) has just launched its Open Data Roadmap for the UK, setting out steps the next UK government can take to harness the benefits of open data for improved policy making and social, environmental and economic benefit.
It contains nine recommendations for the UK to continue to build on its open data successes including:

• an ‘all of government’ data strategy that brings together all aspects of data policy in one place in the Cabinet Office
• the creation of a new Chief Data Officer to develop, advocate and progress all aspects of this strategy
• building data publication into everything the Government Digital Service does,
• empowering trading funds to release more open data, including Companies House, Ordnance survey, Met Office and the Land Registry
• greater investment in data training for government, business and citizens
• a mandate that public procurement contracts require the release of open data, in line with the UK’s commitment under the G8 Open Data Charter to ‘open by default’

Here’s the link: http://theodi.org/roadmap-uk-2015

It’s good to see further pressure for more data to be made open.


December 13, 2014

A prototype for a future ONS website – your comments would be very welcome by 5 Jan 2015

Here’s an extract of a message from ONS, which is seeking users’ comments on a new prototype for its website.

We hope that you might find a few minutes amidst the Christmas rush – or between Christmas and New Year – to have a look.

“This website, http://alpha.ons.gov.uk/#!/  is the culmination of three months development and is based on feedback from hundreds of users of the current website. The considerable engagement we have had from users throughout this project has been invaluable and without them this prototype would never have happened.

The primary goal of this prototype is to test an approach based on the feedback we have documented over the last nine months. Before we start the next stage of development we want to engage a larger group of users to validate or challenge the decisions we have made so far.

The site is limited in its coverage of ONS content and does not aim to provide current statistical information. It should not be seen as an alternative or replacement for the ONS website at this point. However, we hope it provides enough depth of content to allow users to get a real sense of what does and does not work for them. The alpha website will be publicly available until Monday 5 January 2015. The more feedback we receive during this period the better the resulting future ONS website will be.

Feedback can be provided via the link on the prototype itself or by emailing web.comments@ons.gov.uk  . Please remember any detail, no matter how small, will be invaluable. I hope you find the prototype a step in the right direction and we look forward to any feedback you might have.”


December 6, 2014

Passenger statistics for every railway station in Britain: April 2013 to March 2014

Here are some new trainspotting stats from the Office of Rail Regulation:

“Waterloo has retained its position as the busiest station in Great Britain as top stations continue to attract growing numbers of passengers, reveal statistics by the Office of Rail Regulation (ORR).

London Victoria comes second with 81,356,330 entries and exits (an increase of 5.2%) and London Liverpool Street third with 63,004,002 (up 7.8%). The biggest climber this year in the top 10 stations was London Euston which clocked 41,911,706 entries and exits – a 9.4% jump on last year. Eight out of the top ten busiest stations in Britain were located in London, with Birmingham New Street coming in eighth (34,748,984 entries and exits, up 8.3%) and Leeds in tenth place (27,729,453 entries and exits, up 5.8%).”

If this is for you, put on your anorak and bobble hat and open the link below to scroll through estimates of passengers for every station in Britain:


December 3, 2014

Staying-in fashion: Britons spending more on rent and home entertainment – ONS's Family Spending Survey 2014

Here’s the Guardian’s take on ONS’s latest Family Spending Report 2014:

“British households are spending less on eating out, hotels, alcohol and smoking, according to the annual official survey of family budgets, which also reveals how rent, gas and electricity are taking up an ever larger proportion of spending.

Although spending rose last year compared with the previous one, much of the increase was taken up by rent payments as the property boom ate into household expenditure, the Office for National Statistics figures show.

As rising housing costs exposed the consequences of failing to get on the property ladder, the ONS also found disparities between the lowest and highest earners – with the richest 10% spending three times as much on food as the bottom 10%.”

There’s more here: http://www.theguardian.com/money/2014/dec/02/british-households-spend-less-entertainment-bills-rent-survey

And here’s a link to ONS’s report: http://www.ons.gov.uk/ons/rel/family-spending/family-spending/2014-edition/index.html


November 19, 2014

DUG Members’ needs for data from government – our Data Manifesto – latest update

DUG’s Data Manifesto, which was initiated by Mike Whitelegge at M&S, continues to make an impact in Whitehall, and is proving a very useful summary wishlist / reference document. Here is the latest version, which you might find useful if you are asked “What more government data would you like to see made freely available?”


November 18, 2014

Ordnance Survey accused of stifling competition in open data row

Here’s yet more criticism of Ordnance Survey…….

“UK mapping agency Ordnance Survey stands accused of using £800m of government contracts to stifle competition in a row over the release of geographical information as open data. Geo-data firm Getmapping claimed Ordnance Survey received illegal state aid in a complaint it lodged at the European Commission (EC) last Friday (7 November 2014).”

“Getmapping claimed Ordnance Survey used two 10-year government contracts worth £800m to stifle smaller, private geographical intelligence firms. The complaint seeks to make Ordnance Survey curtail its ambitions so private companies can prosper in its stead. Getmapping said the Department for Communities and Local Government (DCLG) let the contracts wrongfully in 2010 and 2011, because it didn't put them to public tender.”

"Ordnance Survey has been granted these contracts, which has made it highly profitable – much more profitable than most industry companies. It is using its state aid to compete with the rest of the partner network. Ordnance Survey is a serial abuser of its partners. And the government knows it."

You can read more here: http://www.computerweekly.com/news/2240234712/Ordnance-Survey-accused-of-stifling-competition-in-open-data-row  

November 4, 2014

UK Statistics Authority publishes a new Statement of Strategy – including the importance of business users

Following the talk by the National Statistician, John Pullinger, at DUG’s Conference http://www.demographicsusergroup.co.uk/24.html  , here is the UK Statistics Authority’s new Statement of Strategy, entitled “Better Statistics, Better Decisions”.

It’s good to see the emphasis on the importance of businesses as users of stats, e.g. “Decision-makers across the spectrum – individuals, businesses and government”.

There are also several mentions of the UKSA seeking to be “helpful” to its customers – which is good to see, setting the expectation. 

October 25, 2014

The DUG Conference “Can sharing data within and between organisations create social and economic benefits?” – and our annual Award

At our annual conference on 8 October, the winner of The Demographics User Group Award 2014 for “Better information from government” was…………

The Office for National Statistics

for its active engagement with private sector users when carrying out the most successful Census in recent decades

You’ll find the citation, ONS’s Press Release, and a photo, together with the conference presentations, at the tab on the left.

October 24, 2014

Supermarket locations, free from Geolytix

Our friends at Geolytix have produced another open (free) dataset…..a list of supermarket locations.

“There is no commercially sensitive information attached to the stores, just the store name, and an address from the retailers store locator, our GeoLytix town and locality names, and an exact rooftop location (in British National Grid and web friendly WGS84 lat/long). Yes, we have probably spent more time directing ourselves around the country in google street view than the google van spent capturing the imagery (and yes, we have manually located each of the 6,872 stores with the aim to get a really good dataset).

Current retailers include Aldi, Asda, Booths, Budgens, Marks and Spencer, Morrisons, Sainsbury’s, Tesco, Waitrose, Wholefoods. Lidl and Co-op are coming soon.”

Here’s the link: http://geolytix.co.uk/blog/category/open-data/geodata/supermarket-locations/


October 20, 2014

Retail Research and Data Master's Dissertations – call for projects now, to commence in 2015

Many of you will be aware of the opportunity for companies to propose projects for Master’s students that we have supported in the last three years.

New funding from the ESRC is making it possible to run this is the longer term, and to start earlier (ideally this month & next), to maximise the chance of getting the pick of the best students for 2015.

Here’s more detail from Guy Lansley at UCL, who is running the scheme this year.


Please start thinking about possible projects now, and don’t hesitate to get in contact with Guy to sound out your ideas.


October 14, 2014

Book Launch: London: The Information Capital – Friday, 31 October 2014 from 18:30 to 20:30

If you are interested in analysing, understanding, and mapping London, this will be right up your street.
Here below is a message from Dr James Cheshire of UCL about his book, and its launch – a great opportunity to meet up with young researchers who are exploring and visualising free open data for London.

“A lot of the discussion points – data hacking, visualisation, open data, user privacy, new datasources – are topics covered in my new book entitled London: The Information Capital (http://theinformationcapital.com/  ). The general theme throughout the book is that London is world leading in its data availability since it benefits from the likes of the 2011 Census as well as TfL data and large numbers of social media users. We visualise all these things alongside some previously unseen data such as Londoner's dating preferences (from Lovestruck) and what the police helicopters get up to on any given day (thanks to surprisingly successful FOI request).

It will be published on the 30th October and available in all good bookshops (as they say). Its RRP is £25 but the Guardian and Amazon currently have it heavily discounted at the moment (£15 and £13.60 respectively).

I am also hosting a launch event at UCL on the 31st October to which all are welcome. Sign up here: http://bit.ly/1w0bdih  
We hope the book will add to the growing momentum behind open data and to help London lead by example.



October 14, 2014

Census Research User Conference 2014 – 31 October 2014, Royal Statistical Society, London

The UK Data Service is holding a one-day conference to provide users and data creators with an opportunity to meet and discuss developments in census research.

The day will be composed of presentations from census agencies on developments in data and from users who have used the data in their research. The presentations will be based on analysis of the 2011 and earlier census products including aggregate tables, flow data and microdata

The conference will be followed by the Enhancing and Enriching Historic Census Microdata (EEHCM) event to launch new census products.

This is pitched at academic users, but might provide an opportunity to develop new ideas and make contact with census specialists.

You’ll find more details, and can book here:


September 27, 2014

ONS’s new web page – "Get involved with Census Analysis"

ONS has announced a new Census webpage, which includes details of planned and previous user events, a link to the StatsUserNet census discussion forum [good stuff for Census anoraks], and news of tools and what others have done with 2011 Census data. Here’s the link:



August 22, 2014

DUG's 11th Annual Conference: “Can sharing data within and between organisations create social and economic benefits?” – Wednesday 8 October 2014, at the Royal Society

This year our topic is “Can sharing data within and between organisations create social and economic benefits?”, and we will again have more than 100 delegates, enabled by financial support from the Economic and Social Research Council.

The topic has been prompted by discussion amongst DUG members, and meetings with the Cabinet Office, ONS (planning Beyond 2011), and the ESRC (with its big investments). This seems an ideal time to debate an increasingly hot issue. You’ll see on the provisional programme that our speakers include the National Statistician John Pullinger, and a there’s workshop to discuss public acceptability, future opportunities, and challenges and how to overcome them.

As always, the purpose of the conference is to bring together people from DUG's 16 member companies, and also specially invited guests in government and universities, to spread knowledge and stimulate new ideas. This is DUG’s eleventh annual conference, and we are convinced that increased dialogue brings enormous benefits to all parties.
As in previous years, the proceedings will be posted on this website.


July 25, 2014

"Big Data and Location": an AGI event in partnership with DUG – 30 September, IBM South Bank, London

The Association for Geographic Information has been in touch, seeking DUG’s involvement in its conference on “Big Data and Location”, and we are very pleased to partner in this event, which we think will be of real interest to many DUG members involved in geographical analysis. It will feature Mike Whitelegge of M&S as both a speaker and chair, and the programme also includes speakers from IBM, Deloitte, Cloudera, MapR, CapGemini, Esri and Ordnance Survey together with exhibitor stands from a range of suppliers.

Here’s the programme: http://www.geobig5.com/files/6114/0569/9029/ProgrammeLondonv1.pdf

You can book at http://www.geobig5.com/events/big-data/

July 28, 2014

Francis Maude confirms plans for the next Census in 2021 – and DUG is quoted in the Financial Times

The Government’s response to the Public Administration Select Committee’s report (see DUG News April 21) has just been published, and includes a letter from Francis Maude, confirming that the government will back the National Statistician’s recommendation that there should be a census in 2021, conducted at least partly online.

Mr Maude reiterated his view that the census is “outdated” and could be delivered more effectively and more cheaply, but said he accepted the 2021 Census should go ahead, alongside use of existing data and surveys. But he added: “It is the government’s ambition that beyond 2021 the decennial census would not be undertaken, instead more regular and timely administrative data would be used to produce statistics.”

You can see the report here: http://www.publications.parliament.uk/pa/cm201415/cmselect/cmpubadm/601/601.pdf

There’s a good article in the Financial Times (behind its paywall) by Emily Cadman, which includes:

“Keith Dugmore, who organises the Demographic Users Group (DUG), a group of some of the UK’s biggest employers in consumer-facing businesses, including Barclays, Boots, John Lewis and Sainsbury’s, said the conclusion was the right one.

The group had expressed grave concern about the option of moving immediately to solely administrative data, warning that any loss of data of very small geographic areas would be “catastrophic” for business users.

Mr Dugmore said that while “we don’t want to be lumbered with a traditional census forever” there was a “sense of relief” that the traditional census will be carried out again while research is done to see whether administrative sources could provide a true alternative.”

July 25, 2014

"Harnessing Open Data for Business Advantage" – MRS Census Geodemographics Group seminar – 5th November

This seminar will focus on how users have been applying census and other open data, with an emphasis on ‘how to’ case studies and useful lessons learnt.

You’ll see that there is a fine array of speakers, and that there is a discount if you book before 6 October: https://www.mrs.org.uk/event/course/2436

July 22, 2014

New, free, geodemographics – ONS releases the 2011 Output Area Classification (OAC) for the UK

ONS has announced the release of the new version of OAC:

We also have:
• Interactive maps, produced by UCL: http://public.cdrc.ac.uk/

• A new website for the Output Area Classification User Group (OACUG): https://plus.google.com/u/0/communities/111157299976084744069  

July 22, 2014

Companies House to make all of its digital data available free of charge

Whilst not many DUG members use Companies House data about businesses, here’s some encouraging news about further moves to make government data open and free.

“Companies House is to make all of its digital data available free of charge. This will make the UK the first country to establish a truly open register of business information.
As a result, it will be easier for businesses and members of the public to research and scrutinise the activities and ownership of companies and connected individuals. Last year (2013/14), customers searching the Companies House website spent £8.7 million accessing company information on the register.
This is a considerable step forward in improving corporate transparency; a key strand of the G8 declaration at the Lough Erne summit in 2013. It will also open up opportunities for entrepreneurs to come up with innovative ways of using the information……….”.

You can read more at: http://www.companieshouse.gov.uk/freeDataPressNotice/

July 16, 2014

New Geodemographic Classification Systems – a review by Peter Sleight

Peter Sleight, of the Target Marketing Consultancy, continues to provide very useful summaries of the competing geodemographic classifications – Mosaic, Acorn, OAC, and several others.

Here’s his latest:

July 14, 2014

UKSA Independent Assessment of the 2011 Census – DUG's Response

DUG has submitted this response to the UK Statistics Authority.


July 12, 2014

Land Registry privatisation vetoed by Vince Cable

There’s more news in today’s Guardian about Whitehall infighting over the government agencies and their data:

“Plans to privatise the Land Registry have been abandoned by the government after business secretary Vince Cable vetoed the scheme following a series of rows between Tory and Liberal Democrat ministers.
It had been hoped that a sell-off would raise at least £1.2 bn for the Treasury, with ministers deciding on the appropriate structure, such as creating a deal with a joint venture company.
But this week, ministers will tell parliament that the planned move has been suspended indefinitely. The Conservatives had been in favour of a partial sell-off via the creation of a joint venture.”

Read all about it here:

July 11, 2014

“How UK businesses are securing and managing talent to grow in a big data world” – a new report

Here’s news of a report on a hot topic for many DUG members. It’s authored by Nesta (an innovation charity http://www.nesta.org.uk ) in association with the Royal Statistical Society, which invited DUG to be represented on the panel at a launch on Wednesday 9 July:

June 28, 2014

New website "Datashine" – detailed and fast mapping of Census statistics across England & Wales

Following on from the news of “Maps of 2011 Workplace Zones in London” (25 June), here’s another initiative from UCL, this time providing online mapping of a wide range of Census variables at detailed level across the whole of England & Wales.

Here’s the link: http://blog.datashine.org.uk/

Warning: once you start playing, you might find it difficult to stop….

June 25, 2014

Maps of 2011 Workplace Zones in London – See the local differences in hours worked. education, transport, health, etc.

A PhD research student, Robin Edwards, at UCL’s Centre for Advanced Spatial Analysis (CASA) has been very quick – days after the release of the statistics – to publish a series of maps of c.10 variables by 2011 workplace zones in London:


It’s fascinating stuff – and may well lead to you further digging into the data.

June 17, 2014

Output Area Classification (OAC) – Its current use on government surveys, and the launch of OAC 2011

Many of you will be familiar with ONS’s (free) geodemographic Output Area Classification (OAC). If not, you can find out about it at http://areaclassification.org.uk/

We have learned from ONS that the following government surveys are now coded with OAC:

1. Annual Population Survey (APS) (which is based on waves from the Labour Force Survey)
2. Crime Survey for England and Wales (CSEW)
3. Dental Health Survey of Children and Young People (CDH)
4. Household Assets Survey (HAS) – formerly the Wealth and Assets Survey (WAS)
5. Integrated Household Survey (IHS)
6. Living Costs and Food Survey (LCF)

The new 2011-based version of OAC is expected to be launched next month.

June 13, 2014

New file of Census data at Output Area level for the whole UK

We are pleased to report that ONS has now produced a single file of statistics at Output Area level, spanning all four countries of the UK. (Experience in 2001 showed that it is so much better for the Census Offices to do this at source, rather than multiple users all try to sort out the sometimes varying definitions themselves). Here’s the announcement from ONS:

Today, 13 June 2014, ONS has made available the Key and Quick Statistics tables at Output Area (OA) level, for all Output Areas in the UK. This combines information previously published separately by the census office responsible for the conduct of the census in each constituent country of the UK.

The tables are equivalent to the local authority level UK data released in three parts in October / December 2012 and January 2014. Between them, those three releases provided all Key and Quick Statistics that were harmonisable across England and Wales, Scotland, and Northern Ireland.

The tables – 26 Key Statistics and 47 Quick Statistics – are provided now for all Output Areas in the UK, in a simple csv form suitable for additional processing by users. The data is accompanied by supporting information that explains the steps taken to merge categories in some tables to produce a UK harmonised table, and further detailed explanations and definitions of terms used in tables. To aid users in re-aggregating the OA data administrative and other common geographies, a look-up file that relates all OAs to every standard higher level geography in each country is also included.

All of the data is available to download in a single zip file from:



June 7, 2014

2011 Census – New (free) postcode directories & lookups released by ONS

The May 2014 editions of ONS’s Postcode Directory (ONSPD) and National Statistics Postcode Lookup (NSPL) have been released on the ONS Geography portal:



May 23, 2014

Workplace Statistics from the 2011 Census published today

Here’s news from ONS of the publication of counts and detailed classifications of workers (so vital when planning store locations) for Workplace Zones in England and Wales.

The workplace population of an area is defined as “all usual residents aged 16 and above who are in employment and whose workplace is in the area”. People who work mainly at or from home or do not have a fixed place of work are included in the area of their usual residence.

This definition focuses on only those people who are working, and the statistics are for a new “Workplace Zone” geography, which is particularly valuable for targeting very small areas within city centres. The statistics cover a wide range of topics. You can find more information and download the data at:


May 23, 2014

Government surveys on Income, Expenditure, & Wealth – User Conference – 7 July

This free full-day meeting in London, organised by the UK Data Service and ONS, will provide a forum for the exchange of information and views between users and producers of the following surveys:

• Family Resources Survey
• Living Costs and Food Survey
• Wealth and Assets Survey

The programme contains a mixture of papers from data producers and researchers.

To view the programme and book a place please go to http://ukdataservice.ac.uk/news-and-events/eventsitem/?id=3749  


May 14, 2014

John Pullinger appointed as the new National Statistician

We are very pleased to report that John Pullinger is to be the new National Statistician, succeeding Jil Matheson. The UK Statistics Authority’s News Release is here.

John has always emphasised the importance of the users of statistics, being very supportive of DUG over many years, and has been a regular guest at our annual Conference.

April 21, 2014

House of Commons Select Committee Report: "Census should not be scrapped"

Following our earlier news (see March 1 below) about the Public Administration Select Committee undertaking a study on the Future of the Census, it has now reported, urging the Government not to scrap the 2021 Census, but also promoting more use of data from administrative sources:

“Good figures on the people in the country are of fundamental importance to the statistical system, policy makers and society more widely, and the ten-yearly census gives detailed information on small areas. The Report follows the National Statistician’s announcement in March 2014 that she recommends that Government keep the census in 2021, but that it should be conducted largely online, and that the Government should make much greater use of the data which it already holds in order to improve the accuracy of population estimates. The Committee supports the recommendation from the National Statistician, but urges the Office for National Statistics to do much more to make the best use of the data which the Government already collects, for example through the Department for Work and Pensions, HM Revenue and Customs and the Department of Health. The Committee says that the Office for National Statistics’ work on the future of the census has, to date, been limited, and recommends that the Office for National Statistics now sets out a much more ambitious vision for the use of this data to provide rich and valuable population statistics.”

Bernard Jenkin MP, Chair of the Committee, said:

“The Government has previously indicated that the ten-yearly census should be axed, but we believe that it is too soon to decide whether or not to scrap the census. We think there should still be a census in 2021. The Government should make far greater use of the data it already holds in order to improve population estimates. We urge the Government to address public concerns about data sharing and to learn the lessons from the flawed rollout of the “care.data” programme, so that the best use can be made of the information already collected and held by the Government. The Government’s objective of “better, quicker information, more frequently and cheaper” depends upon it.”

You can see the report at: http://www.publications.parliament.uk/pa/cm201314/cmselect/cmpubadm/1090/109002.htm

It’s good to see in the Summary: “Witnesses emphasised, among other things, the great financial benefits to business provided by census data and these have been quantified.” 


April 15, 2014

Big Data – Open/Free data available for use by businesses – Conference organised by the Research Councils & Government

“This unique day aims to remove the barriers for companies to access this open data, revealing how to extract value and insight for the benefit of your company- of all sizes and sectors. You will be equipped with expert guidance on how to take advantage of the data sets on offer, as well as introduced to enabling funding schemes and organisations that can help you to take raw data and make it into something valuable.”

This sounds right up our street.
Time: Tuesday, 06 May 2014, 09:00 - 17:00
Place: Science & Technology Facilities Council, Rutherford Appleton Laboratory, Didcot, Oxford OX11 0QX

More details, & booking: http://biginnovationcentre.com/Events/133/Big-Data-Week-UK-big-data-open-data-available-for-business

April 12, 2014

UK Data Service makes 2011 Census statistics available to all

Here’s some good news from the ESRC’s UK Data Service:

“InFuse provides both academic and non-academic researchers with access to 2001 (England and Wales) and 2011 (UK) Census aggregate statistics under the Open Government Licence. This means there's no need to register or login to extract the data.

InFuse makes it quick and easy for the user to find and download information for places of interest across the UK and it is free of charge.”

You can read all about it at: http://census.ukdataservice.ac.uk/news-and-events/newsitem/?id=3761

This should be an important step in opening up data held by the academic community to wider access. 

March 28, 2014

National Statistician’s recommendation on the future of the Census and population statistics in England and Wales

Many of you will be aware that DUG has been very closely involved with ONS in its assessment of possible options for the next Census “Beyond 2011”.

We also responded to the formal consultation late last year, and we now have news of Jil Matheson’s recommendation to government http://www.ons.gov.uk/ons/about-ons/who-ons-are/programmes-and-projects/beyond-2011/beyond-2011-report-on-autumn-2013-consultation--and-recommendations/index.html
• An online census of all households and communal establishments in England and Wales in 2021 as a modern successor to the traditional, paper-based decennial census, combined with…..
• Increased use of administrative data and surveys in order to enhance the statistics from the 2021 Census and improve annual statistics between censuses.

This is a good result – but only so far, because Treasury has to accept the cost-benefit analysis, and Government has to approve the recommendation.

DUG was quoted in the Financial Times (apologies that you might have to register to view it):
http://www.ft.com/cms/s/0/907c1262-b598-11e3-a1bd-00144feabdc0.html#axzz2xMJK3EMs _____________________________________

March 18, 2014

PASC report on Statistics and Open Data – "The sale of PAF with the Royal Mail was a mistake"

Hard on the heels of the message about BIS’s report on the National Address Gazetteer, the Public Administration Select Committee (PASC) has published its report on Statistics and Open Data, which gives more support to DUG’s cause:

“In a report on statistics and open data published today, the Commons Public Administration Select Committee (PASC) says the benefits – potentially running into billions of pounds for the economy and society as a whole – of harvesting the masses of unused knowledge and information that Government collects outweigh the risks to privacy, and that there must be a “presumption of openness” with public data.”

“Some government data sets are of huge direct value to the economy. The Committee says that the Postcode Address File (PAF) was included in the sale of Royal Mail to boost the share price at flotation: this move took an immediate but narrow view of the value of such datasets. PAF should have been retained as a public data set, as a national asset.”

(We sent our submission to PASC on 5 September, urging it to press for the National Address Gazetteer and the Postcode Address File to be made Open Data).

You can see further news, and also download the report here:
There has also been good coverage in the media, e.g.
Guardian: “MPs and open-data advocates slam postcode selloff” (with KD being quoted) http://www.theguardian.com/technology/2014/mar/17/mps-and-open-data-advocates-slam-postcode-selloff
BBC: http://www.bbc.co.uk/news/business-26605375  
Such comments will help the Cabinet Office’s cause in its continuing battle with BIS and the Treasury.

March 15, 2014

An open (free) national address gazetteer? DUG's letter to BIS

Following the news on February 22, here is DUG’s letter to BIS.
You’ll see that we are continuing to make the case that the National Address Gazetteer should be free open data (but reluctantly see the proposed Freemium model as a step towards the ultimate goal).

March 2, 2014

Passenger statistics for every railway station in Britain: April 2012 – March 2013

Here are some new trainspotting stats from the Office of Rail Regulation:

• London Waterloo remains the most heavily used station in Britain clocking nearly 96 million passenger entries and exits last year – an increase of 1.8m over 2011-12.
• The number of people using stations across Britain in 2012-13 increased by over 82m – a 3.3% rise compared to the year before.
• The list of top 10 stations has remained unchanged since 2011-12 with eight of the 10 stations located in London.
• All the top 10 stations reported a rise in usage with the largest increase at London Euston, which saw an increase of 1.8m entries and exits – due to the increasing demand for London Overground services.

If this is of interest, you might want to put on your anorak and bobble hat before you open the link below to scroll through estimates of passengers for every station in Britain:

March 1, 2014

House of Commons Select Committee on the Future of the Census – DUG representation

The Public Administration Select Committee is undertaking a study on the Future of the Census, and an oral evidence session was held at the House of Commons on Tuesday 25 February.

Keith Dugmore spoke on behalf of DUG, and the following is now on the parliamentary website:
Video: http://www.parliamentlive.tv/Main/Player.aspx?meetingId=14944

Transcript: http://www.parliament.uk/business/committees/committees-a-z/commons-select/public-administration-select-committee/inquiries/parliament-2010/statistics/future-of-the-census/

We also followed up with this submission.

February 22, 2014

An open (free) national address gazetteer? Here's an independent report for government on its feasibility

With our long campaign for a definitive and free National Address Gazetteer in mind, if you are an addressing anorak, please read on…..

The Department of Business, Innovation & Skills has now published an independent report by Katalysis:

It recommends that a basic address product should be free to all users at the point of use under the Open Government Licence while premium versions would still be sold, leaving current production and maintenance roles much as they are today. Whether the government will act on this is not known, but the report is certainly valuable in its summary of the current state of play regarding address files and their ownership, and in setting out future options.

February 11, 2014

Access to data from the Valuation Office Agency

The VOA for England and Wales collects some valuable data, including:

Prices paid on sale or transfer of properties
VOA Council Tax – addresses, characteristics and attributes of properties
VOA Residential Private Rented Sector (PRS) rents

Non domestic rating:
• Rents paid on letting of properties
• Receipts/expenditure and costs of construction
• Property addresses, floor areas, characteristics and attributes

The recently-appointed Director of Information and Analysis gave a really good summary presentation of the current situation, and future plans to make more data available, at an HMRC meeting in October, and this is now public at:


February 9, 2014

Land Registry tipped for part-privatisation – Public Consultation

Several DUG members use information from the Land Registry, which is responsible for keeping and maintaining details of the ownership of land and property in England and Wales, and is a source of data on house prices.

The above disconcerting news appeared in both the Daily Telegraph:
and also the Guardian:

The Government announcement suggests we won’t lose the supply of free data, with reassuring words such as “maximising the reuse of property data for the benefit of the economy”, but if Land Registry data is important to you, you might like to have a close look at the consultation document:

The consultation will close on 20 March 2014. 


February 6, 2014

Scotland's Census 2011 – Further news of outputs

Many of you will be aware that 2011 Census outputs for Scotland have lagged behind those for England & Wales, and Northern Ireland.

We are pleased to report that, since the news (see below, 5 July) about population estimates and household estimates by Postcodes & Output Areas, and geography products, including OA boundaries and Postcode to OA look-up files, various statistics at OA level have now been published.

These are summarised in the latest Outputs Prospectus http://www.scotlandscensus.gov.uk/en/censusresults/outputsprospectus.html  (see page 4), which also includes news of plans for further releases.

January 27, 2014

Retailers: Do you have projects now that Master's students can help you to progress this spring & summer? 

If you work for a UK retail company, this is an opportunity for you.

The ESRC’s “Retail Research & Data” initiative, which involved commercial companies proposing projects for Masters’ students, was a real success in both 2012 and 2013, and Professor Paul Longley & I are pleased to report that the ESRC has decided to support this initiative in the longer term.

We are therefore seeking proposals for new projects as soon as possible.

This presents a great opportunity to get a Master’s student to help you to make progress with:
• Major current issues, such as multi-channel marketing and customer insight, store networks, transport, social media, and many others.
• Projects that you’ve long had in mind, but not had the resource to carry out.
• Working with data – both your own customer data, and also maximising the value to be obtained from government initiatives such as www.data.gov.uk, and the arrival of more 2011 Census statistics (including Daytime and Workplace populations) for the whole UK.

If you have a project in mind, please email Keith Dugmore ASAP (ideally by this Friday 31 January) at dugmore@demographic.co.uk and I’ll send you a Project Proposal Form.

It’s essential to move quickly on this, to publicise projects early and get the choice of the best students.

Our previous experience illustrates the main features of the initiative:
• ESRC’s publicity for the 2013 scheme, including a list of the 11 projects that ran: http://www.esrc.ac.uk/collaboration/business-sector/retail/retail-masters/
• Presentations by two prize winners at the 2013 DUG Conference: http://www.demographicsusergroup.co.uk/23.html
• Presentations by 12 of the 16 students at the 2012 DUG Conference: http://www.demographicsusergroup.co.uk/22.html


January 14, 2014

Happy New Year........... & here's a summary of DUG’s many activities in 2013

Very best wishes for a successful and enjoyable 2014.

Looking back on the past year, we hope that you’ll find this summary of the activities of the Demographics User Group during 2013 both interesting and useful.

You’ll see that it’s been another busy and successful year, with members sharing their experience in numerous ways, pressing government for further progress in the policy of making more data freely available, and also deepening our links with the ESRC, which is increasing the priority given to retail research.


January 9, 2014

Land Registry open data: House price paid data (November 2013) – down to postcode level

The latest alert from Data4nr includes house price stats right down to postcode level:

“The monthly residential property price data, tracks all of the residential property sales in England and Wales that are lodged with the land for registration. The data includes: the full address of the property (Primary addressable object name (PAON)), where available, secondary addressable object name (SAON), street, postcode, locality (if available), town, district, county), the price paid for the property, the date of transfer, the property type (Detached, Semi, Terraced, Flat/Maisonette), whether the property is new build or not, whether the property is freehold or leasehold.”

Here’s the link: http://www.data4nr.net/resources/1674/


January 7, 2014

A message from Martin Callingham and Steven Bond: Keith Dugmore made an MBE

"We are delighted to announce that Keith Dugmore, creator of the Demographics User Group, was made an MBE in the New Year’s Honours List. The citation was:

MBE – Keith Dugmore, Founding Chair Statistics Users Forum – “For services to UK Business”

This is a great personal achievement for Keith who has for many years successfully brokered and nurtured dialogue between various agencies of government, academia and the private sector. The benefits of the greater closeness of business to both government and academia are already apparent and will directly impact on the UK's economy and competitiveness into the future.

Many of you receiving this note will have been directly involved in this process and will know that only Keith could have made this happen. We are sure you will join with us in congratulating him on this richly deserved honour.

Martin & Steve"

(DUG’s Non-Executive Directors)


January 7, 2014

Full UK Census Data Pack from GeoLytix

Following up their earlier news below (18 February last year) it’s good to see that Geolytix are making a UK-wide (NB, yes, including Scotland) Census Data Pack available free of charge:

“Today, the full 200 variable GeoLytix UK Census Open Data pack is available for immediate free download. http://www.geolytix.co.uk/geodata/census

This release includes the Scottish data for all variables. We have also made three additions to the dataset since the last release:
• Added the Household Social Grade Variables (AB/C1/C2/DE)
• Included our own GeoLytix Urban/Rural flag for each OA
• Split the White ethnic group into ‘White British’ and ’White Other’ "

There’s also a brief blog on this release: http://geolytix.co.uk/blog/ 


December 22, 2013

HMRC consultation on sharing and publishing data for public benefit – the Government response

Following our submission (see 30 September below), HMRC has now issued its response:


This is positive towards making HMRC data more easily accessible, and will provide good context for when discussing priorities for the release of more datasets.


November 30, 2013

Retail Research Data – A website to help retailers find free data easily – new release

“The primary purpose of Retail Research Data (RRD) website is to enable insight and store location analysts working in retail organisations to get easy access to free datasets which may be of value to their businesses.”

This new release includes more 2011 Census statistics, and improved mapping. It is part of the ESRC’s Retail Research and Data project.

Here’s the link: http://www.retailresearchdata.com/

November 25, 2013

New report from BIS: "Seizing the Data Opportunity"

Here’s news of a new report from the Department for Business, Innovation and Skills: "Seizing the Data Opportunity".
Universities and Science Minister David Willetts has announced “A new strategy will ensure the UK is well placed to take advantage of the opportunities for using data. Seizing the Data Opportunity sets out plans to create a skilled workforce in data analytics, develop the UK’s data infrastructure that will allow access to high-end computing, and help data to be shared and linked to support ground-breaking research”

The report focuses on some of DUG’s goals, especially to improve the data infrastructure, and to promote training.

You can find the press release, and a link to the report here: http://news.bis.gov.uk/Press-Releases/Strategy-positions-UK-as-world-leader-on-data-science-69543.aspx 

November 22, 2013

ONS Consultation: "The census and future provision of population statistics in England and Wales" – DUG response

Following the earlier news below (September 24), DUG’s response has now been submitted to ONS.


October 31, 2013

2011 Census – ONS's "Workday Population" statistics for Output Areas in England and Wales

ONS has just sent out an alert announcing the above good news.

We’ve been pushing for this as an alternative and supplement to Workplace statistics (which will be for a special Workplace Zone geography, rather than for our regular Output Areas).

Here’s the link: http://www.ons.gov.uk/ons/rel/census/2011-census/workday-population-statistics-for-output-areas-in-england-and-wales--part-1-/index.html

12 tables provide information on some of the key characteristics of the workday population of England and Wales such as population size, general health, religion and ethnic group at Output Area level.
Similar tables for Workplace Zones are expected in Q1 next year. 

October 24, 2013

The DUG Conference “Understanding millions of mobile consumers”

Our Awards: (1) "ESRC wins top accolade from UK businesses"; and (2) Professor Martin Callingham

At our annual conference on 9 October, DUG made two awards:

(1) The Economic and Social Research Council, “for its increasing focus on retail research, and developing training in quantitative skills”
The ESRC produced a fine Press Release. This year we also made a DUG Personal Award to……….

(2) Professor Martin Callingham, formerly of Whitbread, and now Birkbeck College London, “in recognition of his role in encouraging the formation of DUG in 1998, and his immense contribution since, including training, playing with data, and providing an endless stream of new ideas”.

Both awards are hugely deserved.

You can see the Conference presentations at the tab on the left. 


September 30, 2013

HMRC consultation on sharing and publishing data for public benefit – DUG's response

Following the announcement of HMRC’s consultation (see July 29, below), you’ll find DUG’s response here.


September 27, 2013

ONS's Census News – including Postcode products, and Origin / Destination stats

ONS's latest News Alert includes:

• Migration stats at Output Area level, available from the Nomis website: https://www.nomisweb.co.uk

• Updated Postcode to OA look-up table:

• Origin / Destination (including commuting) statistics – are due in Q1 next year. 


September 25, 2013

ESRC Retail Research & Data – A website to help retailers find free data easily – Test Release – Your comments are very welcome

“The primary purpose of Retail Research Data (RRD) website is to enable insight and store location analysts working in retail organisations to get easy access to free datasets which may be of value to their businesses.”

Here’s the link: http://www.retailresearchdata.com/

This is part of the ESRC’s Retail Research and Data project. The first release was made available earlier this year. This new test release now includes information from the 2011 Census, the first release of the new Output Area Classification (OAC), and improved mapping.

It would be really helpful to get your feedback to dugmore@demographic.co.uk before we publicise the site more widely. 


September 24, 2013

ONS Consultation: "The census and future provision of population statistics in England and Wales"

As part of its investigation of alternatives to another traditional Census "Beyond 2011", ONS is now consulting on what it sees as two possible approaches:

• a census once a decade - similar to the 2011 Census but primarily online; or
• a census using existing administrative data and compulsory annual surveys.

Both approaches would provide annual statistics about the size of the population, nationally and for local authorities. A census using existing data and annual surveys would provide statistics about population characteristics every year. An online census would provide more detailed statistics but only once a decade.

The consultation runs until 13th December 2013, and ONS has arranged a number of events.
You can find the consultation document and a link to the online questionnaire here:


September 5, 2013

Public Administration Select Committee (PASC): Study into Open Data – DUG's submission

Following the earlier message below (July 27), the key themes of DUG’s submission are to:

• Recognise the importance of government open data to business
• Welcome the acceleration in progress in recent years
• Alert PASC to the fact that users outside the public sector (such as businesses and charities) are not able to enjoy all the free access arrangements which have been made for public sector users
• Urge PASC to press for the National Address Gazetteer and the Postcode Address File to be made open data


August 31, 2013

Job vacancies

Several new job vacancies in DUG member companies can be seen at the tab to the left.


August 25, 2013

The Guardian Data Blog, and Data Journalism

Lee Madden, former member of DUG whilst at Barclays, and now at Ernst & Young, is a great fan of the Guardian Data Blog, and has posted news on the DUG Network, on LinkedIn (it appears that you’ll need to sign in, rather than providing a direct link here).

“The Guardian’s data blog is a great source of both commentary on data and datasets. They also promote courses on data visualisation and have an active interest in data journalism. I'd recommend checking it out – I particularly enjoy it when they provide data commentary on stories that are in the public eye, e.g. mapping of the London riots or analysis on the A-Level and GCSE results.”

This also leads to “Data journalism at the Guardian: what is it and how do we do it?” Illuminating stuff. 


August 20, 2013

Tracking a decade of changing Britain using the Census and other Open Data – MRS Census Geodemographics Group seminar – Wednesday 6th November, 2013

Here’s news of a seminar “Tracking a decade of changing Britain: How businesses are benefiting from early use of Census and other Open Data”

You’ll see that there is a fine array of speakers, including VARs, and that there is a discount if you book before 6 October. https://www.mrs.org.uk/event/course/1946   


August 11, 2013

DUG's10th Annual Conference: “Understanding millions of mobile consumers” – Wednesday 9 October 2013, The Royal Society

This year’s DUG Conference at the Royal Society is on the topic “Understanding millions of mobile consumers”, and we will again have >100 delegates, enabled by financial support from the Economic and Social Research Council.

The topic has been prompted by discussion amongst DUG members seeking new insights into the drivers of consumer behaviour in our "always connected" world", and the need for better information about footfall, workplace populations, new shopping behaviour enabled by the growth in the use of smart phones, and the necessity to analyse massive volumes of data. The provisional programme is here.

In the morning session our expert speakers will explore these issues. The afternoon focuses on the ESRC’s support for retail research, including a showcase for some of the research projects specified by retailers for Masters’ students: http://www.esrc.ac.uk/collaboration/business-sector/retail/retail-masters/  and also has two Workshops:
• Research Issues: “What’s rattling your cage?” to identify the top priorities
• Skills Training for Insight analysts, which reflects members’ growing concern about difficulties of recruiting numerate staff, and the need for training

As always, the purpose of the conference is to bring together people from DUG's member companies) and also specially invited guests in government and universities, to spread knowledge and stimulate new ideas. This is DUG’s tenth annual conference, and we are convinced that increased dialogue brings enormous benefits to all parties.

As in previous years, the proceedings will be posted on this website. 


August 1, 2013

2011 Census data at Output Area level in E&W published – ethnicity, identity, language and religion

The ONS has now announced the publication of 19 tables for Output Areas in England and Wales on ethnicity, identity, language and religion – some of the variables that are of most interest to many DUG members.

You can download them from NOMIS: https://www.nomisweb.co.uk  


August 1, 2013

The Open Data User Group (ODUG) is seeking new members – might you consider applying? (Closing date: 23 August 2013)

Many of you will be aware of ODUG, an independent advisory group to government, which highlights public sector data with potential to bring social and economic benefit, and works with the Government to promote its release as open data. This is very much in support of DUG’s own long-established objectives.

ODUG is now seeking new members:


July 29, 2013

HMRC consultation on sharing and publishing data for public benefit (Closing date: 24 September 2013)

HMRC has launched this consultation https://www.gov.uk/government/consultations/sharing-and-publishing-data-for-public-benefit

“This consultation invites views on proposals for HMRC to increase the scope for the department to share non-identifying information (that is, information that does not relate to identifiable individuals or legal entities), and on proposed safeguards. Options cover general and aggregate data as well as anonymised data sets.

HMRC also seeks views on the potential benefits, costs / risks and necessary safeguards for proposals to share VAT registration data, either publicly or under controlled conditions for specified purposes, for example, credit rating. Views are welcome on the principles underlying these proposals and on the suggested approaches to implementation.”

DUG has long had its eyes on HMRC data (such as Incomes), and data held by the Valuation Office Agency (which reports to HMRC) such as Council Tax, and will respond to the consultation. 

July 27, 2013

Public Administration Select Committee (PASC): Study into Open Data (Responses due by 3 September)

PASC’s latest enquiry is focusing on the role of Open Data, what the government’s aims should be, what other data could be made available and how, as well as reviewing the data currently available and finding examples of best practice. It also asks which datasets are the most important and who uses them. Here’s the link:



July 22, 2013

Council Tax statistics from the Valuation Office - new classifications

Several DUG members already make use of statistics on Dwelling Stock by Council Tax Band, which are published at Output Area level for England and Wales: 


There are now some extra stats which go only down to Lower Super Output Area level, but which give more detail about dwellings. “These statistics were calculated from domestic property data extracted from the Valuation Office Agency’s administrative database on 1 August 2012. They provide a series of tables on the number of dwellings by Council Tax Band, property type (e.g. house, flat, bungalow, etc) and number of bedrooms and by various recognised statistical geographies, spanning England and Wales”. They also include “build period”:



July 5, 2013

Scotland's Census 2011 – Further news of outputs – Some OA-level data in August

http://www.scotlandscensus.gov.uk/en/news/articles/outputsprospectus.html the latest timetable for Census outputs for Scotland, includes, on 15 August: 

• Population estimates and household estimates by Postcodes, Output Areas and Datazones
• Geography products, including Area (OA) boundaries (shapefiles), Postcode to OA and OA to other geographies look-up files

So we’ll then have some stats for all four countries of the UK, but you’ll also see that Scotland’s Key and Quick Statistics (for Output Areas) are scheduled “from [NB] late September 2013”.

June 28, 2013

2011 Census – ONS, England and Wales – Postcode Headcounts, Key Stats for Built-Up Areas – published

These were published today, & here’s the link: http://www.ons.gov.uk/ons/guide-method/census/2011/census-data/index.html


June 18, 2013

ONS Census Analysis – Free Workshops – Manchester, 18th July, & London, 24th July 2013

For those of you already up to your necks in, or limbering up for, analysis 2011 Census, here’s the chance of some therapy being provided free by ONS: http://www.ons.gov.uk/ons/guide-method/census/analysis/summer-workshops/index.html

“Census Analysis workshops will take place in Manchester and London in July. This is your opportunity find the latest information on the 2011 Census outputs progress, what's coming next and to discuss the census analysis articles published so far date and the planned work for the future. The link provides all the information as well as an email address to secure your place: Census.analysis.inbox@ons.gsi.gov.uk

June 15, 2013

Shakespeare Report – Reviewing Open Data – The Government's Response

Following the publication of Stephan Shakespeare’s report on Open Data last month (see below), the Government has just published its response:

It is positive, and includes:

“Building on the recommendations of the Shakespeare Review, the government will include details of a new national Information Infrastructure in our revised Open Government Partnership National Action Plan due to published in October.

Working with data users, we will identify the data we hold that should be part of that new National Information Infrastructure. This data will be available for release to support economic growth and the developers and entrepreneurs who use open data to build information-led businesses – helping us compete in the global race.”

This could be another step towards the National Address Gazetteer being made freely available, but Royal Mail won’t easily give up its rights to sell PAF:

“The Postcode Address File (PAF) is a fully integrated part of Royal Mail’s business and will continue to remain so. In recognition of the continued importance of this dataset to innovation and growth, the Government has been working closely with Royal Mail to improve the licensing regime and drive greater take-up of the data. The Government is now announcing that, from the beginning of July, Royal Mail will provide the PAF for free to independent* small charitable organisations and free to independent* micro-businesses for one year. Royal Mail will also be increasing the number of free online address look-ups on its website from 15 to 50 per day. In addition, Royal Mail will consult in July on a radical simplification of the licensing regime for all customers based on a permissive licence with minimal restrictions.”
* Independent – not associated or affiliated with any existing Solutions Provider

June 7, 2013

ONS's new Open Geography Portal – for downloading geographic data such as postcode directories, boundaries, names and codes for ONS's area statistics

Here’s news of ONS’s Open Geography Portal “which allows users to discover, view and download geographical reference data to support National Statistics”
The link https://geoportal.statistics.gov.uk/geoportal/catalog/main/home.page  looks unexciting, but by roaming around (e.g. Browse) you can find some useful stuff.

June 5, 2013

Public Administration Select Committee – Communicating and publishing statistics – Report now published

Recalling DUG’s submission on 13 October last year, the House of Commons PASC report on Communicating Statistics has now been published.

The recommendations are not revolutionary, but add further pressure in the right direction, including:

• The UK Statistics Authority should work proactively to bring together and clearly present key statistics, from various sources, around common themes or events, such as elections and referendums, as well as broader topics such as the labour market and economic trends.
• The ONS website must be improved. Government statisticians should work much more closely with different kinds of users in order to present statistics in ways which meet their different needs.
• The Committee says the Statistics Authority should find more creative ways of communicating statistics, for example, through interactive guides. This should be in addition to the publication of more raw data in machine-readable format for experts who want the full results, not just the edited highlights presented in releases for a mass audience.
• In addition to the many routinely-produced statistics, government statisticians produce thousands of pieces of data on demand, known as “ad hoc statistics”. This is positive, but more of this kind of data should be published proactively, rather than simply in reaction to requests.

You can find more here: http://www.parliament.uk/business/committees/committees-a-z/commons-select/public-administration-select-committee/inquiries/parliament-2010/statistics/

May 19, 2013

2011 Census results for England and Wales – the latest news from ONS

ONS has just published the latest edition of its Census Newsletter. It includes information on data releases, the commissioned tables service, and an update on plans to improve access to data via the ONS website. Here's the link:

May 16, 2013

Shakespeare Report published – Reviewing Open Data to explore growth opportunities – further positive news

Following the announcement in October (see below), the Shakespeare Review of Public Sector Information was launched at a packed meeting yesterday.

Its recommendations include:
• The government should produce and take forward a clear, predictable, accountable ‘National Data Strategy’ which encompasses PSI in its entirety.
• A National Data Strategy for publishing PSI should include a twin-track policy for data release, which recognises that the perfect should not be the enemy of the good: a simultaneous 'publish early even if imperfect’ imperative AND a commitment to a 'high quality core'. This twin-track policy will maximise the benefit within practical constraints. It will reduce the excuses for poor or slow delivery; it says 'get it all out and then improve'.
• “One would be hard-pressed to find any expert who, asked to create new structures for core reference data from scratch, would advocate the current Trading Fund model”. The overarching aim of the Trading Funds [such as Ordnance survey] should be to deliver maximum economic value from public data assets they provide and support, by working to open up the markets their data serves. …..This means they should work towards opening up all raw data components, under the Open Government Licence (OGL) for use and re-use.

And also:
• We should develop a model of a 'mixed economy' of public data so that everyone can benefit from some forms of two-way sharing between the public and the commercial sectors.

These recommendations very much follow the opinions that we expressed when Stephan Shakespeare was a guest at our meeting in September last year, and subsequent representations by DUG members, and will give another hefty shove toward opening up more government data.

Here are some links:
Shakespeare Review Report: https://www.gov.uk/government/publications/shakespeare-review-of-public-sector-information
Deloitte Market assessment: https://www.gov.uk/government/publications/public-sector-information-market-assessment
Ministerial quotes from Matthew Hancock and Francis Maude have also been published on the Gov.UK website:
The Guardian’s Public Leaders’ Network blog: http://www.guardian.co.uk/public-leaders-network/2013/may/15/stephan-shakespeare-publicsector-open-data  


May 13, 2013

The Census: Now and in the Future – SRA Summer Event – Afternoon, 26 June 2013

On 26 June the Social Research Association’s summer event “The Census: Now and in the Future” will examine this issue from a variety of expert perspectives.

“Chaired by Mark Easton of BBC Home Affairs, this promises to be a lively and engaging afternoon”, with presentations from:
• “How is the Census faring” Ian Cope, Director, Population and Demography, ONS
• “What’s needed locally?” Ludi Simpson, University of Manchester, president of the British Society for Population Studies
• “Can the use of ‘Big Data’ eliminate the need for yet another traditional Census in 2021?” Keith Dugmore, Director, Demographic Decisions Ltd, and Honorary Professor, UCL
• Discussant: Simon Briscoe, author and journalist (and advisor to Bernard Jenkin, MP)

The event will be held at the Local Government Association in Westminster, and the cost is £65. You can find a flyer and booking details here: http://the-sra.org.uk/event-registration/?ee=74 

May 12, 2013

Land Registry to release free historical property data 

Here’s some good news from the Land Registry:

“We are pleased to announce that we will release a number of free datasets as part of our commitment to the Open Data agenda.

The following datasets will be available to download from our website under Open Government Licence:
• 30 May 2013 – historical House Price Index (HPI) tables. These will be in CSV format and from July they will be in machine readable format as linked data. The tables form part of the HPI calculation and contain up-to-date indices, average house prices and sales volumes dating back to 1995 for all England and Wales.
• 28 June 2013 – historical Price Paid Data (PPD) in CSV format. This will be available as linked data from July. Historical PPD will contain records of the price paid for every residential property sold at full market value lodged for registration between 1 January 2009 and 31 January 2012 in England and Wales.”

There’s news of further datasets at: http://www.landregistry.gov.uk/announcements/2013/land-registry-to-release-free-historical-property-data

And you can see commentary from Geolytix at: http://geolytix.co.uk/blog/?p=175


April 20, 2013

"Everyone's postcodes to be privatised in Royal Mail flotation, despite objections from Sir Tim Berners-Lee" – Daily Telegraph

A report in yesterday’s Daily Telegraph starts:

“A national database containing the postcodes of everyone in Britain is to be privatised along with Royal Mail, despite direct "very forceful" protests to David Cameron from Sir Tim Berners-Lee, the founder of the world wide web.”

and includes:

“Cabinet Office minister Francis Maude has been privately battling with Business minister Michael Fallon over whether to include the database, worth between £500million and £900million, in Royal Mail's flotation, which is expected as early as October.”  (You might have spotted that both ministers are past winners of our annual Award).

Here’s the link to the full article:

Yes, a further battle in the address wars. DUG will continue to support the case for national addressing to be free.



April 8, 2013

Electoral Register opt-out rates for Local Authorities (ranging between 8% and 80%)

Callcredit Information Group has today published its '2013 Regional Electoral Opt-Out Report'. The message starts:

“Of the 46m consumers eligible to vote within the UK, 18m have chosen to opt-out and have therefore have made themselves unavailable for marketing activity and wider benefits it can bring. The over-all opt-out rate for 2013 is 39.8%, a 0.2% decrease from 2012.

There remains a large regional variation with the lowest opt-out rate of 8.03% and the highest of 80.25%, therefore marketers will need to continue to utilise additional data sources in order to verify their own customer data.”

You can read more at: http://www.callcredit.co.uk/press-office/news/2013/04/2013-electoral-register-opt-out-rates-begin-to-slow


April 4, 2013

Passenger statistics for every railway station in Britain: April 2011 – March 2012

“Waterloo has retained its position as the busiest station in Great Britain as top stations continue to attract growing numbers of passengers, reveal statistics published today by the Office of Rail Regulation (ORR).”

If this is of interest, you might want to put on your anorak and bobble hat before you open the link http://www.rail-reg.gov.uk/server/show/nav.1529 to scroll through estimates of passengers for every station in Britain.



March 26, 2013

2011 Census release 2.4 – Including Social Grade at OA level for England and Wales

ONS has announced further news about the release of more statistics, including:

• Approximated social grade (household reference persons aged 16 to 64) – at Output Area level (downloadable from NOMIS www.nomisweb.co.uk/ )

• Also, the Commissioned Tables service has been launched much earlier than planned. This could be useful if you want more detailed categories for topics such as birthplace, ethnicity, religion, etc. 


March 17, 2013

ODUG's response to the OFCOM review of PAF: "Don't sell our postcodes"; and our message to Michael Fallon MP

Coinciding with Heather Savory, Chair of the Open Data User Group, speaking at the DUG meeting on 15 March, here’s news of her blog. ODUG (including Bob Barr) has just made an excellent response to OFCOM about the future of the Postcode Address File:

Also, DUG has written about “the need for free use by citizens and the private sector, as well as the public sector” to Michael Fallon, who is now the Minister for Business and Enterprise, having been Chair of the Treasury Sub Committee, when it recommended the creation of a national address file. You may remember that he won our award at the DUG Conference in 2008 http://www.demographicsusergroup.co.uk/resources/2008conf-DUG_AWARD_2008.pdf

Let’s hope that this mounting pressure finally achieves a breakthrough in government policy, and the arrival of free addressing. 


March 5, 2013

DUG 2011 Census Workshop session – Presentations

DUG’s Census Workshop at the Royal Society on 27 February went well, and here are the three presentations:

• The value of small area Census data products to commercial companies (Annette Dellevoet)

• 2011 Census – Statistics (Emma White, Head of Policy and Analysis, 2011 Census Outputs at ONS)

• 2011 Census – Geography (Ian Coady, ONS Geography) 

February 24, 2013

Beyond 2011 – DUG’s minimum needs for Census-type statistics: our views, now sent to ONS

With many thanks to the members who contributed, here is a summary of DUG’s needs for future Census-type statistics, which has now been sent to ONS.

The ONS Beyond 2011 team is continuing to develop the business case to the Treasury.


February 24, 2013

The Shakespeare Review – & the possibility of more free Open Data – an invitation to respond to this YouGov Survey

Stephan Shakespeare, the Chair of the government’s Data Strategy Board, who was a guest at DUG’s September meeting, is reviewing public sector information. Here’s the message from the DSB:

“The Shakespeare review of Public Sector Information led by Stephan Shakespeare will examine ways in which the public sector can improve access to its data and information to stimulate economic growth, promote efficiencies, and bring wider benefits to society.

At the end of the review, recommendations will be made to Ministers on how to widen access to public sector information and how to exploit new and innovative opportunities for open data including issues around data protection and copyright.

Stephan is keen to gather views from as many types of users, re-users and potential users of public sector information as possible and invites you to complete an online survey https://start.yougov.com/refer/vvCYQdvVkq6JVB  . The survey will close on Friday 15th March 2013.

Your contribution to this survey is important and will help shape the future of the UK’s public sector information and open data policy.”

Do please complete it if you have 10 minutes – the greater the number of replies will increase the probability of more data (such as address files and mapping) being made freely available.


February 20, 2013

2011 Census Key and Quick Statistics for England and Wales – now other geographies, including postal sectors

Following the news on 30 January about 2011 Census data down at Output Area level for England and Wales, here’s news of the same statistics for other geographies, including postal sectors.

“ONS has now published 2011 Census Key Statistics for national parks, and Key and Quick Statistics for health areas, postcode sectors and National Assembly for Wales constituencies and Assembly for Wales electoral regions.”

The Key Statistics for national parks are only available via the ONS website at: http://www.ons.gov.uk/ons/guide-method/census/2011/index.html

The data for health areas, postcode sectors and National Assembly for Wales constituencies and Assembly for Wales electoral regions will be available via the NOMIS website at:


February 18, 2013

2011 Census Pack free from Geolytix

Here’s some good news from Geolytix. It starts:

“Today GeoLytix are releasing harmonised Output Area Census Data covering the c.200 core variables as a single easy-to-use flat file. This data is being released with no charge under the same Open Government License that the censuses themselves use. There are no additional attribution requirements, you can use this data in exactly the same way you can use raw census data. We’ve also written a swish User Guide, prepared meta-data tables and even included a single consolidated look-up file to higher geographies and proportional look-ups to the 2001 Output Areas.”

You can read all about it at: http://geolytix.co.uk/blog/ 


February 15, 2013

ESRC: Master's projects proposed by retailers – news of the first 10

Following the news on 12 January (below) there are already 10 retail projects announced on the ESRC’s website:


February 14, 2013

"Mapping the 2011 Census: how one man produced a library for all" – a report in The Guardian

Here’s news from the Guardian:

“Alex Singleton downloaded every single census dataset for every local authority in England - and then produced a free library of downloadable PDFs. Find out what he did…….

Singleton has basically produced a detailed mapping report - as a PDF and vectored images - on every one of the local authorities of England & Wales. He automated the process and has provided the code for readers to correct and do something with. In each report there are 391 pages, each with a map. That means, for the 354 local authorities in England & Wales, he has produced 127,466 maps.” [Goodness…..]


February 6, 2013

Further Benefits of an Open National Address Dataset – Paper by the Open Data User Group

It’s good to see the Open Data User Group is piling on the pressure for a free national address file, with an excellent paper about the benefits:



January 30, 2013

New stats after 10 years: 2011 Census Key and Quick Statistics for Output Areas published today

Here’s news from ONS of fresh Census data down at Output Area level for England and Wales:

There’s more information about Bulk Data in csv format:

And also further geography products for electronic download:

January 22, 2013

ESRC – Retail Research Data – A new website to help retailers find free Open Data easily

As well as the Masters’ projects for retail companies, another element of the ESRC’s Retail Research & Data project is a new website:
“The primary purpose of Retail Research Data (RRD) website is to enable insight and store location analysts working in retail organisations to get easy access to free datasets which may be of value to their businesses.”

The website has 6 types of data:
1. Small area geodemographic classifications - such as the Output Area Classification (OAC)
2. Small area statistics – particularly from the 2001 Census, on many topics
3. Lists of addresses, with postcodes, and places
4. Postcode directories – for linking postcoded addresses (including customer records) to Output Areas and other geographical areas, or to grid references for mapping
5. Map data – digital boundaries (such as Output Areas), and also background mapping
6. Sample surveys – giving lots of detail about particular topics

RRD also includes a mapping facility to view the Classifications and Small Area Statistics for the United Kingdom and its constituent countries.

Here’s the link: http://www.retailresearchdata.org

January 12, 2013

Do you have projects now that Master's students can help you to progress this spring & summer?

If you work for a UK retail company, this is an opportunity for you. The ESRC’s “Retail Research & Data” initiative, which involved commercial companies proposing projects for Masters’ students, was a great success last year, and the ESRC has decided to repeat it in 2013. We are therefore seeking proposals for new projects as soon as possible.

This presents a great opportunity to get a Master’s student to help you to make progress with:
• Major current issues, such as multi-channel marketing and customer insight, regional variations in the economy, store networks, transport, and social media.
• Projects that you’ve long had in mind, but not had the resource to carry out
• Working with data – both your own customer data, and also maximising the value to be obtained from government initiatives such as www.data.gov.uk, and the perfectly-timed arrival of the 2011 Census

If you have a project in mind, please email dugmore@demographic.co.uk ASAP for a Project Proposal Form. It’s essential to move quickly on this, to publicise projects early and get the choice of the best students.

Last year’s experience illustrates the main features of the initiative:
• The list of the 16 projects specified by 10 retailers: http://www.esrc.ac.uk/funding-and-guidance/collaboration/working-with-business/dissertation-opportunities.aspx
• ESRC’s publicity and explanation of the features to students and academic supervisors: http://www.esrc.ac.uk/funding-and-guidance/collaboration/working-with-business/retail-masters-student-projects.aspx
• Presentations by 12 of the 16 students at the Demographics User Group Conference: http://www.demographicsusergroup.co.uk/resources/ESRC+-+Masters+Student+presentations+-+Final.pdf

January 5, 2013

Members’ needs for data from government – our updated Manifesto

DUG’s Data Manifesto, which was initiated by Mike Whitelegge at M&S, has made an impact in Whitehall, and is proving a very useful summary wishlist / reference document.

Here is the latest version, which you might find useful if you are challenged with “What more government data would you like to see made freely available?” 


January 1, 2013

Happy New Year........... & here's a summary of our many activities in 2012

Very best wishes for a successful and enjoyable 2013.

Looking back on the past year, we hope that you’ll find this summary of the activities of the Demographics User Group during 2012 both interesting and useful.

As you’ll see, it’s been another busy and successful year, with members sharing their experience in numerous ways, pressing government for further progress in the policy of making more data freely available, and also a major initiative with the ESRC to provide Masters’ students with relevant research projects.


December 18, 2012

First 2011 Census results for Scotland

After all the activity by ONS for England and Wales, we now have the first 2011 Census statistics for Scotland.

The following link leads to some headline results, and also summarises the timetable for future releases (with Release 2 – Key and Quick statistics tables; Census Profiles; Population and Household estimates by postcode and Geography products – not becoming available until “from summer 2013”): http://www.scotlandscensus.gov.uk/en/censusresults/

And here’s the BBC’s “11 things we learned from the Scottish 2011 census”: http://www.bbc.co.uk/news/uk-scotland-20754751


December 13, 2012

New funding to accelerate benefits of open data – £7.5m Data Strategy Board Breakthrough Fund; a new £850,000 Open Data Immersion Programme; & easier OS data

The Data Strategy Board team reports:

“The Government has announced three new complementary Open Data investments
• A new £7.5m Data Strategy Board Breakthrough Fund to which public sector bodies can apply;
• A new £850,000 Open Data Immersion Programme to which companies can apply; and
• Upgrades to the format in which Ordnance Survey data can be downloaded to make it easier, more accurate and more flexible for companies, government, charities and individuals to use.”

All of the measures announced are expected to be in place by early April 2013. You can see more detail at: http://news.bis.gov.uk/Press-Releases/New-funding-to-accelerate-benefits-of-open-data-684c1.aspx

It’s another step in the right direction.

December 12, 2012

2011 Census – Key Statistics at Local Authority level now published

You will no doubt have seen some of the extensive media coverage about the latest release of Census statistics. Here’s ONS’s announcement:

“ONS has now published the 2011 Census key statistics at local authority level on its website at http://www.ons.gov.uk/ons/guide-method/census/2011/index.html

The release is supported by data visualisation, statistical bulletins, with commentary and short stories on international migration, ethnic group and identity, religion, and (in Wales) Welsh language. The release is also supported by a technical report comparing the estimates of economic activity to the regular labour market statistics ONS produces. Frequently asked questions will be available on the ONS website to help you answer census related queries.”

And here’s some mapping of the results:

You’ll see that these stats are at Local Authority level. The next releases will be:
• 30 January 2013: all of the Key and Quick Statistics tables at Output Area, ward, parish and parliamentary constituency levels.
• By the end of February 2013: Key and Quick Statistics for the remaining geographies, (including postal sectors) together with postcode estimates of population by *gender*, and the number of households.

December 6, 2012

"Transport costs are eating into family budgets, the ONS says" – Living Costs and Food Survey

The Guardian reports:

“British households have cut back on clothes and furniture spending to cope with a sharp rise in petrol bills, according to the government's annual Family Spending survey. The Office for National Statistics revealed that households on average spent a record £483.60 a week in 2011, up £10 on the year before – but much of the increase was gobbled up by a 9% rise in spending on petrol. The figures also reveal that spending is falling fastest among the poorest, and suggest the "squeezed middle" may be a statistical myth.”

You can read more at: http://www.guardian.co.uk/money/2012/dec/04/household-spending-record-high-rising-petrol-prices  
And here’s the link to the survey report: http://www.ons.gov.uk/ons/rel/family-spending/family-spending/family-spending-2012-edition/index.html


December 5, 2012

Top 10% of households are 850 times wealthier than the bottom – ONS

The Guardian has reported the latest results from ONS’s Wealth and Assets Survey:
& here’s a link to the ONS website:


November 23, 2012

2011 Census latest release – first estimates (Population & Households) at Output Area level for England and Wales

ONS has just sent an alert announcing the publication of population and household estimates at Output Area and ward level for England and Wales. The release is available on the ONS website at: http://www.ons.gov.uk/ons/guide-method/census/2011/index.html

You’ll see that it’s region-by-region, and in Excel format (DUG has been pressing for csv for future releases, and ONS has promised it). This follows the earlier publication of geography products such as OA boundaries (see DUG News 31 October)

ONS’s announcement continues:

“The first phase of the second release is planned to commence on 11 December with a full package of key statistics and a selection of quick statistics tables, for England and Wales. This will be supported by statistical bulletins for England and Wales, for Wales (in English and Welsh), with commentary and short stories on international migration, ethnic group and religion, and Welsh language. The release will be accompanied by data visualisations and by a fact pack product that provides quick reference 2001-comparability information for each local authority. Table layouts will be published prior to the release.

On 30 January 2013, we will issue all of the Key and Quick Statistics tables at output area, ward, parish and parliamentary constituency geographies. These outputs will also be accompanied by a statistical bulletin and census analysis. By the end of February 2013, we will issue the Key and Quick Statistics for the remaining geographies such as Welsh Government devolved constituencies, National Parks and health areas together with postcode estimates by [*gender*] and for the number of households. We will announce the precise date in February as soon as we can and at least four weeks prior to that publication.”


November 23, 2012

Open Data chief says postcodes should be free

Just in case you haven’t spotted this in “Civil Service World”, there’s more support for the PAF being made free…….

“The UK’s postcode address database should be released for free use by private companies, the chief executive of the new Open Data Institute (ODI) has told CSW”.

Speaking in an interview with CSW, Gavin Starks argued that the UK’s Postcode Address File (PAF) is “such a fundamental building block for pretty much every business that needs an address that it’s kind of crazy that we don’t have that as a properly open file.”

You can read more here: http://network.civilservicelive.com/news/csw/read/711763/open-data-chief-says-postcodes-should-be-free  
& here’s a bit more about the ODI’s recently-appointed chief, Gavin Starks: http://www.theodi.org/odi-top-jobs


November 21, 2012

GeoLytix release full Postal Boundary Set as Open Data

Here’s news of a really encouraging step forward with more (free) Open Data. The announcement reads:

“Today, GeoLytix are pleased to release our full set of Autumn 2012 GB postal sector boundaries under an Open Data license. Postal boundaries are a critical component of our countries spatial infrastructure. They are used in everything from defining media boundaries and store delivery areas to web map searching, analysing customer data and producing a wide range of maps for the public.
You are free to use and exploit these data in any way you see fit with the single requirement that you attribute GeoLytix, the Ordnance Survey, the National Statistical Agencies and the Royal Mail as the copyright holders. Postal Geography constantly changes and this Open Data version is frozen. Updates and support are available under a paid-for GeoLytix license.”

The story behind their creation is at: http://geolytix.co.uk/blog/ . You can access the product documentation including the license, an FAQ and the user guide together with the full data here http://www.geolytix.co.uk/data-and-software/postal-boundaries  (no registration required)


November 12, 2012

APPSI paper on a National Information Framework for Public Sector Information and Open Data – Call for comments

Government’s Advisory Panel on Public Sector Information (APPSI) has just published this paper, which supports the case for a strategic approach to getting better access to government information such as address files, property information, and map data:


APPSI’s views should help create more momentum for Open Data. You’ll see some more background below. If you have comments on the ideas and proposals, please email them to secretariat@appsi.gsi.gov.uk


November 10, 2012

George Osborne champions science, & The Data Revolution is first on his list

Here’s some encouraging news from a meeting yesterday at the Royal Society.

George Osborne, Chancellor of the Exchequer, spoke of both his belief in the value of science as a driver of the UK economy and his commitment to science funding into the future. He identified eight areas that the government believes that Britain is a world leader.

The first on his list was “The Data Revolution and energy-efficient computing”. His comments included:
“We have some of the world’s best and most complete data-sets in healthcare, demographics, environmental change and food.”

“We have also created the open Data Institute in the East of London in Tech City, to bring together study of all the data from our whole environment. And as a government we are making more and more of that data available. Business will invest more as they see us invest more in computational infrastructure to capture and analyse data flows released by the open data revolution.”

It’s good to see data at the top of the agenda. You can see the full detail at: http://royalsociety.org/news/2012/osborne-at-royal-society/


November 1, 2012

Open Data User Group calls for address files to be made free of charge

It’s great that the Open Data User Group has taken up the cause to make address files to be made free of charge. Here are the headlines:

• ODUG calls for the publication of the Postcode Address File (PAF) as open data under the Open Government Licence
• Charging for all Postcode Address File (PAF) licencing should end
• The Ordnance Survey AddressBase Plus and the National Street Gazetteer (NAG) should be released as open data in conjunction with the Postcode Address File (PAF)

……and you can see more at: http://data.gov.uk/odug 

Many thanks indeed to Sarah Hitchcock, and to other members of DUG who have helped make the case.


October 31, 2012

2011 Census Geography products now available (in advance of statistics next month)

Here’s news from ONS about the geography products that they have just published in advance of the Output Area level statistics that start to flow (with counts of population and households) on 23 November:


You’ll see from the links that they comprise:
• 2011 Census boundary files (.shp) for OAs, LSOAs and MSOAs.
• 2001 Census boundary files (.shp) for OAs, LSOAs and MSOAs.
• 2001 to 2011 Census geography OAs, LSOAs and MSOAs lookup files
• 2011 OA to higher area lookup files
• Postcodes to OAs, LSOAs and MSOAs
• 2011 Census median population weighted centroid files (.shp) for OA, LSOA and MSOAs datasets

We learn that only 2.6 per cent of 2001 OAs have changed as a result of the 2011 Census, with the average population in an OA increasing from 297 in 2001 to 309 in 2011.


October 24, 2012

The DUG Conference “Retail Issues, Big Data, and Research”

and the winner of our 2012 Award for “Better information from government” is ................................The Department for Work and Pensions

The DUG Conference was held on 10 October at the Royal Society, and you can see the programme, presentations, and a list of attendees at the tab on the left.
The Department of Work and Pensions, represented by its Director of Information David Frazer, was present to collect the DUG Award to the Department “for proactively collecting, re-using, and publishing more and better data”.

A DWP Press Release http://www.dwp.gov.uk/newsroom/press-releases/2012/oct-2012/dwp105-12.shtml  includes:
Minister for Welfare Reform Lord Freud said:

 “The Government has a responsibility to be open with data, so we can be more accountable to the public as well as provide the public and private sectors with the tools to plan and improve services.
“Better data also gives companies the information they need to find opportunities for growth, which benefits us all.
“I want to thank the DWP teams working to improve our data and for allowing greater public access of our information resources and to congratulate them on winning this award.”

Our citation reads:
“Commercial companies, when making decisions about targeting investment towards particular types of customers and local markets, welcome the availability of government statistics across the United Kingdom. The Census of Population is one major source, but statistics derived from government departments’ administrative files present a growing opportunity.
The Department for Work and Pensions has led the way, using the information that it collects from the UK’s citizens to create anonymous counts of various populations for very small areas, and making these easily accessible. It is also encouraging the wider use of government surveys, with the Wealth and Assets Survey benefiting from the addition of a neighbourhood classification.
Members of the Demographics User Group (DUG) strongly welcome these developments, and the future plans outlined in the DWP Open Data Strategy, which will enable better understanding of customers and increase the efficiency and growth of businesses”


October 24, 2012

Government review of open data to explore growth opportunities – Chaired by Stephan Shakespeare

Here’s news of a new government review which “aims to improve access to data and help promote economic growth by examining the market for public sector information.”

It’s to be led by Stephan Shakespeare, Chair of the Data Strategy Board, who was guest speaker at DUG’s September meeting.

At the end of the review, recommendations will be made to Ministers on how to widen access and consider new and innovative opportunities for open data. In particular, the review will look at the following areas:
• the current use of public sector information within government, the private sector and by the general public;
• the opportunities for innovation and developing new data services with public sector information;
• the constraints on developing new services with public sector information and the role that current funding models play;
• the impact of competitiveness on existing and new businesses entering the market, as well as the way in which organisations secure access to data; and
• an audit of current and past studies, as well as taking stock of the current use and re-use of public sector information.

There’s more information at: http://news.bis.gov.uk/Press-Releases/Review-of-open-data-to-explore-growth-opportunities-68203.aspx 


October 23, 2012

2011 Census stats on second address / second homes

ONS has just published stats from the 2011 Census on the number of people with second addresses in local authorities in England and Wales.

The BBC has run the story http://www.bbc.co.uk/news/uk-20026773 together with mapping http://www.bbc.co.uk/news/uk-19759650  

The data can be accessed here: http://www.ons.gov.uk/ons/guide-method/census/2011/index.html

The release includes two separate tables which provide, down to local authority level:

 • the number of people who spend more than 30 days a year at a second address in a local authority where they do not usually live, and
• the number of people usually resident in each local authority who had a second address elsewhere


October 13, 2012

Public Administration Select Committee – Study 4: Communicating and publishing statistics – Submission on behalf of the Demographics User Group

Following our response to the first of House of Commons Public Administration Select Committee’s studies on statistics and their use in Government (see August 4), another topic is “Communicating and publishing statistics”:


You can see DUG’s submission here. The key themes are summarised as the need to:

• Put more emphasis on “the last 100 yards”, making it easier to find and use data.
• Shift some resources from statistical production to marketing (identifying needs and meeting them) within the various parts of the Government Statistical Service.


October 13, 2012

ONS's Beyond 2011 Project – papers, & free seminars

Following the news about ONS’s Beyond 2011 consultation, looking at alternatives to another Census (see September 26), here’s the latest:

1) News of research work being undertaken, & the evaluation of options: http://www.ons.gov.uk/ons/about-ons/what-we-do/programmes---projects/beyond-2011/index.html

2) Seminars. Here’s a message from ONS:

“You may be interested to know that we will be running a series of seminars during November as part of our ongoing engagement with our stakeholders.

Seminars are planned for the following dates and locations and will run from 10.30 -15.45:
• 26 November 2012 – Manchester
• 28 November 2012 – London – Royal Statistical Society

During the day we plan to cover developments in the Programme, including our investigations into the options, research progress and how we are taking forward the business case and quantification of benefits. There will be the opportunity to participate in group work and a panel session in the afternoon. If you wish to attend one of these seminars please email beyond2011@ons.gov.uk  stating the date and location you would like to attend and the organisation, if any, you will be representing. Please note that there are limited places at these events and places are allocated on a first come first served basis.” 


September 26, 2012

Will there be another traditional Census? News of two government reports

Here’s news of two separate government investigations into whether there should be another traditional Census in 2021.

1. House of Commons Science and Technology Committee:

“MPs have warned that a decision to discontinue the census would have a negative impact on the Government's social science analysis, as well as social science research in the UK, and that alternative systems for collecting population data may not prove to be any cheaper.” You can see the committee’s report at:

2. ONS’s Beyond 2011 User Requirements Consultation

This report provides the results of the first Beyond 2011 public consultation which ran from October 2011 to January 2012 (you can see DUG’s submission below on 17 January 2012). A total of 266 organisations and individuals provided responses to the consultation and 207 people attended the supporting workshops.

The consultation focused on two broad areas:
• the current and future requirements for population and socio-demographic statistics on different topics; and,
• the trade-off between accuracy, geography and the frequency at which the statistics are produced.
You’ll find the report at: http://www.ons.gov.uk/ons/about-ons/what-we-do/programmes---projects/beyond-2011/news-and-events/reports-and-publications/index.htm

An ONS statement also includes:
“In October 2012 we plan to release a whole range of papers on this research work undertaken over the past 6 months. These will explain much more about the approach we are taking to evaluate the options and what the next steps are. We will be reporting much more regularly from now on with further releases later in 2012 and early in 2013”. This will give us the opportunity to learn much more. 


September 7, 2012

ONS's 2011 Census Newsletter – September 2012

Here is the latest from ONS about the release of 2011 Census statistics, including news of the first results, details of the next Census data release - planned for between November 2012 and February 2013 and available via the NeSS website – and a summary of the Census Statistical Disclosure Control methods applied to protect the confidentiality of individuals and entities such as businesses.


August 30 , 2012

Marks & Spencer customer poll suggests better times ahead – Guardian report quotes Steve Bond

Here’s news from yesterday’s Guardian:

“Marks & Spencer said on Tuesday that the latest poll of confidence among its shoppers had delivered its most positive score in more than two years, providing some good news for its embattled chief executive, Marc Bolland. The high street chain is regarded as a barometer of consumer spending and its authoritative monthly poll involves interviews with 4,000 customers.

M&S's director of customer insight and loyalty, Steve Bond, said the latest reading for the four weeks to 17 August was 12 points higher than a year ago. Optimism was riding high, he said, thanks to the feelgood effect of events such as the Olympics, although he cautioned that the poll, launched by Bolland in May 2010, had only ever been conducted in a downturn.

The "most subdued group" in the marketplace remained families battling higher food and fuel prices, added Bond”.

You can read more at: http://www.guardian.co.uk/business/2012/aug/28/marks-and-spencer-increased-consumer-confidence 


August 21 , 2012

ONS: Glen Watson appointed as Director General

Following the retirement of Steve Penneck, it’s good to see that Glen Watson has been appointed as the new Director General of ONS.

Glen is currently Director of Census at ONS, having previously been appointed Director of Neighbourhood Statistics in 2004 (when he spoke at the first DUG conference).

Here’s ONS’s announcement: http://www.ons.gov.uk/ons/media-centre/statements/new-director-general/index.html?format=print 


August 15 , 2012

ONS/BIS Meeting on "The Changing Shape of UK Business" – Tuesday 6 November 2012

For those of you interested in stats about businesses, here’s advance news of an all-day event, coordinated jointly by the ONS and the Department for Business, Innovation and Skills, whose objectives are to:

• “share what we know about the structure, content and performance of UK business;
• raise awareness of the importance and coverage of our statistics;
• hear from users about the range of uses to which our statistics are put; and
• discuss future plans and priorities with a range of users.”

The event is free of charge, but please note that numbers will be limited. You’ll find more detail at:


August  8 , 2012

What is a data scientist worth? Read all about it in the Guardian

A DUG member has spotted this nugget:


It’s clear that if you are a “Data Scientist” you can earn more than if you are a “Data Analyst”…….


August 4, 2012

Public Administration Select Committee – Statistics and Registration Service Act 2007 – Submission on behalf of the Demographics User Group

Following the session with Bernard Jenkin, MP, Chair of the House of Commons PASC, at the DUG meeting in March, and our letter of 1 May (see below), the committee is launching a programme of work on statistics and their use in Government. It is conducting its scrutiny by means of ten short “studies”:

The first of these is on the Statistics and Registration Service Act 2007. DUG has made a submission, with the key themes being the need to:
• Understand and meet the needs of users outside government
• Divert some resources from production into improving access to statistics, and hence increasing use, and maximising value: “the last 100 yards”
• Make much more use of administrative files to create statistics at the lowest Output Area level
• Put more emphasis on speed of delivery, rather than marginal gains in quality 


August 2, 2012

Public Accounts Committee publishes report on Transparency /Open Data

The House of Commons Public Accounts Committee has just published a report 'Implementing the Transparency Agenda'. The Rt Hon Margaret Hodge MP, Chair of the Committee of Public Accounts, said:

"This Committee fully supports the principle of greater openness and its potential to strengthen accountability and drive improvements in public services. But the Government has a lot more work to do before that potential is realised.

It is simply not good enough to dump large quantities of raw data into the public domain. It must be accessible, relevant and easy for us all to understand. Otherwise the public cannot use it to make comparisons and exercise choice, which is the key objective of the transparency agenda”.

There’s more comment, plus a link to the full report, here:


July 27, 2012

Republic of Ireland – 2011 Census results – Part 2

Following the first publication summarising the results of Ireland’s 2011 Census (see 3 April below), here’s the second, which examines social and economic topics such as employment, occupations, education and skills and health related issues:

Again, you’ll see that the report has lots of eye-catching maps & charts. 


July 26, 2012

DUG Members’ needs for data from government – our Manifesto (Updated July 2012)

DUG’s Data Manifesto, which was initiated by Mike Whitelegge at M&S, has made an impact in Whitehall, and is proving a very useful summary wishlist / reference document.

Here is the current version, which you might find useful if you are challenged with “What more data would you like to see made freely available?” 


July 26, 2012

First meeting of the Open Data User Group (ODUG) – Match Report

The Cabinet Office’s Open Data User Group (ODUG) – see the earlier news on 10 July – has held its first meeting.

Heather Savory, the Chair, has written a blog http://www.guardian.co.uk/public-leaders-network/blog/2012/jul/18/open-data-group-clear-agenda and Sarah Hitchcock of Sainsbury’s tells me that the group got off to a good start.

The note of the meeting and slides are on data.gov.uk at: http://data.gov.uk/library/minutes-odug-10072012 and: http://data.gov.uk/library/slides-odug-10072012 


July 23, 2012

2011 Census – ONS's plans for the 2nd release (November – February), including lots of stats at Output Area level

ONS has now published its plans (subject to any redesign as a result of the application of statistical disclosure control) for its Second Release, which gives 36 tables of Key Statistics, and 71 tables of Quick Statistics, at Output Area level.

The second release will also include postcode estimates (numbers of people by gender in each postcode), and estimates of occupied households in each postcode will be produced.

You can see the detailed tables here:

“ONS consulted extensively with census users about requirements for the output of census statistics and consequently the vast majority of 2001 Census outputs will be repeated in 2011. This is for comparability and continuity purposes, and takes account of user satisfaction with the design of the 2001 statistics.

The statistics in the second release will correspond to the 2001 Census key statistics tables produced for univariate information (counts and as percentages), univariate tables and headcounts. A look-up file is available to download. The look-up file will be developed to be relevant to every release of 2011 Census statistics.” 


July 16, 2012

2011 Census – First results for England and Wales – Largest population increase shown by any Census since 1801

The ONS has just published the first (Local Authority level) results from the 2011 Census for England and Wales: www.ons.gov.uk/census  

There’s some good summary coverage on the BBC website, which starts:

“The population of England and Wales has reached 56.1 million, up by 3.7 million in a decade, the 2011 Census shows.
It is the largest growth shown by any census since they began in 1801.
The population in England was 53 million on 27 March 2011 when the census was taken, about 400,000 more than official estimates. In Wales the population was 3.1 million.
Around 55% of the 3.7m increase was attributed to net migration.
The census showed that the population for England and Wales was greater than official estimates rolled forward from 2001 - by just over 500,000.”

You can read more at: http://www.bbc.co.uk/news/uk-18838540  


July 14, 2012

Data Strategy Board (DSB) – Department for Business Innovation & Skills lists its members

Following hard on the heels of the announcement of the membership of the Open Data User Group (see 10 July), the Department for Business Innovation & Skills has news of the new Data Strategy Board: http://www.bis.gov.uk/transparency/data-strategy-board

If you follow the link to the note of the meeting on 11 July, you’ll see who has been appointed. The three places reserved for “data users outside the public sector” have gone to:

Chris Yiu, Head of Digital Government Unit, Policy Exchange
Bill Roberts, CEO, Swirrl IT Ltd
Steve Thomas, MD Strategic Development, Experian Plc

It is striking that (unlike Sainsbury’s representation on ODUG), the Department for Business has not included any representative of the huge existing community of business end-users. Rather, we have a Think Tank, and two Value-Added Resellers / Publishers – indeed, you might have spotted that Experian has seats on both ODUG and the DSB. 


July 10, 2012

Open Data User Group (ODUG) – Cabinet Office announces members – including Sarah Hitchcock of Sainsbury's

Following the earlier news (June 17) about the launch of ODUG, it’s really good to report that Sarah Hitchcock, Head of Network & Planning at Sainsbury’s, has been appointed as one of the 12 members.

Here’s the Cabinet Office’s announcement:

You’ll see that other members of the Group also include Bob Barr (Manchester Geomatics), and Mick Cory (Chair, UK Location User Group), plus a representative from Experian.

ODUG will add to the pressure that we’ve been applying since 1998 for more Open / Free Data. 


June 29, 2012

The Government's new Open Data White Paper: Unleashing the Potential

The Government has just published its “Open Data White Paper: Unleashing the Potential”

“……..which sets out how we’re putting data and transparency at the heart of government and public services. We’re making it easier to access public data; easier for data publishers to release data in standardised, open formats; and engraining a ‘presumption to publish’ unless specific reasons (such as privacy or national security) can be clearly articulated. From the Prime Minister down, central Government is committed to making Open Data an effective engine of economic growth, social wellbeing, political accountability and public service improvement.”

At the launch event yesterday (our DUG Award winner) Francis Maude gave an evangelical speech – the White Paper has a big ambition, and there’s no doubt that he means every word of it.

The link http://www.cabinetoffice.gov.uk/resource-library/open-data-white-paper-unleashing-potential leads to both the White Paper, and a document which gives each Department’s plans for releasing more data. 


June 28, 2012

Job vacancies

Several new job vacancies in DUG member companies can be seen at the tab to the left. 


June 25, 2012

2 reports: Deloitte's on Open Data, and the Royal Society on Open Science

The momentum in favour of open data continues to grow, and this is illustrated by two new reports:

1. Deloitte’s “Open Data – driving growth, ingenuity and innovation”

“Welcome to our first white paper in a series of research reports and white papers from Deloitte Analytics on the topic of ‘competing on insights’.
Open data – Driving growth, ingenuity and innovation has been researched and written in collaboration with Professor Nigel Shadbolt, head of the Web and Internet Science Group at the University of Southampton and co-director of the Government’s new Open Data Institute.
This publication presents Deloitte’s vision for open data – a subject that has, until now, been overshadowed unhelpfully by considerations of ‘big data’. Open data is about much more than improving government transparency and it is now emerging from its silent revolution……….”
You can see more at: http://www.deloitte.com/view/en_GB/uk/market-insights/deloitte-analytics/de0d882044ad7310VgnVCM3000001c56f00aRCRD.htm  

2. The Royal Society’s “Science as an open enterprise”

This report “aims to identify the principles, opportunities and problems of sharing and disclosing scientific information and asks how scientific information should be managed to support innovative and productive research that reflects public values.” The blockbuster 105-page report can be downloaded at:
Alongside this project, Research Councils UK in partnership with JISC, the Royal Society and Sciencewise-ERC, commissioned a public dialogue exercise on open data, aiming to engage the public with issues around open data to potentially inform future policies in this area. The report of this exercise is can be downloaded from the same link. 


June 22, 2012

Scotland – 2011 Census – News of plans for outputs, & timing

Following the news from ONS about its plans for outputs for England & Wales, and their timing (see news on April 3), National Records for Scotland have now released their 2011 Census Output Prospectus and set out a timetable for a phased release beginning with the first outputs in December this year. The dataset release phases are similar in content to those in England & Wales but will follow six months behind those planned by ONS.

You can see the full detail at: http://www.gro-scotland.gov.uk/files2/the-census/progress-reports/census-2011-outputs-prospectus.pdf 


June 20, 2012

ONS and UCL work on an updated 2011 Output Area Classification

Following DUG’s response to the consultation on plans for a new UK Output Area Classification (see news on 23 March), ONS has now published a summary of the responses: http://www.ons.gov.uk/ons/guide-method/geography/products/area-classifications/ns-area-classifications/new-uk-output-area-classification/index.html

You’ll see the conclusions & next steps which, taken in the context of DUG’s views, look very encouraging. 


June 18, 2012

ONS: "First results from the 2011 Census will be released on Monday 16th July"

Hot news from ONS………

These stats will be populations by age and [gender], and occupied households estimates for England and for Wales (just at Local Authority level).

Key and Quick statistics, including many simple counts at Output Area level (NB), plus postcode populations, are due to follow on between November 2012 – February 2013. 


June 18, 2012

New statistics from DECC on fuel poverty for small areas

Here’s a link to new data on fuel poverty in England from the Department of Energy and Climate Change: http://www.decc.gov.uk/en/content/cms/statistics/fuelpov_stats/regional/regional.aspx

You’ll see that you can request data right down to Output Area level.


June 17, 2012

Open Data User Group (ODUG) – Cabinet Office Introductory Event 14 June 2012

The Cabinet Office held an event last Thursday (14 June) to introduce the Open Data User Group (ODUG). The event attracted a big crowd, and you can see the presentation slides here

ODUG will add to the pressure for more Open / Free Data that we’ve been applying since 1998. 


June 8, 2012

ESRC's Retail Research & Data – 17 Master's projects now underway

The ESRC is seeking to increase its engagement with the retail sector (http://www.esrc.ac.uk/funding-and-guidance/collaboration/working-with-business/  ) and an element of this is the Retail Research and Data project that Professor Paul Longley of UCL and Keith Dugmore are running.

We are really pleased to report that 17 Master’s projects which have been specified by retail companies (most of which are DUG members) now have students working on them. You’ll see it’s a fascinating list of topics: http://www.esrc.ac.uk/funding-and-guidance/collaboration/working-with-business/retail-masters-student-projects.aspx

We are hoping that the ESRC will be asking retailers to specify more projects towards the end of this year. 


June 1, 2012

News from ONS about 2011 Census Geography Products – Boundaries and Lookup Files

ONS has just published its 2011 Census Geography Prospectus. This contains details of what and when geographic products (digital boundaries and lookup tables) will be released. When available, these will be available to download from the ONS website or supplied on hard media from ONS Customer Services.

The first products will be available in October 2012 (subject to the Census outputs for Output Areas being published on time in November) and the second will follow in January 2013.

There’s a summary here:
and you can download the Prospectus itself, and also a FAQ document, e.g:

Q: Will the output area (OA)/super output area (SOA) and workplace zone (WZ) boundaries be free, including for commercial re-use?
A: The Office for National Statistics (ONS) has submitted a request that these boundaries will be available as Open Data, under Open Government Licence (OGL) terms and conditions. This would make them available for free including for commercial re-use. We are awaiting the decision from this process, expected at the end of June 2012.

& “We don’t anticipate that more than 5% of the OAs/SOAs overall across England and Wales will change”.

The anoraks amongst us will also be pleased to see that there’s the option to have OA boundaries clipped to the coastline. 


June 1, 2012

Job vacancies

Several new job vacancies in DUG member companies can be seen at the tab to the left.


June 1, 2012

Information Commissioner's Office consults on new anonymisation code of practice

“The Information Commissioner’s Office (ICO) has begun a public consultation on a new anonymisation code of practice. The code will provide guidance on how information can be successfully anonymised and how to assess the risks of identification. The ICO has also launched a tendering process to establish a network of experts to share best practice around the release of data in an anonymised form. Anonymisation techniques can convert personal data into a form so that individuals are no longer identifiable. The consultation will be relevant to any organisation that wants to release anonymised data, for example under the government’s open data agenda.”

See more at: http://www.ico.gov.uk/news/latest_news/2012/ico-consults-on-new-anonymisation-code-of-practice-31052012.aspx 


May 23, 2012

News of the Open Data Institute from Nigel Shadbolt & Tim Berners Lee

The Open Data Institute will “study the best ways to exploit the growing amounts of data being made public by the government, & train people to use the data in business and advise on what kinds of information should be released.”

The ODI was first announced in the Autumn Statement and now its business plan has been given the go-ahead by the government. A total of £10m of public funds have been pledged over five years to get the ODI going, but the cash is contingent on a similar sum being pledged by businesses.”

You can find out more at: http://epsiplatform.eu/content/shadbolt-tbl-outline-open-data-institute-uk 


May 23, 2012

Chair of the Open Data User Group (ODUG) appointed – Heather Savory

Following up the news on 19 March about ODUG:

“The Minister for the Cabinet Office, Francis Maude, has appointed Heather Savory as the chair of the Open Data User Group (ODUG). Heather has a background in high-technology businesses and strong strategic and governance experience of central government, entrepreneurial businesses and consulting.”

You can read more at: http://www.cabinetoffice.gov.uk/news/chair-open-data-user-group-appointed 


May 15, 2012

Generating Value from the 2011 Census – MRS Census Geodemographics Group seminar – afternoon, 19th September 2012

Here’s news of a seminar “Generating Value from the 2011 Census – How business can benefit from the new free Census data” which will follow hard on the heels of the first release of 2011 Census stats (scheduled for July).

You’ll see that the speakers include the ONS, and that the price is very reasonable (& even lower if you book before 30 June). 


May 15, 2012

ONS 2011 Census – News of plans for outputs, & timing........... detailed comments from the GLA/SASPAC team

Following ONS’s publication of its plans for Census Outputs, the Greater London Authority / SASPAC team has pulled together some detailed comments.

The issues raised include postcode headcounts, urban & settlement areas, flows data, geographical products, and workplace geographies – all techie stuff, but they might come back & bite us in future unless they are discussed with ONS soon. 


May 5, 2012

Retail Sector growth April 2012 – presentation by the British Retail Consortium & the Association of Convenience Stores

The Department of Business Innovation & Skills held a Retail Growth Strategy workshop on 17 April 2012.
Here is a presentation by the BRC and ACS on the shape of, opportunities for, and challenges to the UK retail sector – you might find some of the stats & graphics useful. 


May 4, 2012

Job vacancies

Several new job vacancies in DUG member companies can be seen at the tab to the left.


May 1, 2012

Open Data – Report by the National Audit Office: "Implementing Transparency

The National Audit Office’s newly published report “recognises the strength of the strategic case for greater transparency, and highlights progress across government in fulfilling most of its initial commitments. However, government needs a better understanding of costs, benefits and use to assess whether transparency is meeting its objectives of increasing accountability, supporting service improvement and stimulating economic growth……”

Some key facts:

• £16bn (NB!) is the Government’s estimate of the value of public sector information to the UK economy in 2011
• 7,865 data sets were linked to the www.data.gov.uk website in December 2011
• 47m estimated number of visits made to the police crime map website between February and December 2011

You can find more here: http://www.nao.org.uk/publications/1012/implementing_transparency.aspx  


May 1, 2012

DUG’s letter to Bernard Jenkin MP: the private sector’s need for government statistics and data – potential issues for Inquiries

Following DUG members meeting Bernard Jenkin, MP, Chair of the House of Commons Public Administration Select Committee, here is a follow-up letter.

It highlights 3 areas for potential inquiries by the committee:

1. The government’s Trading Fund business model severely limits the use that can be made of existing data, and needs fundamental reform. (This is referring to Ordnance Survey, Valuation Office Agency, etc.).
2. The use of government administrative files to create new statistics needs to be accelerated. (e.g. HMRC data on income).
3. Government suppliers of data need to put more emphasis on “the last 100 yards”, making it easier for new users, particularly small businesses, to find and use data. 


April 30, 2012

UCL's Centre for Advanced Spatial Analysis – Conference highlights using scoop.it

UCL's Centre for Advanced Spatial Analysis held its Smart Cities conference on 20 April. Here’s a report:

& media coverage included stories on:
• Crunch commuter data to track changing communities
• Urban analysis specialists launch the City Dashboard, a hub for real-time data on major UK cities
• How data could help fix London's transport woes
• Meticulous map shows every London bus journey

You can see more at: http://www.scoop.it/t/spatial-analysis 


April 30, 2012

Scotland's 2011 Census – Update on Statistical Outputs

Following the news from ONS about its plans for outputs for England & Wales, and their timing (see news on April 3), National Records of Scotland has published the first in a series of quarterly progress notes designed to keep users up to date on the statistical outputs from the 2011 Census:

The headlines are:
• Table outlines will be published very shortly together with a note of any significant changes made since they were first published in June 2011
• First 2011 Census results are planned for late 2012 (NB: delayed from the original July date) – a full output prospectus will be published around the end of May
• A lot of effort is being put into producing compatible statistics across the whole of the UK wherever possible 


April 12, 2012

Government seeking a Chair of the Data Strategy Board

Following up the news of 19 March about the government’s plans for open data, it is now seeking a Chair for its new Data Strategy Board:

“Outstanding Chair to deliver better access to public data
The Government has taken significant steps towards extending access to public sector information. The new Data Strategy Board (DSB) will advise Ministers on what public sector data should be released, initially from the four Trading Funds that make up the Public Data Group (PDG): the Met Office, Ordnance Survey, Companies House and the Land Registry. The data from these organisations and others across the public sector has the potential to unlock growth and create new opportunities for high-value businesses of all sizes across the UK.
.……………. To help realise our ambitions we are now seeking an experienced Chair to lead DSB in its formative years……..” 

April 11, 2012

Centrica joins the Demographics User Group

Great news that Centrica has announced its joining DUG. Here’s the press release:

“Centrica is the latest company to join the Demographics User Group (DUG), whose membership already includes major companies across a range of sectors.

Centrica is a top 30 FTSE 100 company with energy businesses in the UK and North America. Upstream it sources, generates, processes, trades and stores energy. Downstream, it supplies gas and electricity to millions of homes and businesses and offers a distinctive range of home energy solutions and low carbon products and services.

Centrica’s Customer Insight team sits within British Gas’s Commercial Marketing directorate. Their activity spans a range of disciplines from Analytics & Modelling to Market Research and it strives to help the business make great commercial decisions in its role at the heart of the business.

Senior Analyst, Harry Harrison stated "We are passionate about putting the customer at the heart of our activity and the more we understand them, the better the chances of us being able to achieve this aim. We are very pleased to be joining DUG and we look forward to sharing our own experiences and learning from the experience of others across different industry sectors to facilitate this aim." 


April 3, 2012

Republic of Ireland – 2011 Census results

“This report is the first of two publications summarising the results of the 2011 census. This first one, This is Ireland - Part 1, looks at overall population change by county; it also examines age, marriage, households and families as well as including first results on nationality, foreign languages, the Irish language, religion and housing……..”

You’ll see that the report has lots of eye-catching maps & charts:

April 3, 2012

ONS 2011 Census - News of plans for outputs, & timing

ONS has now published its Outputs Prospectus, which sets out the release plans for the 2011 Census statistics in England & Wales:

You’ll find the detail at the tab on the left: “Release plans for 2011 Census statistics”. There are four releases
First release July 2012: Age and sex, and occupied households estimates for England and for Wales (just at Local Authority level)
Second release November 2012 – February 2013: Key and Quick statistics (including many simple counts at Output Area level (NB), plus postcode populations)
Third release March 2013 – June 2013: Local Characteristics (including more detailed OA tables)
Fourth release July 2013 – October 2013: Detailed Characteristics, Detailed Theme and Armed Forces
“Subsequent releases” will include Workplace (& possibly Daytime) population.

There’s a tab on the right “Prospectus tables and schedule” to see a spreadsheet which gives lots of detail.

Scotland and Northern Ireland will be releasing their own prospectuses later this month.

March 30, 2012

Mary Portas High Street plans get government go ahead

“The government has announced it has accepted "virtually all" 28 recommendations from Mary Portas in its bid to rejuvenate UK High Streets. These will include creating dedicated "town teams" to manage High Streets and making parking more affordable. It has also unveiled additional measures, including funding schemes and cutting back on red tape……..”.

You can read more at: http://www.bbc.co.uk/news/business-17554053

& the official source: http://www.communities.gov.uk/news/corporate/2120114   

March 29, 2012

Ofcom publishes more detailed Broadband Availability data for England

DUG’s Data Manifesto (see February 6 below) includes the wish for data on Broadband coverage, & here is some news:

“Following a request from the UK governments Department for Culture, Media and Sport (DCMS), Ofcom, the communications regulator, has today published some of its “early” 2011 Infrastructure Report broadband data at a “more detailed geographical level“. It’s for England, and at Local Authority level. Here’s the link:

And the Ofcom (July 2011) report is here:

March 23, 2012

Output Area Classification (OAC) for 2011 – User Consultation – DUG's Response

ONS and University College London (UCL) have been consulting on plans for a new UK Output Area Classification:


The original classification was created for ONS by Dr Daniel Vickers at the University of Leeds using 2001 Census data (it won DUG’s first Award in 2005). Our response to ONS’s consultation is here

March 19, 2012

"Open Data – resist Sir Humphrey’s last stand" – article by Michael Cross

Michael Cross, founder of the Guardian’s “Free Our Data” campaign (& winner of our Award in 2006), is continuing the battle:

“You wait all year for a think tank to report on how government should embrace Big Data, then two do at once. Over the past week, both Demos and the Policy Exchange have published studies on the opportunities created by David Cameron’s pledge to create the world’s most transparent government.
One urges the government to get on with it; to create its promised “right to data” and let the private sector and concerned citizens do their best with it.
The other counsels caution - the transparency revolution will happen only when government departments are equipped with new IT systems to collect and disseminate accurate data, and civil servants and the general public educated to make sense of it.
Wonk-watchers will guess which philosophy emerged from which think tank. Anyone who has worked with government will guess which approach the civil service and wider public sector will find more congenial……..”

Read on at: http://www.thecommentator.com/article/988/open_data_resist_sir_humphrey_s_last_stand


March 19, 2012

Government response to Consultation on Public Data Corporation; also Terms of Reference for Data Strategy Board, Public Data Group, and the Open Data User Group (ODUG)

The Government has now published its response to ‘A Consultation on Data Policy for a Public Data Corporation'. The views that we expressed on 11 October (see below) were shared by many of the 412 respondents.

 The Government has also published the terms of reference for the new Data Strategy Board (DSB) and the Public Data Group (PDG); “This marks a significant step towards embedding a culture of transparency and better access to Government data”.

You can see details of both at:

The Cabinet Office is also setting up Open Data User Group (ODUG). “This Group will give data users and re-users an opportunity to influence the release of Government data”: http://www.cabinetoffice.gov.uk/content/open-data-user-group  

All this will require continuing effort on our part, but it’s good to see that the tide is still flowing strongly in the direction of more open / free data.

March 14, 2012

More free data, signposted by Data4nr

The latest update from http://www.data4nr.net/introduction/  includes business stats for small areas; various health indictors; and road distances to food shops, GPs, Post Offices, etc.


March 14, 2012

2011 Census Evaluation

Here’s a link to information about ONS’s evaluation programme in general which also links to a download about internet completion take-up:


And another relates more to the 2011 Census campaign itself, including an insight into how ONS selected the languages for the translation leaflets:



March 7, 2012

Francis Maude speech to Information Commissioner's conference

Francis Maude gave a speech to the Information Commissioner’s conference yesterday, 6 March 2012. The draft text is on the Cabinet Office web site at:

He spoke on: Transparency; Data Sharing; Fraud, Debt and Error; Identity Assurance; Census; and Privacy. In particular:

Data Sharing:
“In May we will publish proposals that will make data sharing easier – and, in particular, we will revisit the recommendations of the Walport-Thomas Review that would make it easier for legitimate requests for data sharing to be agreed with a view to considering their implementation.”

“And the census is another area where I want to bust the myths around the complexities of data sharing.
I find it incredible that we haven’t fundamentally changed the way we approach the census since it began in 1801.
We are currently supporting the work by the UK Statistics Authority to look at census alternatives that will be cheaper and easier for the customer without compromising the privacy of the information. Technology has a critical part to play in enabling this.”

“I want us to reach a position where the citizen is driving the disclosure of their personal data wherever possible – with effective mechanisms so they can truly decide what they want to consent to and be able to change their mind.”


March 3, 2012

ONS's 2011 Census Newsletter – February 2012

Here is ONS’s latest Census Newsletter. You’ll see that it includes items on Outputs, Analysis, Web Data Access (including an unexpected mention of the Retail Sales Index) an application programming interface, and also some Roadshows in April:

 “ONS will hold a series of roadshows in April 2012 to keep stakeholders informed of progress to deliver the 2011 Census outputs and the planned web data access channels. There will be a demonstration of the planned tools and how these will help users to access and exploit ONS data. There will also be a workshop session to seek your feedback about the opportunities and challenges presented by the planned web dissemination tools. Further information about these events and how you can register, will be issued in March 2012.

Locations & Dates:
• Manchester 17 April
• York 18 April
• London 19 April
• Birmingham 25 April
• Newport 26 April
• London 27 April”


March 2, 2012

Press Coverage of ONS's "Measuring Well-Being"

Wednesday’s papers gave extensive coverage to ONS’s efforts to measure well-being, for example:

“This must be a heartening week for those wallowing in self-pity. Suddenly everyone seems to care how happy we are. Just as a campaign was launched yesterday to dissuade urban elderly people from retiring to the countryside, lest among other horrors they find themselves with neighbours resembling a cross between Nick Griffin and Margo Leadbetter, the Office for National Statistics also published a comprehensive report on who the happiest people in Britain are, and where they live”

Guardian: http://www.guardian.co.uk/commentisfree/2012/feb/29/ons-happy-scots-northern-irish

& the BBC: http://www.bbc.co.uk/news/uk-17192780

You can find ONS’s detailed report as a pdf at:


February 18, 2012

"The Changing face of poverty" – The Daily Mail publicises Charles Booth's famous map of London

A DUG member (who wishes to remain anonymous) tipped me off about an article in the Daily Mail:

“The changing face of poverty: Updated maps of London's poorest areas show epidemic of 'junk food' diabetes in same streets where Victorians died of malnutrition”:

It features Charles Booth’s famous map, which has been the regular Award at our annual Conferences
If you are interested in Booth’s maps, there’s more at the LSE’s Online Archive: http://booth.lse.ac.uk/


February 6, 2012

DUG’s Data Manifesto

Here is a manifesto of DUG members’ needs for data from government, which has been prepared for the UK Location Programme.


February 4, 2012

Tim Kelsey appointed as the new Executive Director for Transparency and Open Data

The Cabinet Office has announced the appointment of Tim Kelsey as the new Executive Director for Transparency and Open Data within its Efficiency and Reform Group (ERG).

Tim gave the Keynote address “Open Data: Making it Real” at our Conference in October: http://www.demographicsusergroup.co.uk/resources/Tim+Kelsey+DUG+2011.pdf  


February 4, 2012

Job Vacancy – Boots

The latest job advertisement – Boots in Nottingham seeking a Portfolio Analyst – can be seen at the tab to the left.


February 4, 2012

National Statistical Offices: Independent, Identical, Simultaneous Actions Thousands of Miles Apart – a view from the USA

The Director of the US Census Bureau has written about initiatives that are common across Anglophone statistical offices around the world:

1. The volume of data generated outside the government statistical systems is increasing much faster than the volume of data collected by the statistical systems; almost all of these data are digitized in electronic files.
2. As this occurs, the leaders expect that relative cost, timeliness, and effectiveness of traditional survey and census approaches of the agencies may become less attractive.
3. Blending together multiple available data sources (administrative and other records) with traditional surveys and censuses (using paper, internet, telephone, face-to-face interviewing) to create high quality, timely statistics that tell a coherent story of economic, social and environmental progress must become a major focus of central government statistical agencies.
4. This requires efficient record linkage capabilities, the building of master universe frames that act as core infrastructure to the blending of data sources, and the use of modern statistical modeling to combine data sources with highest accuracy.
5. Agencies will need to develop the analytical and communication capabilities to distil insights from more integrated views of the world and impart a stronger systems view across government and private sector information.
6. There are growing demands from researchers and policy-related organizations to analyze the micro-data collected by the agencies, to extract more information from the data.”

You can see the full article at: http://blogs.census.gov/directorsblog/2012/02/national-statistical-offices-independent-identical-simultaneous-actions-thousands-of-miles-apart.html


February 2, 2012

Land Registry releases data free of charge

Following the mention of Land Registry Data in the Autumn Statement (see December 6 below), here’s some more news:

“The UK Land Registry as of 30 January is giving access to their monthly transaction data free of charge. This is to be followed by the release of 'Price Paid' data in March 2012”.

There’s more detail at: http://epsiplatform.eu/content/uk-land-registry-releases-data-free-charge  


February 2, 2012

Cabinet Office Open Data consultation – summary of responses

Following our submission to the Open Data consultation (see October 11 below), the Cabinet Office has now published a summary of responses:

“There was a very positive response to the consultation. Launched in August, there were nearly 500 responses submitted before its closure in October.

The consultation set out a series of questions aimed at informing Government’s strategy to embed transparency and open data as core operating principles of the public services. Many of you expressed your support for the principles of open data, your confidence in its ability to bring about a positive change and the benefits it can deliver, including enhancing the performance and delivery of public services and fostering economic growth.”

In short, encouraging stuff.

You can read the detail at:


January 29, 2012

Living Costs and Food Survey – Users' meeting – Tuesday 20 March 2012

Various DUG members are interested in the ONS’s Living Costs and Food Survey (formerly the Expenditure & Food Survey), which is now coded with ONS’s OAC geodemographic classification.

Here’s news of a meeting:
“…….organised by the Economic and Social Data Service, which will provide a forum for data users and producers to meet and discuss new developments and exchange information about the Living Costs and Food Survey. The programme contains a mixture of papers from data producers and researchers. The meeting is free to attend and lunch is provided.

To view the programme and book a place please go to www.ccsr.ac.uk/esds/events/2012-03-20/  ”


January 24 , 2012

Eurostat's new vision for European Statistics

The National Statistician organised a meeting on 18 January which featured Walter Radermacher, Chief Statistician of the EU, talking about “A new vision for European Statistics” (here) to a large audience from ONS and departments such as DWP, HMRC, etc.

Keith Dugmore was invited to give the user view on the panel session, & had a chance to:

• Warn of the danger that EU officials might get an inside track on priorities in the same way as Government Departments do in the UK.
• Welcome the focus on making more use of admin data, better dissemination, and the importance of political support for Open Data in changing culture.
• Make the case for more stats at small area level, plus the necessary boundaries & mapping being freely available.


January 17 , 2012

Mapping of Urban Sprawl using open data from www.data.gov.uk

Blair Freebairn is exploring Open Data, generating new urban sprawl boundaries.

Here is a presentation which defines the current problem, shows what he’s done & why it’s better, and gives some examples.

If you would like more information, email blair.freebairn@geolytix.co.uk  


January 17 , 2012

ONS's "Beyond 2011" User Requirements Consultation – DUG’s Response

Following the earlier news item (October 27) here is DUG’s response to ONS.


January 11 , 2012

Sub-regional public and private sector employment in the UK

Here’s a new article on the rarely-discussed topic of analysing public and private sector employment in different sub-regions of the UK.
“The level of public sector employment in the UK economy has increasingly become a policy issue over recent years. To help inform the debate, ONS produces quarterly public sector employment data that measure the total level of public sector employment at the national and regional level. At present, however, there have been relatively few data releases or articles examining the sub-regional distribution of public sector employment. Furthermore, this distribution is of interest to policy makers both nationally and locally. This article seeks to fill this gap by providing an overview of some of the data available from ONS that can be used to examine the sub-regional distribution of public sector employment and also the sub-regional distribution of private sector employment.”


January 8, 2012

Letter to the Rt. Hon. Francis Maude MP - "The National Address Gazetteer: being run in the interests of Ordnance Survey, to the detriment of citizens and the private sector"

You can see this letter from DUG to Francis Maude here

In particular:
“...........For anyone outside the public sector, the prices for GB coverage and 100+ users range between £130,000 p.a. for the basic version to £189,000 p.a. for the full product. Even a single user would have to pay £24,000 p.a. These prices are prohibitive for almost all potential customers, and appear designed to protect OS’s existing policy of setting high prices for a small captive market.”


January 7, 2012

ONS is seeking users' involvement in designing its website for delivering 2011 Census outputs

ONS is currently developing its website to provide access to 2011 Census outputs, which are planned to start in July. It is seeking users’ views during January & February.

If you would like to learn more, please contact julie.peters@ons.gsi.gov.uk before the end of January.


January 6, 2012

RSS Open Meeting "Measuring Well-Being" - 8th March 2012, 5PM- c.7PM

Several DUG members have expressed interest in this topic: here’s a chance to catch up on developments at a 2-hour meeting after work – see the details here


January 5, 2012

Job Vacancies – Sainsbury's (closing date 22 January 2012)

Here's the latest job advertisement, with Sainsbury’s seeking Senior Location Analysts:

Earlier Job Vacancy announcements can also be seen at the tab to the left.


January 2, 2012

Happy New Year........... & here's a summary of our all activities in 2011

Looking back on the past year, you’ll find a summary of DUG’s activities during 2011 here