Welcome to the Demographics User Group

Representing the needs of commercial users of information

Get alerted to future DUG news stories by entering your email address in this box....

DUG: More News

November 19, 2014

DUG Members’ needs for data from government – our Data Manifesto – latest update

DUG’s Data Manifesto, which was initiated by Mike Whitelegge at M&S, continues to make an impact in Whitehall, and is proving a very useful summary wishlist / reference document.

Here is the latest version, which you might find useful if you are asked “What more government data would you like to see made freely available?” 

November 18, 2014

Ordnance Survey accused of stifling competition in open data row

Here’s yet more criticism of Ordnance Survey…….

“UK mapping agency Ordnance Survey stands accused of using £800m of government contracts to stifle competition in a row over the release of geographical information as open data. Geo-data firm Getmapping claimed Ordnance Survey received illegal state aid in a complaint it lodged at the European Commission (EC) last Friday (7 November 2014).”

“Getmapping claimed Ordnance Survey used two 10-year government contracts worth £800m to stifle smaller, private geographical intelligence firms. The complaint seeks to make Ordnance Survey curtail its ambitions so private companies can prosper in its stead. Getmapping said the Department for Communities and Local Government (DCLG) let the contracts wrongfully in 2010 and 2011, because it didn't put them to public tender.”

"Ordnance Survey has been granted these contracts, which has made it highly profitable – much more profitable than most industry companies. It is using its state aid to compete with the rest of the partner network. Ordnance Survey is a serial abuser of its partners. And the government knows it."

You can read more here: http://www.computerweekly.com/news/2240234712/Ordnance-Survey-accused-of-stifling-competition-in-open-data-row  

November 4, 2014

UK Statistics Authority publishes a new Statement of Strategy – including the importance of business users

Following the talk by the National Statistician, John Pullinger, at DUG’s Conference http://www.demographicsusergroup.co.uk/24.html  , here is the UK Statistics Authority’s new Statement of Strategy, entitled “Better Statistics, Better Decisions”.

It’s good to see the emphasis on the importance of businesses as users of stats, e.g. “Decision-makers across the spectrum – individuals, businesses and government”.

There are also several mentions of the UKSA seeking to be “helpful” to its customers – which is good to see, setting the expectation. 

October 25, 2014

The DUG Conference “Can sharing data within and between organisations create social and economic benefits?” – and our annual Award

At our annual conference on 8 October, the winner of The Demographics User Group Award 2014 for “Better information from government” was…………

The Office for National Statistics

for its active engagement with private sector users when carrying out the most successful Census in recent decades

You’ll find the citation, ONS’s Press Release, and a photo, together with the conference presentations, at the tab on the left.

October 24, 2014

Supermarket locations, free from Geolytix

Our friends at Geolytix have produced another open (free) dataset…..a list of supermarket locations.

“There is no commercially sensitive information attached to the stores, just the store name, and an address from the retailers store locator, our GeoLytix town and locality names, and an exact rooftop location (in British National Grid and web friendly WGS84 lat/long). Yes, we have probably spent more time directing ourselves around the country in google street view than the google van spent capturing the imagery (and yes, we have manually located each of the 6,872 stores with the aim to get a really good dataset).

Current retailers include Aldi, Asda, Booths, Budgens, Marks and Spencer, Morrisons, Sainsbury’s, Tesco, Waitrose, Wholefoods. Lidl and Co-op are coming soon.”

Here’s the link: http://geolytix.co.uk/blog/?p=238


October 20, 2014

Retail Research and Data Master's Dissertations – call for projects now, to commence in 2015

Many of you will be aware of the opportunity for companies to propose projects for Master’s students that we have supported in the last three years.

New funding from the ESRC is making it possible to run this is the longer term, and to start earlier (ideally this month & next), to maximise the chance of getting the pick of the best students for 2015.

Here’s more detail from Guy Lansley at UCL, who is running the scheme this year.


Please start thinking about possible projects now, and don’t hesitate to get in contact with Guy to sound out your ideas.


October 14, 2014

Book Launch: London: The Information Capital – Friday, 31 October 2014 from 18:30 to 20:30

If you are interested in analysing, understanding, and mapping London, this will be right up your street.
Here below is a message from Dr James Cheshire of UCL about his book, and its launch – a great opportunity to meet up with young researchers who are exploring and visualising free open data for London.

“A lot of the discussion points – data hacking, visualisation, open data, user privacy, new datasources – are topics covered in my new book entitled London: The Information Capital (http://theinformationcapital.com/  ). The general theme throughout the book is that London is world leading in its data availability since it benefits from the likes of the 2011 Census as well as TfL data and large numbers of social media users. We visualise all these things alongside some previously unseen data such as Londoner's dating preferences (from Lovestruck) and what the police helicopters get up to on any given day (thanks to surprisingly successful FOI request).

It will be published on the 30th October and available in all good bookshops (as they say). Its RRP is £25 but the Guardian and Amazon currently have it heavily discounted at the moment (£15 and £13.60 respectively).

I am also hosting a launch event at UCL on the 31st October to which all are welcome. Sign up here: http://bit.ly/1w0bdih  
We hope the book will add to the growing momentum behind open data and to help London lead by example.



October 14, 2014

Census Research User Conference 2014 – 31 October 2014, Royal Statistical Society, London

The UK Data Service is holding a one-day conference to provide users and data creators with an opportunity to meet and discuss developments in census research.

The day will be composed of presentations from census agencies on developments in data and from users who have used the data in their research. The presentations will be based on analysis of the 2011 and earlier census products including aggregate tables, flow data and microdata

The conference will be followed by the Enhancing and Enriching Historic Census Microdata (EEHCM) event to launch new census products.

This is pitched at academic users, but might provide an opportunity to develop new ideas and make contact with census specialists.

You’ll find more details, and can book here:


September 27, 2014

ONS’s new web page – "Get involved with Census Analysis"

ONS has announced a new Census webpage, which includes details of planned and previous user events, a link to the StatsUserNet census discussion forum [good stuff for Census anoraks], and news of tools and what others have done with 2011 Census data. Here’s the link:



August 22, 2014

DUG's 11th Annual Conference: “Can sharing data within and between organisations create social and economic benefits?” – Wednesday 8 October 2014, at the Royal Society

This year our topic is “Can sharing data within and between organisations create social and economic benefits?”, and we will again have more than 100 delegates, enabled by financial support from the Economic and Social Research Council.

The topic has been prompted by discussion amongst DUG members, and meetings with the Cabinet Office, ONS (planning Beyond 2011), and the ESRC (with its big investments). This seems an ideal time to debate an increasingly hot issue. You’ll see on the provisional programme that our speakers include the National Statistician John Pullinger, and a there’s workshop to discuss public acceptability, future opportunities, and challenges and how to overcome them.

As always, the purpose of the conference is to bring together people from DUG's 16 member companies, and also specially invited guests in government and universities, to spread knowledge and stimulate new ideas. This is DUG’s eleventh annual conference, and we are convinced that increased dialogue brings enormous benefits to all parties.
As in previous years, the proceedings will be posted on this website.


July 25, 2014

"Big Data and Location": an AGI event in partnership with DUG – 30 September, IBM South Bank, London

The Association for Geographic Information has been in touch, seeking DUG’s involvement in its conference on “Big Data and Location”, and we are very pleased to partner in this event, which we think will be of real interest to many DUG members involved in geographical analysis. It will feature Mike Whitelegge of M&S as both a speaker and chair, and the programme also includes speakers from IBM, Deloitte, Cloudera, MapR, CapGemini, Esri and Ordnance Survey together with exhibitor stands from a range of suppliers.

Here’s the programme: http://www.geobig5.com/files/6114/0569/9029/ProgrammeLondonv1.pdf

You can book at http://www.geobig5.com/events/big-data/

July 28, 2014

Francis Maude confirms plans for the next Census in 2021 – and DUG is quoted in the Financial Times

The Government’s response to the Public Administration Select Committee’s report (see DUG News April 21) has just been published, and includes a letter from Francis Maude, confirming that the government will back the National Statistician’s recommendation that there should be a census in 2021, conducted at least partly online.

Mr Maude reiterated his view that the census is “outdated” and could be delivered more effectively and more cheaply, but said he accepted the 2021 Census should go ahead, alongside use of existing data and surveys. But he added: “It is the government’s ambition that beyond 2021 the decennial census would not be undertaken, instead more regular and timely administrative data would be used to produce statistics.”

You can see the report here: http://www.publications.parliament.uk/pa/cm201415/cmselect/cmpubadm/601/601.pdf

There’s a good article in the Financial Times (behind its paywall) by Emily Cadman, which includes:

“Keith Dugmore, who organises the Demographic Users Group (DUG), a group of some of the UK’s biggest employers in consumer-facing businesses, including Barclays, Boots, John Lewis and Sainsbury’s, said the conclusion was the right one.

The group had expressed grave concern about the option of moving immediately to solely administrative data, warning that any loss of data of very small geographic areas would be “catastrophic” for business users.

Mr Dugmore said that while “we don’t want to be lumbered with a traditional census forever” there was a “sense of relief” that the traditional census will be carried out again while research is done to see whether administrative sources could provide a true alternative.”

July 25, 2014

"Harnessing Open Data for Business Advantage" – MRS Census Geodemographics Group seminar – 5th November

This seminar will focus on how users have been applying census and other open data, with an emphasis on ‘how to’ case studies and useful lessons learnt.

You’ll see that there is a fine array of speakers, and that there is a discount if you book before 6 October: https://www.mrs.org.uk/event/course/2436

July 22, 2014

New, free, geodemographics – ONS releases the 2011 Output Area Classification (OAC) for the UK

ONS has announced the release of the new version of OAC:

We also have:
• Interactive maps, produced by UCL: http://public.cdrc.ac.uk/

• A new website for the Output Area Classification User Group (OACUG): https://plus.google.com/u/0/communities/111157299976084744069  

July 22, 2014

Companies House to make all of its digital data available free of charge

Whilst not many DUG members use Companies House data about businesses, here’s some encouraging news about further moves to make government data open and free.

“Companies House is to make all of its digital data available free of charge. This will make the UK the first country to establish a truly open register of business information.
As a result, it will be easier for businesses and members of the public to research and scrutinise the activities and ownership of companies and connected individuals. Last year (2013/14), customers searching the Companies House website spent £8.7 million accessing company information on the register.
This is a considerable step forward in improving corporate transparency; a key strand of the G8 declaration at the Lough Erne summit in 2013. It will also open up opportunities for entrepreneurs to come up with innovative ways of using the information……….”.

You can read more at: http://www.companieshouse.gov.uk/freeDataPressNotice/

July 16, 2014

New Geodemographic Classification Systems – a review by Peter Sleight

Peter Sleight, of the Target Marketing Consultancy, continues to provide very useful summaries of the competing geodemographic classifications – Mosaic, Acorn, OAC, and several others.

Here’s his latest:

July 14, 2014

UKSA Independent Assessment of the 2011 Census – DUG's Response

DUG has submitted this response to the UK Statistics Authority.


July 12, 2014

Land Registry privatisation vetoed by Vince Cable

There’s more news in today’s Guardian about Whitehall infighting over the government agencies and their data:

“Plans to privatise the Land Registry have been abandoned by the government after business secretary Vince Cable vetoed the scheme following a series of rows between Tory and Liberal Democrat ministers.
It had been hoped that a sell-off would raise at least £1.2 bn for the Treasury, with ministers deciding on the appropriate structure, such as creating a deal with a joint venture company.
But this week, ministers will tell parliament that the planned move has been suspended indefinitely. The Conservatives had been in favour of a partial sell-off via the creation of a joint venture.”

Read all about it here:

July 11, 2014

“How UK businesses are securing and managing talent to grow in a big data world” – a new report

Here’s news of a report on a hot topic for many DUG members. It’s authored by Nesta (an innovation charity http://www.nesta.org.uk ) in association with the Royal Statistical Society, which invited DUG to be represented on the panel at a launch on Wednesday 9 July:

June 28, 2014

New website "Datashine" – detailed and fast mapping of Census statistics across England & Wales

Following on from the news of “Maps of 2011 Workplace Zones in London” (25 June), here’s another initiative from UCL, this time providing online mapping of a wide range of Census variables at detailed level across the whole of England & Wales.

Here’s the link: http://blog.datashine.org.uk/

Warning: once you start playing, you might find it difficult to stop….

June 25, 2014

Maps of 2011 Workplace Zones in London – See the local differences in hours worked. education, transport, health, etc.

A PhD research student, Robin Edwards, at UCL’s Centre for Advanced Spatial Analysis (CASA) has been very quick – days after the release of the statistics – to publish a series of maps of c.10 variables by 2011 workplace zones in London:


It’s fascinating stuff – and may well lead to you further digging into the data.

June 17, 2014

Output Area Classification (OAC) – Its current use on government surveys, and the launch of OAC 2011

Many of you will be familiar with ONS’s (free) geodemographic Output Area Classification (OAC). If not, you can find out about it at http://areaclassification.org.uk/

We have learned from ONS that the following government surveys are now coded with OAC:

1. Annual Population Survey (APS) (which is based on waves from the Labour Force Survey)
2. Crime Survey for England and Wales (CSEW)
3. Dental Health Survey of Children and Young People (CDH)
4. Household Assets Survey (HAS) – formerly the Wealth and Assets Survey (WAS)
5. Integrated Household Survey (IHS)
6. Living Costs and Food Survey (LCF)

The new 2011-based version of OAC is expected to be launched next month.

June 13, 2014

New file of Census data at Output Area level for the whole UK

We are pleased to report that ONS has now produced a single file of statistics at Output Area level, spanning all four countries of the UK. (Experience in 2001 showed that it is so much better for the Census Offices to do this at source, rather than multiple users all try to sort out the sometimes varying definitions themselves). Here’s the announcement from ONS:

Today, 13 June 2014, ONS has made available the Key and Quick Statistics tables at Output Area (OA) level, for all Output Areas in the UK. This combines information previously published separately by the census office responsible for the conduct of the census in each constituent country of the UK.

The tables are equivalent to the local authority level UK data released in three parts in October / December 2012 and January 2014. Between them, those three releases provided all Key and Quick Statistics that were harmonisable across England and Wales, Scotland, and Northern Ireland.

The tables – 26 Key Statistics and 47 Quick Statistics – are provided now for all Output Areas in the UK, in a simple csv form suitable for additional processing by users. The data is accompanied by supporting information that explains the steps taken to merge categories in some tables to produce a UK harmonised table, and further detailed explanations and definitions of terms used in tables. To aid users in re-aggregating the OA data administrative and other common geographies, a look-up file that relates all OAs to every standard higher level geography in each country is also included.

All of the data is available to download in a single zip file from:



June 7, 2014

2011 Census – New (free) postcode directories & lookups released by ONS

The May 2014 editions of ONS’s Postcode Directory (ONSPD) and National Statistics Postcode Lookup (NSPL) have been released on the ONS Geography portal:



May 23, 2014

Workplace Statistics from the 2011 Census published today

Here’s news from ONS of the publication of counts and detailed classifications of workers (so vital when planning store locations) for Workplace Zones in England and Wales.

The workplace population of an area is defined as “all usual residents aged 16 and above who are in employment and whose workplace is in the area”. People who work mainly at or from home or do not have a fixed place of work are included in the area of their usual residence.

This definition focuses on only those people who are working, and the statistics are for a new “Workplace Zone” geography, which is particularly valuable for targeting very small areas within city centres. The statistics cover a wide range of topics. You can find more information and download the data at:


May 23, 2014

Government surveys on Income, Expenditure, & Wealth – User Conference – 7 July

This free full-day meeting in London, organised by the UK Data Service and ONS, will provide a forum for the exchange of information and views between users and producers of the following surveys:

• Family Resources Survey
• Living Costs and Food Survey
• Wealth and Assets Survey

The programme contains a mixture of papers from data producers and researchers.

To view the programme and book a place please go to http://ukdataservice.ac.uk/news-and-events/eventsitem/?id=3749  


May 14, 2014

John Pullinger appointed as the new National Statistician

We are very pleased to report that John Pullinger is to be the new National Statistician, succeeding Jil Matheson. The UK Statistics Authority’s News Release is here.

John has always emphasised the importance of the users of statistics, being very supportive of DUG over many years, and has been a regular guest at our annual Conference.

April 21, 2014

House of Commons Select Committee Report: "Census should not be scrapped"

Following our earlier news (see March 1 below) about the Public Administration Select Committee undertaking a study on the Future of the Census, it has now reported, urging the Government not to scrap the 2021 Census, but also promoting more use of data from administrative sources:

“Good figures on the people in the country are of fundamental importance to the statistical system, policy makers and society more widely, and the ten-yearly census gives detailed information on small areas. The Report follows the National Statistician’s announcement in March 2014 that she recommends that Government keep the census in 2021, but that it should be conducted largely online, and that the Government should make much greater use of the data which it already holds in order to improve the accuracy of population estimates. The Committee supports the recommendation from the National Statistician, but urges the Office for National Statistics to do much more to make the best use of the data which the Government already collects, for example through the Department for Work and Pensions, HM Revenue and Customs and the Department of Health. The Committee says that the Office for National Statistics’ work on the future of the census has, to date, been limited, and recommends that the Office for National Statistics now sets out a much more ambitious vision for the use of this data to provide rich and valuable population statistics.”

Bernard Jenkin MP, Chair of the Committee, said:

“The Government has previously indicated that the ten-yearly census should be axed, but we believe that it is too soon to decide whether or not to scrap the census. We think there should still be a census in 2021. The Government should make far greater use of the data it already holds in order to improve population estimates. We urge the Government to address public concerns about data sharing and to learn the lessons from the flawed rollout of the “care.data” programme, so that the best use can be made of the information already collected and held by the Government. The Government’s objective of “better, quicker information, more frequently and cheaper” depends upon it.”

You can see the report at: http://www.publications.parliament.uk/pa/cm201314/cmselect/cmpubadm/1090/109002.htm

It’s good to see in the Summary: “Witnesses emphasised, among other things, the great financial benefits to business provided by census data and these have been quantified.” 


April 15, 2014

Big Data – Open/Free data available for use by businesses – Conference organised by the Research Councils & Government

“This unique day aims to remove the barriers for companies to access this open data, revealing how to extract value and insight for the benefit of your company- of all sizes and sectors. You will be equipped with expert guidance on how to take advantage of the data sets on offer, as well as introduced to enabling funding schemes and organisations that can help you to take raw data and make it into something valuable.”

This sounds right up our street.
Time: Tuesday, 06 May 2014, 09:00 - 17:00
Place: Science & Technology Facilities Council, Rutherford Appleton Laboratory, Didcot, Oxford OX11 0QX

More details, & booking: http://biginnovationcentre.com/Events/133/Big-Data-Week-UK-big-data-open-data-available-for-business

April 12, 2014

UK Data Service makes 2011 Census statistics available to all

Here’s some good news from the ESRC’s UK Data Service:

“InFuse provides both academic and non-academic researchers with access to 2001 (England and Wales) and 2011 (UK) Census aggregate statistics under the Open Government Licence. This means there's no need to register or login to extract the data.

InFuse makes it quick and easy for the user to find and download information for places of interest across the UK and it is free of charge.”

You can read all about it at: http://census.ukdataservice.ac.uk/news-and-events/newsitem/?id=3761

This should be an important step in opening up data held by the academic community to wider access. 

March 28, 2014

National Statistician’s recommendation on the future of the Census and population statistics in England and Wales

Many of you will be aware that DUG has been very closely involved with ONS in its assessment of possible options for the next Census “Beyond 2011”.

We also responded to the formal consultation late last year, and we now have news of Jil Matheson’s recommendation to government http://www.ons.gov.uk/ons/about-ons/who-ons-are/programmes-and-projects/beyond-2011/beyond-2011-report-on-autumn-2013-consultation--and-recommendations/index.html
• An online census of all households and communal establishments in England and Wales in 2021 as a modern successor to the traditional, paper-based decennial census, combined with…..
• Increased use of administrative data and surveys in order to enhance the statistics from the 2021 Census and improve annual statistics between censuses.

This is a good result – but only so far, because Treasury has to accept the cost-benefit analysis, and Government has to approve the recommendation.

DUG was quoted in the Financial Times (apologies that you might have to register to view it):
http://www.ft.com/cms/s/0/907c1262-b598-11e3-a1bd-00144feabdc0.html#axzz2xMJK3EMs _____________________________________

March 18, 2014

PASC report on Statistics and Open Data – "The sale of PAF with the Royal Mail was a mistake"

Hard on the heels of the message about BIS’s report on the National Address Gazetteer, the Public Administration Select Committee (PASC) has published its report on Statistics and Open Data, which gives more support to DUG’s cause:

“In a report on statistics and open data published today, the Commons Public Administration Select Committee (PASC) says the benefits – potentially running into billions of pounds for the economy and society as a whole – of harvesting the masses of unused knowledge and information that Government collects outweigh the risks to privacy, and that there must be a “presumption of openness” with public data.”

“Some government data sets are of huge direct value to the economy. The Committee says that the Postcode Address File (PAF) was included in the sale of Royal Mail to boost the share price at flotation: this move took an immediate but narrow view of the value of such datasets. PAF should have been retained as a public data set, as a national asset.”

(We sent our submission to PASC on 5 September, urging it to press for the National Address Gazetteer and the Postcode Address File to be made Open Data).

You can see further news, and also download the report here:
There has also been good coverage in the media, e.g.
Guardian: “MPs and open-data advocates slam postcode selloff” (with KD being quoted) http://www.theguardian.com/technology/2014/mar/17/mps-and-open-data-advocates-slam-postcode-selloff
BBC: http://www.bbc.co.uk/news/business-26605375  
Such comments will help the Cabinet Office’s cause in its continuing battle with BIS and the Treasury.

March 15, 2014

An open (free) national address gazetteer? DUG's letter to BIS

Following the news on February 22, here is DUG’s letter to BIS.
You’ll see that we are continuing to make the case that the National Address Gazetteer should be free open data (but reluctantly see the proposed Freemium model as a step towards the ultimate goal).

March 2, 2014

Passenger statistics for every railway station in Britain: April 2012 – March 2013

Here are some new trainspotting stats from the Office of Rail Regulation:

• London Waterloo remains the most heavily used station in Britain clocking nearly 96 million passenger entries and exits last year – an increase of 1.8m over 2011-12.
• The number of people using stations across Britain in 2012-13 increased by over 82m – a 3.3% rise compared to the year before.
• The list of top 10 stations has remained unchanged since 2011-12 with eight of the 10 stations located in London.
• All the top 10 stations reported a rise in usage with the largest increase at London Euston, which saw an increase of 1.8m entries and exits – due to the increasing demand for London Overground services.

If this is of interest, you might want to put on your anorak and bobble hat before you open the link below to scroll through estimates of passengers for every station in Britain:

March 1, 2014

House of Commons Select Committee on the Future of the Census – DUG representation

The Public Administration Select Committee is undertaking a study on the Future of the Census, and an oral evidence session was held at the House of Commons on Tuesday 25 February.

Keith Dugmore spoke on behalf of DUG, and the following is now on the parliamentary website:
Video: http://www.parliamentlive.tv/Main/Player.aspx?meetingId=14944

Transcript: http://www.parliament.uk/business/committees/committees-a-z/commons-select/public-administration-select-committee/inquiries/parliament-2010/statistics/future-of-the-census/

We also followed up with this submission.

February 22, 2014

An open (free) national address gazetteer? Here's an independent report for government on its feasibility

With our long campaign for a definitive and free National Address Gazetteer in mind, if you are an addressing anorak, please read on…..

The Department of Business, Innovation & Skills has now published an independent report by Katalysis:

It recommends that a basic address product should be free to all users at the point of use under the Open Government Licence while premium versions would still be sold, leaving current production and maintenance roles much as they are today. Whether the government will act on this is not known, but the report is certainly valuable in its summary of the current state of play regarding address files and their ownership, and in setting out future options.

February 11, 2014

Access to data from the Valuation Office Agency

The VOA for England and Wales collects some valuable data, including:

Prices paid on sale or transfer of properties
VOA Council Tax – addresses, characteristics and attributes of properties
VOA Residential Private Rented Sector (PRS) rents

Non domestic rating:
• Rents paid on letting of properties
• Receipts/expenditure and costs of construction
• Property addresses, floor areas, characteristics and attributes

The recently-appointed Director of Information and Analysis gave a really good summary presentation of the current situation, and future plans to make more data available, at an HMRC meeting in October, and this is now public at:


February 9, 2014

Land Registry tipped for part-privatisation – Public Consultation

Several DUG members use information from the Land Registry, which is responsible for keeping and maintaining details of the ownership of land and property in England and Wales, and is a source of data on house prices.

The above disconcerting news appeared in both the Daily Telegraph:
and also the Guardian:

The Government announcement suggests we won’t lose the supply of free data, with reassuring words such as “maximising the reuse of property data for the benefit of the economy”, but if Land Registry data is important to you, you might like to have a close look at the consultation document:

The consultation will close on 20 March 2014. 


February 6, 2014

Scotland's Census 2011 – Further news of outputs

Many of you will be aware that 2011 Census outputs for Scotland have lagged behind those for England & Wales, and Northern Ireland.

We are pleased to report that, since the news (see below, 5 July) about population estimates and household estimates by Postcodes & Output Areas, and geography products, including OA boundaries and Postcode to OA look-up files, various statistics at OA level have now been published.

These are summarised in the latest Outputs Prospectus http://www.scotlandscensus.gov.uk/en/censusresults/outputsprospectus.html  (see page 4), which also includes news of plans for further releases.

January 27, 2014

Retailers: Do you have projects now that Master's students can help you to progress this spring & summer? 

If you work for a UK retail company, this is an opportunity for you.

The ESRC’s “Retail Research & Data” initiative, which involved commercial companies proposing projects for Masters’ students, was a real success in both 2012 and 2013, and Professor Paul Longley & I are pleased to report that the ESRC has decided to support this initiative in the longer term.

We are therefore seeking proposals for new projects as soon as possible.

This presents a great opportunity to get a Master’s student to help you to make progress with:
• Major current issues, such as multi-channel marketing and customer insight, store networks, transport, social media, and many others.
• Projects that you’ve long had in mind, but not had the resource to carry out.
• Working with data – both your own customer data, and also maximising the value to be obtained from government initiatives such as www.data.gov.uk, and the arrival of more 2011 Census statistics (including Daytime and Workplace populations) for the whole UK.

If you have a project in mind, please email Keith Dugmore ASAP (ideally by this Friday 31 January) at dugmore@demographic.co.uk and I’ll send you a Project Proposal Form.

It’s essential to move quickly on this, to publicise projects early and get the choice of the best students.

Our previous experience illustrates the main features of the initiative:
• ESRC’s publicity for the 2013 scheme, including a list of the 11 projects that ran: http://www.esrc.ac.uk/collaboration/business-sector/retail/retail-masters/
• Presentations by two prize winners at the 2013 DUG Conference: http://www.demographicsusergroup.co.uk/23.html
• Presentations by 12 of the 16 students at the 2012 DUG Conference: http://www.demographicsusergroup.co.uk/22.html


January 14, 2014

Happy New Year........... & here's a summary of DUG’s many activities in 2013

Very best wishes for a successful and enjoyable 2014.

Looking back on the past year, we hope that you’ll find this summary of the activities of the Demographics User Group during 2013 both interesting and useful.

You’ll see that it’s been another busy and successful year, with members sharing their experience in numerous ways, pressing government for further progress in the policy of making more data freely available, and also deepening our links with the ESRC, which is increasing the priority given to retail research.


January 9, 2014

Land Registry open data: House price paid data (November 2013) – down to postcode level

The latest alert from Data4nr includes house price stats right down to postcode level:

“The monthly residential property price data, tracks all of the residential property sales in England and Wales that are lodged with the land for registration. The data includes: the full address of the property (Primary addressable object name (PAON)), where available, secondary addressable object name (SAON), street, postcode, locality (if available), town, district, county), the price paid for the property, the date of transfer, the property type (Detached, Semi, Terraced, Flat/Maisonette), whether the property is new build or not, whether the property is freehold or leasehold.”

Here’s the link: http://www.data4nr.net/resources/1674/


January 7, 2014

A message from Martin Callingham and Steven Bond: Keith Dugmore made an MBE

"We are delighted to announce that Keith Dugmore, creator of the Demographics User Group, was made an MBE in the New Year’s Honours List. The citation was:

MBE – Keith Dugmore, Founding Chair Statistics Users Forum – “For services to UK Business”

This is a great personal achievement for Keith who has for many years successfully brokered and nurtured dialogue between various agencies of government, academia and the private sector. The benefits of the greater closeness of business to both government and academia are already apparent and will directly impact on the UK's economy and competitiveness into the future.

Many of you receiving this note will have been directly involved in this process and will know that only Keith could have made this happen. We are sure you will join with us in congratulating him on this richly deserved honour.

Martin & Steve"

(DUG’s Non-Executive Directors)


January 7, 2014

Full UK Census Data Pack from GeoLytix

Following up their earlier news below (18 February last year) it’s good to see that Geolytix are making a UK-wide (NB, yes, including Scotland) Census Data Pack available free of charge:

“Today, the full 200 variable GeoLytix UK Census Open Data pack is available for immediate free download. http://www.geolytix.co.uk/geodata/census

This release includes the Scottish data for all variables. We have also made three additions to the dataset since the last release:
• Added the Household Social Grade Variables (AB/C1/C2/DE)
• Included our own GeoLytix Urban/Rural flag for each OA
• Split the White ethnic group into ‘White British’ and ’White Other’ "

There’s also a brief blog on this release: http://geolytix.co.uk/blog/ 


December 22, 2013

HMRC consultation on sharing and publishing data for public benefit – the Government response

Following our submission (see 30 September below), HMRC has now issued its response:


This is positive towards making HMRC data more easily accessible, and will provide good context for when discussing priorities for the release of more datasets.


November 30, 2013

Retail Research Data – A website to help retailers find free data easily – new release

“The primary purpose of Retail Research Data (RRD) website is to enable insight and store location analysts working in retail organisations to get easy access to free datasets which may be of value to their businesses.”

This new release includes more 2011 Census statistics, and improved mapping. It is part of the ESRC’s Retail Research and Data project.

Here’s the link: http://www.retailresearchdata.com/

November 25, 2013

New report from BIS: "Seizing the Data Opportunity"

Here’s news of a new report from the Department for Business, Innovation and Skills: "Seizing the Data Opportunity".
Universities and Science Minister David Willetts has announced “A new strategy will ensure the UK is well placed to take advantage of the opportunities for using data. Seizing the Data Opportunity sets out plans to create a skilled workforce in data analytics, develop the UK’s data infrastructure that will allow access to high-end computing, and help data to be shared and linked to support ground-breaking research”

The report focuses on some of DUG’s goals, especially to improve the data infrastructure, and to promote training.

You can find the press release, and a link to the report here: http://news.bis.gov.uk/Press-Releases/Strategy-positions-UK-as-world-leader-on-data-science-69543.aspx 

November 22, 2013

ONS Consultation: "The census and future provision of population statistics in England and Wales" – DUG response

Following the earlier news below (September 24), DUG’s response has now been submitted to ONS.


October 31, 2013

2011 Census – ONS's "Workday Population" statistics for Output Areas in England and Wales

ONS has just sent out an alert announcing the above good news.

We’ve been pushing for this as an alternative and supplement to Workplace statistics (which will be for a special Workplace Zone geography, rather than for our regular Output Areas).

Here’s the link: http://www.ons.gov.uk/ons/rel/census/2011-census/workday-population-statistics-for-output-areas-in-england-and-wales--part-1-/index.html

12 tables provide information on some of the key characteristics of the workday population of England and Wales such as population size, general health, religion and ethnic group at Output Area level.
Similar tables for Workplace Zones are expected in Q1 next year. 

October 24, 2013

The DUG Conference “Understanding millions of mobile consumers”

Our Awards: (1) "ESRC wins top accolade from UK businesses"; and (2) Professor Martin Callingham

At our annual conference on 9 October, DUG made two awards:

(1) The Economic and Social Research Council, “for its increasing focus on retail research, and developing training in quantitative skills”
The ESRC produced a fine Press Release. This year we also made a DUG Personal Award to……….

(2) Professor Martin Callingham, formerly of Whitbread, and now Birkbeck College London, “in recognition of his role in encouraging the formation of DUG in 1998, and his immense contribution since, including training, playing with data, and providing an endless stream of new ideas”.

Both awards are hugely deserved.

You can see the Conference presentations at the tab on the left. 


September 30, 2013

HMRC consultation on sharing and publishing data for public benefit – DUG's response

Following the announcement of HMRC’s consultation (see July 29, below), you’ll find DUG’s response here.


September 27, 2013

ONS's Census News – including Postcode products, and Origin / Destination stats

ONS's latest News Alert includes:

• Migration stats at Output Area level, available from the Nomis website: https://www.nomisweb.co.uk

• Updated Postcode to OA look-up table:

• Origin / Destination (including commuting) statistics – are due in Q1 next year. 


September 25, 2013

ESRC Retail Research & Data – A website to help retailers find free data easily – Test Release – Your comments are very welcome

“The primary purpose of Retail Research Data (RRD) website is to enable insight and store location analysts working in retail organisations to get easy access to free datasets which may be of value to their businesses.”

Here’s the link: http://www.retailresearchdata.com/

This is part of the ESRC’s Retail Research and Data project. The first release was made available earlier this year. This new test release now includes information from the 2011 Census, the first release of the new Output Area Classification (OAC), and improved mapping.

It would be really helpful to get your feedback to dugmore@demographic.co.uk before we publicise the site more widely. 


September 24, 2013

ONS Consultation: "The census and future provision of population statistics in England and Wales"

As part of its investigation of alternatives to another traditional Census "Beyond 2011", ONS is now consulting on what it sees as two possible approaches:

• a census once a decade - similar to the 2011 Census but primarily online; or
• a census using existing administrative data and compulsory annual surveys.

Both approaches would provide annual statistics about the size of the population, nationally and for local authorities. A census using existing data and annual surveys would provide statistics about population characteristics every year. An online census would provide more detailed statistics but only once a decade.

The consultation runs until 13th December 2013, and ONS has arranged a number of events.
You can find the consultation document and a link to the online questionnaire here:


September 5, 2013

Public Administration Select Committee (PASC): Study into Open Data – DUG's submission

Following the earlier message below (July 27), the key themes of DUG’s submission are to:

• Recognise the importance of government open data to business
• Welcome the acceleration in progress in recent years
• Alert PASC to the fact that users outside the public sector (such as businesses and charities) are not able to enjoy all the free access arrangements which have been made for public sector users
• Urge PASC to press for the National Address Gazetteer and the Postcode Address File to be made open data


August 31, 2013

Job vacancies

Several new job vacancies in DUG member companies can be seen at the tab to the left.


August 25, 2013

The Guardian Data Blog, and Data Journalism

Lee Madden, former member of DUG whilst at Barclays, and now at Ernst & Young, is a great fan of the Guardian Data Blog, and has posted news on the DUG Network, on LinkedIn (it appears that you’ll need to sign in, rather than providing a direct link here).

“The Guardian’s data blog is a great source of both commentary on data and datasets. They also promote courses on data visualisation and have an active interest in data journalism. I'd recommend checking it out – I particularly enjoy it when they provide data commentary on stories that are in the public eye, e.g. mapping of the London riots or analysis on the A-Level and GCSE results.”

This also leads to “Data journalism at the Guardian: what is it and how do we do it?” Illuminating stuff. 


August 20, 2013

Tracking a decade of changing Britain using the Census and other Open Data – MRS Census Geodemographics Group seminar – Wednesday 6th November, 2013

Here’s news of a seminar “Tracking a decade of changing Britain: How businesses are benefiting from early use of Census and other Open Data”

You’ll see that there is a fine array of speakers, including VARs, and that there is a discount if you book before 6 October. https://www.mrs.org.uk/event/course/1946   


August 11, 2013

DUG's10th Annual Conference: “Understanding millions of mobile consumers” – Wednesday 9 October 2013, The Royal Society

This year’s DUG Conference at the Royal Society is on the topic “Understanding millions of mobile consumers”, and we will again have >100 delegates, enabled by financial support from the Economic and Social Research Council.

The topic has been prompted by discussion amongst DUG members seeking new insights into the drivers of consumer behaviour in our "always connected" world", and the need for better information about footfall, workplace populations, new shopping behaviour enabled by the growth in the use of smart phones, and the necessity to analyse massive volumes of data. The provisional programme is here.

In the morning session our expert speakers will explore these issues. The afternoon focuses on the ESRC’s support for retail research, including a showcase for some of the research projects specified by retailers for Masters’ students: http://www.esrc.ac.uk/collaboration/business-sector/retail/retail-masters/  and also has two Workshops:
• Research Issues: “What’s rattling your cage?” to identify the top priorities
• Skills Training for Insight analysts, which reflects members’ growing concern about difficulties of recruiting numerate staff, and the need for training

As always, the purpose of the conference is to bring together people from DUG's member companies) and also specially invited guests in government and universities, to spread knowledge and stimulate new ideas. This is DUG’s tenth annual conference, and we are convinced that increased dialogue brings enormous benefits to all parties.

As in previous years, the proceedings will be posted on this website. 


August 1, 2013

2011 Census data at Output Area level in E&W published – ethnicity, identity, language and religion

The ONS has now announced the publication of 19 tables for Output Areas in England and Wales on ethnicity, identity, language and religion – some of the variables that are of most interest to many DUG members.

You can download them from NOMIS: https://www.nomisweb.co.uk  


August 1, 2013

The Open Data User Group (ODUG) is seeking new members – might you consider applying? (Closing date: 23 August 2013)

Many of you will be aware of ODUG, an independent advisory group to government, which highlights public sector data with potential to bring social and economic benefit, and works with the Government to promote its release as open data. This is very much in support of DUG’s own long-established objectives.

ODUG is now seeking new members:


July 29, 2013

HMRC consultation on sharing and publishing data for public benefit (Closing date: 24 September 2013)

HMRC has launched this consultation https://www.gov.uk/government/consultations/sharing-and-publishing-data-for-public-benefit

“This consultation invites views on proposals for HMRC to increase the scope for the department to share non-identifying information (that is, information that does not relate to identifiable individuals or legal entities), and on proposed safeguards. Options cover general and aggregate data as well as anonymised data sets.

HMRC also seeks views on the potential benefits, costs / risks and necessary safeguards for proposals to share VAT registration data, either publicly or under controlled conditions for specified purposes, for example, credit rating. Views are welcome on the principles underlying these proposals and on the suggested approaches to implementation.”

DUG has long had its eyes on HMRC data (such as Incomes), and data held by the Valuation Office Agency (which reports to HMRC) such as Council Tax, and will respond to the consultation. 

July 27, 2013

Public Administration Select Committee (PASC): Study into Open Data (Responses due by 3 September)

PASC’s latest enquiry is focusing on the role of Open Data, what the government’s aims should be, what other data could be made available and how, as well as reviewing the data currently available and finding examples of best practice. It also asks which datasets are the most important and who uses them. Here’s the link:



July 22, 2013

Council Tax statistics from the Valuation Office - new classifications

Several DUG members already make use of statistics on Dwelling Stock by Council Tax Band, which are published at Output Area level for England and Wales: 


There are now some extra stats which go only down to Lower Super Output Area level, but which give more detail about dwellings. “These statistics were calculated from domestic property data extracted from the Valuation Office Agency’s administrative database on 1 August 2012. They provide a series of tables on the number of dwellings by Council Tax Band, property type (e.g. house, flat, bungalow, etc) and number of bedrooms and by various recognised statistical geographies, spanning England and Wales”. They also include “build period”:



July 5, 2013

Scotland's Census 2011 – Further news of outputs – Some OA-level data in August

http://www.scotlandscensus.gov.uk/en/news/articles/outputsprospectus.html the latest timetable for Census outputs for Scotland, includes, on 15 August: 

• Population estimates and household estimates by Postcodes, Output Areas and Datazones
• Geography products, including Area (OA) boundaries (shapefiles), Postcode to OA and OA to other geographies look-up files

So we’ll then have some stats for all four countries of the UK, but you’ll also see that Scotland’s Key and Quick Statistics (for Output Areas) are scheduled “from [NB] late September 2013”.

June 28, 2013

2011 Census – ONS, England and Wales – Postcode Headcounts, Key Stats for Built-Up Areas – published

These were published today, & here’s the link: http://www.ons.gov.uk/ons/guide-method/census/2011/census-data/index.html


June 18, 2013

ONS Census Analysis – Free Workshops – Manchester, 18th July, & London, 24th July 2013

For those of you already up to your necks in, or limbering up for, analysis 2011 Census, here’s the chance of some therapy being provided free by ONS: http://www.ons.gov.uk/ons/guide-method/census/analysis/summer-workshops/index.html

“Census Analysis workshops will take place in Manchester and London in July. This is your opportunity find the latest information on the 2011 Census outputs progress, what's coming next and to discuss the census analysis articles published so far date and the planned work for the future. The link provides all the information as well as an email address to secure your place: Census.analysis.inbox@ons.gsi.gov.uk

June 15, 2013

Shakespeare Report – Reviewing Open Data – The Government's Response

Following the publication of Stephan Shakespeare’s report on Open Data last month (see below), the Government has just published its response:

It is positive, and includes:

“Building on the recommendations of the Shakespeare Review, the government will include details of a new national Information Infrastructure in our revised Open Government Partnership National Action Plan due to published in October.

Working with data users, we will identify the data we hold that should be part of that new National Information Infrastructure. This data will be available for release to support economic growth and the developers and entrepreneurs who use open data to build information-led businesses – helping us compete in the global race.”

This could be another step towards the National Address Gazetteer being made freely available, but Royal Mail won’t easily give up its rights to sell PAF:

“The Postcode Address File (PAF) is a fully integrated part of Royal Mail’s business and will continue to remain so. In recognition of the continued importance of this dataset to innovation and growth, the Government has been working closely with Royal Mail to improve the licensing regime and drive greater take-up of the data. The Government is now announcing that, from the beginning of July, Royal Mail will provide the PAF for free to independent* small charitable organisations and free to independent* micro-businesses for one year. Royal Mail will also be increasing the number of free online address look-ups on its website from 15 to 50 per day. In addition, Royal Mail will consult in July on a radical simplification of the licensing regime for all customers based on a permissive licence with minimal restrictions.”
* Independent – not associated or affiliated with any existing Solutions Provider

June 7, 2013

ONS's new Open Geography Portal – for downloading geographic data such as postcode directories, boundaries, names and codes for ONS's area statistics

Here’s news of ONS’s Open Geography Portal “which allows users to discover, view and download geographical reference data to support National Statistics”
The link https://geoportal.statistics.gov.uk/geoportal/catalog/main/home.page  looks unexciting, but by roaming around (e.g. Browse) you can find some useful stuff.

June 5, 2013

Public Administration Select Committee – Communicating and publishing statistics – Report now published

Recalling DUG’s submission on 13 October last year, the House of Commons PASC report on Communicating Statistics has now been published.

The recommendations are not revolutionary, but add further pressure in the right direction, including:

• The UK Statistics Authority should work proactively to bring together and clearly present key statistics, from various sources, around common themes or events, such as elections and referendums, as well as broader topics such as the labour market and economic trends.
• The ONS website must be improved. Government statisticians should work much more closely with different kinds of users in order to present statistics in ways which meet their different needs.
• The Committee says the Statistics Authority should find more creative ways of communicating statistics, for example, through interactive guides. This should be in addition to the publication of more raw data in machine-readable format for experts who want the full results, not just the edited highlights presented in releases for a mass audience.
• In addition to the many routinely-produced statistics, government statisticians produce thousands of pieces of data on demand, known as “ad hoc statistics”. This is positive, but more of this kind of data should be published proactively, rather than simply in reaction to requests.

You can find more here: http://www.parliament.uk/business/committees/committees-a-z/commons-select/public-administration-select-committee/inquiries/parliament-2010/statistics/

May 19, 2013

2011 Census results for England and Wales – the latest news from ONS

ONS has just published the latest edition of its Census Newsletter. It includes information on data releases, the commissioned tables service, and an update on plans to improve access to data via the ONS website. Here's the link:

May 16, 2013

Shakespeare Report published – Reviewing Open Data to explore growth opportunities – further positive news

Following the announcement in October (see below), the Shakespeare Review of Public Sector Information was launched at a packed meeting yesterday.

Its recommendations include:
• The government should produce and take forward a clear, predictable, accountable ‘National Data Strategy’ which encompasses PSI in its entirety.
• A National Data Strategy for publishing PSI should include a twin-track policy for data release, which recognises that the perfect should not be the enemy of the good: a simultaneous 'publish early even if imperfect’ imperative AND a commitment to a 'high quality core'. This twin-track policy will maximise the benefit within practical constraints. It will reduce the excuses for poor or slow delivery; it says 'get it all out and then improve'.
• “One would be hard-pressed to find any expert who, asked to create new structures for core reference data from scratch, would advocate the current Trading Fund model”. The overarching aim of the Trading Funds [such as Ordnance survey] should be to deliver maximum economic value from public data assets they provide and support, by working to open up the markets their data serves. …..This means they should work towards opening up all raw data components, under the Open Government Licence (OGL) for use and re-use.

And also:
• We should develop a model of a 'mixed economy' of public data so that everyone can benefit from some forms of two-way sharing between the public and the commercial sectors.

These recommendations very much follow the opinions that we expressed when Stephan Shakespeare was a guest at our meeting in September last year, and subsequent representations by DUG members, and will give another hefty shove toward opening up more government data.

Here are some links:
Shakespeare Review Report: https://www.gov.uk/government/publications/shakespeare-review-of-public-sector-information
Deloitte Market assessment: https://www.gov.uk/government/publications/public-sector-information-market-assessment
Ministerial quotes from Matthew Hancock and Francis Maude have also been published on the Gov.UK website:
The Guardian’s Public Leaders’ Network blog: http://www.guardian.co.uk/public-leaders-network/2013/may/15/stephan-shakespeare-publicsector-open-data  


May 13, 2013

The Census: Now and in the Future – SRA Summer Event – Afternoon, 26 June 2013

On 26 June the Social Research Association’s summer event “The Census: Now and in the Future” will examine this issue from a variety of expert perspectives.

“Chaired by Mark Easton of BBC Home Affairs, this promises to be a lively and engaging afternoon”, with presentations from:
• “How is the Census faring” Ian Cope, Director, Population and Demography, ONS
• “What’s needed locally?” Ludi Simpson, University of Manchester, president of the British Society for Population Studies
• “Can the use of ‘Big Data’ eliminate the need for yet another traditional Census in 2021?” Keith Dugmore, Director, Demographic Decisions Ltd, and Honorary Professor, UCL
• Discussant: Simon Briscoe, author and journalist (and advisor to Bernard Jenkin, MP)

The event will be held at the Local Government Association in Westminster, and the cost is £65. You can find a flyer and booking details here: http://the-sra.org.uk/event-registration/?ee=74 

May 12, 2013

Land Registry to release free historical property data 

Here’s some good news from the Land Registry:

“We are pleased to announce that we will release a number of free datasets as part of our commitment to the Open Data agenda.

The following datasets will be available to download from our website under Open Government Licence:
• 30 May 2013 – historical House Price Index (HPI) tables. These will be in CSV format and from July they will be in machine readable format as linked data. The tables form part of the HPI calculation and contain up-to-date indices, average house prices and sales volumes dating back to 1995 for all England and Wales.
• 28 June 2013 – historical Price Paid Data (PPD) in CSV format. This will be available as linked data from July. Historical PPD will contain records of the price paid for every residential property sold at full market value lodged for registration between 1 January 2009 and 31 January 2012 in England and Wales.”

There’s news of further datasets at: http://www.landregistry.gov.uk/announcements/2013/land-registry-to-release-free-historical-property-data

And you can see commentary from Geolytix at: http://geolytix.co.uk/blog/?p=175


April 20, 2013

"Everyone's postcodes to be privatised in Royal Mail flotation, despite objections from Sir Tim Berners-Lee" – Daily Telegraph

A report in yesterday’s Daily Telegraph starts:

“A national database containing the postcodes of everyone in Britain is to be privatised along with Royal Mail, despite direct "very forceful" protests to David Cameron from Sir Tim Berners-Lee, the founder of the world wide web.”

and includes:

“Cabinet Office minister Francis Maude has been privately battling with Business minister Michael Fallon over whether to include the database, worth between £500million and £900million, in Royal Mail's flotation, which is expected as early as October.”  (You might have spotted that both ministers are past winners of our annual Award).

Here’s the link to the full article:

Yes, a further battle in the address wars. DUG will continue to support the case for national addressing to be free.



April 8, 2013

Electoral Register opt-out rates for Local Authorities (ranging between 8% and 80%)

Callcredit Information Group has today published its '2013 Regional Electoral Opt-Out Report'. The message starts:

“Of the 46m consumers eligible to vote within the UK, 18m have chosen to opt-out and have therefore have made themselves unavailable for marketing activity and wider benefits it can bring. The over-all opt-out rate for 2013 is 39.8%, a 0.2% decrease from 2012.

There remains a large regional variation with the lowest opt-out rate of 8.03% and the highest of 80.25%, therefore marketers will need to continue to utilise additional data sources in order to verify their own customer data.”

You can read more at: http://www.callcredit.co.uk/press-office/news/2013/04/2013-electoral-register-opt-out-rates-begin-to-slow


April 4, 2013

Passenger statistics for every railway station in Britain: April 2011 – March 2012

“Waterloo has retained its position as the busiest station in Great Britain as top stations continue to attract growing numbers of passengers, reveal statistics published today by the Office of Rail Regulation (ORR).”

If this is of interest, you might want to put on your anorak and bobble hat before you open the link http://www.rail-reg.gov.uk/server/show/nav.1529 to scroll through estimates of passengers for every station in Britain.



March 26, 2013

2011 Census release 2.4 – Including Social Grade at OA level for England and Wales

ONS has announced further news about the release of more statistics, including:

• Approximated social grade (household reference persons aged 16 to 64) – at Output Area level (downloadable from NOMIS www.nomisweb.co.uk/ )

• Also, the Commissioned Tables service has been launched much earlier than planned. This could be useful if you want more detailed categories for topics such as birthplace, ethnicity, religion, etc. 


March 17, 2013

ODUG's response to the OFCOM review of PAF: "Don't sell our postcodes"; and our message to Michael Fallon MP

Coinciding with Heather Savory, Chair of the Open Data User Group, speaking at the DUG meeting on 15 March, here’s news of her blog. ODUG (including Bob Barr) has just made an excellent response to OFCOM about the future of the Postcode Address File:

Also, DUG has written about “the need for free use by citizens and the private sector, as well as the public sector” to Michael Fallon, who is now the Minister for Business and Enterprise, having been Chair of the Treasury Sub Committee, when it recommended the creation of a national address file. You may remember that he won our award at the DUG Conference in 2008 http://www.demographicsusergroup.co.uk/resources/2008conf-DUG_AWARD_2008.pdf

Let’s hope that this mounting pressure finally achieves a breakthrough in government policy, and the arrival of free addressing. 


March 5, 2013

DUG 2011 Census Workshop session – Presentations

DUG’s Census Workshop at the Royal Society on 27 February went well, and here are the three presentations:

• The value of small area Census data products to commercial companies (Annette Dellevoet)

• 2011 Census – Statistics (Emma White, Head of Policy and Analysis, 2011 Census Outputs at ONS)

• 2011 Census – Geography (Ian Coady, ONS Geography) 

February 24, 2013

Beyond 2011 – DUG’s minimum needs for Census-type statistics: our views, now sent to ONS

With many thanks to the members who contributed, here is a summary of DUG’s needs for future Census-type statistics, which has now been sent to ONS.

The ONS Beyond 2011 team is continuing to develop the business case to the Treasury.


February 24, 2013

The Shakespeare Review – & the possibility of more free Open Data – an invitation to respond to this YouGov Survey

Stephan Shakespeare, the Chair of the government’s Data Strategy Board, who was a guest at DUG’s September meeting, is reviewing public sector information. Here’s the message from the DSB:

“The Shakespeare review of Public Sector Information led by Stephan Shakespeare will examine ways in which the public sector can improve access to its data and information to stimulate economic growth, promote efficiencies, and bring wider benefits to society.

At the end of the review, recommendations will be made to Ministers on how to widen access to public sector information and how to exploit new and innovative opportunities for open data including issues around data protection and copyright.

Stephan is keen to gather views from as many types of users, re-users and potential users of public sector information as possible and invites you to complete an online survey https://start.yougov.com/refer/vvCYQdvVkq6JVB  . The survey will close on Friday 15th March 2013.

Your contribution to this survey is important and will help shape the future of the UK’s public sector information and open data policy.”

Do please complete it if you have 10 minutes – the greater the number of replies will increase the probability of more data (such as address files and mapping) being made freely available.


February 20, 2013

2011 Census Key and Quick Statistics for England and Wales – now other geographies, including postal sectors

Following the news on 30 January about 2011 Census data down at Output Area level for England and Wales, here’s news of the same statistics for other geographies, including postal sectors.

“ONS has now published 2011 Census Key Statistics for national parks, and Key and Quick Statistics for health areas, postcode sectors and National Assembly for Wales constituencies and Assembly for Wales electoral regions.”

The Key Statistics for national parks are only available via the ONS website at: http://www.ons.gov.uk/ons/guide-method/census/2011/index.html

The data for health areas, postcode sectors and National Assembly for Wales constituencies and Assembly for Wales electoral regions will be available via the NOMIS website at:


February 18, 2013

2011 Census Pack free from Geolytix

Here’s some good news from Geolytix. It starts:

“Today GeoLytix are releasing harmonised Output Area Census Data covering the c.200 core variables as a single easy-to-use flat file. This data is being released with no charge under the same Open Government License that the censuses themselves use. There are no additional attribution requirements, you can use this data in exactly the same way you can use raw census data. We’ve also written a swish User Guide, prepared meta-data tables and even included a single consolidated look-up file to higher geographies and proportional look-ups to the 2001 Output Areas.”

You can read all about it at: http://geolytix.co.uk/blog/ 


February 15, 2013

ESRC: Master's projects proposed by retailers – news of the first 10

Following the news on 12 January (below) there are already 10 retail projects announced on the ESRC’s website:


February 14, 2013

"Mapping the 2011 Census: how one man produced a library for all" – a report in The Guardian

Here’s news from the Guardian:

“Alex Singleton downloaded every single census dataset for every local authority in England - and then produced a free library of downloadable PDFs. Find out what he did…….

Singleton has basically produced a detailed mapping report - as a PDF and vectored images - on every one of the local authorities of England & Wales. He automated the process and has provided the code for readers to correct and do something with. In each report there are 391 pages, each with a map. That means, for the 354 local authorities in England & Wales, he has produced 127,466 maps.” [Goodness…..]


February 6, 2013

Further Benefits of an Open National Address Dataset – Paper by the Open Data User Group

It’s good to see the Open Data User Group is piling on the pressure for a free national address file, with an excellent paper about the benefits:



January 30, 2013

New stats after 10 years: 2011 Census Key and Quick Statistics for Output Areas published today

Here’s news from ONS of fresh Census data down at Output Area level for England and Wales:

There’s more information about Bulk Data in csv format:

And also further geography products for electronic download:

January 22, 2013

ESRC – Retail Research Data – A new website to help retailers find free Open Data easily

As well as the Masters’ projects for retail companies, another element of the ESRC’s Retail Research & Data project is a new website:
“The primary purpose of Retail Research Data (RRD) website is to enable insight and store location analysts working in retail organisations to get easy access to free datasets which may be of value to their businesses.”

The website has 6 types of data:
1. Small area geodemographic classifications - such as the Output Area Classification (OAC)
2. Small area statistics – particularly from the 2001 Census, on many topics
3. Lists of addresses, with postcodes, and places
4. Postcode directories – for linking postcoded addresses (including customer records) to Output Areas and other geographical areas, or to grid references for mapping
5. Map data – digital boundaries (such as Output Areas), and also background mapping
6. Sample surveys – giving lots of detail about particular topics

RRD also includes a mapping facility to view the Classifications and Small Area Statistics for the United Kingdom and its constituent countries.

Here’s the link: http://www.retailresearchdata.org

January 12, 2013

Do you have projects now that Master's students can help you to progress this spring & summer?

If you work for a UK retail company, this is an opportunity for you. The ESRC’s “Retail Research & Data” initiative, which involved commercial companies proposing projects for Masters’ students, was a great success last year, and the ESRC has decided to repeat it in 2013. We are therefore seeking proposals for new projects as soon as possible.

This presents a great opportunity to get a Master’s student to help you to make progress with:
• Major current issues, such as multi-channel marketing and customer insight, regional variations in the economy, store networks, transport, and social media.
• Projects that you’ve long had in mind, but not had the resource to carry out
• Working with data – both your own customer data, and also maximising the value to be obtained from government initiatives such as www.data.gov.uk, and the perfectly-timed arrival of the 2011 Census

If you have a project in mind, please email dugmore@demographic.co.uk ASAP for a Project Proposal Form. It’s essential to move quickly on this, to publicise projects early and get the choice of the best students.

Last year’s experience illustrates the main features of the initiative:
• The list of the 16 projects specified by 10 retailers: http://www.esrc.ac.uk/funding-and-guidance/collaboration/working-with-business/dissertation-opportunities.aspx
• ESRC’s publicity and explanation of the features to students and academic supervisors: http://www.esrc.ac.uk/funding-and-guidance/collaboration/working-with-business/retail-masters-student-projects.aspx
• Presentations by 12 of the 16 students at the Demographics User Group Conference: http://www.demographicsusergroup.co.uk/resources/ESRC+-+Masters+Student+presentations+-+Final.pdf

January 5, 2013

Members’ needs for data from government – our updated Manifesto

DUG’s Data Manifesto, which was initiated by Mike Whitelegge at M&S, has made an impact in Whitehall, and is proving a very useful summary wishlist / reference document.

Here is the latest version, which you might find useful if you are challenged with “What more government data would you like to see made freely available?” 


January 1, 2013

Happy New Year........... & here's a summary of our many activities in 2012

Very best wishes for a successful and enjoyable 2013.

Looking back on the past year, we hope that you’ll find this summary of the activities of the Demographics User Group during 2012 both interesting and useful.

As you’ll see, it’s been another busy and successful year, with members sharing their experience in numerous ways, pressing government for further progress in the policy of making more data freely available, and also a major initiative with the ESRC to provide Masters’ students with relevant research projects.


December 18, 2012

First 2011 Census results for Scotland

After all the activity by ONS for England and Wales, we now have the first 2011 Census statistics for Scotland.

The following link leads to some headline results, and also summarises the timetable for future releases (with Release 2 – Key and Quick statistics tables; Census Profiles; Population and Household estimates by postcode and Geography products – not becoming available until “from summer 2013”): http://www.scotlandscensus.gov.uk/en/censusresults/

And here’s the BBC’s “11 things we learned from the Scottish 2011 census”: http://www.bbc.co.uk/news/uk-scotland-20754751


December 13, 2012

New funding to accelerate benefits of open data – £7.5m Data Strategy Board Breakthrough Fund; a new £850,000 Open Data Immersion Programme; & easier OS data

The Data Strategy Board team reports:

“The Government has announced three new complementary Open Data investments
• A new £7.5m Data Strategy Board Breakthrough Fund to which public sector bodies can apply;
• A new £850,000 Open Data Immersion Programme to which companies can apply; and
• Upgrades to the format in which Ordnance Survey data can be downloaded to make it easier, more accurate and more flexible for companies, government, charities and individuals to use.”

All of the measures announced are expected to be in place by early April 2013. You can see more detail at: http://news.bis.gov.uk/Press-Releases/New-funding-to-accelerate-benefits-of-open-data-684c1.aspx

It’s another step in the right direction.

December 12, 2012

2011 Census – Key Statistics at Local Authority level now published

You will no doubt have seen some of the extensive media coverage about the latest release of Census statistics. Here’s ONS’s announcement:

“ONS has now published the 2011 Census key statistics at local authority level on its website at http://www.ons.gov.uk/ons/guide-method/census/2011/index.html

The release is supported by data visualisation, statistical bulletins, with commentary and short stories on international migration, ethnic group and identity, religion, and (in Wales) Welsh language. The release is also supported by a technical report comparing the estimates of economic activity to the regular labour market statistics ONS produces. Frequently asked questions will be available on the ONS website to help you answer census related queries.”

And here’s some mapping of the results:

You’ll see that these stats are at Local Authority level. The next releases will be:
• 30 January 2013: all of the Key and Quick Statistics tables at Output Area, ward, parish and parliamentary constituency levels.
• By the end of February 2013: Key and Quick Statistics for the remaining geographies, (including postal sectors) together with postcode estimates of population by *gender*, and the number of households.

December 6, 2012

"Transport costs are eating into family budgets, the ONS says" – Living Costs and Food Survey

The Guardian reports:

“British households have cut back on clothes and furniture spending to cope with a sharp rise in petrol bills, according to the government's annual Family Spending survey. The Office for National Statistics revealed that households on average spent a record £483.60 a week in 2011, up £10 on the year before – but much of the increase was gobbled up by a 9% rise in spending on petrol. The figures also reveal that spending is falling fastest among the poorest, and suggest the "squeezed middle" may be a statistical myth.”

You can read more at: http://www.guardian.co.uk/money/2012/dec/04/household-spending-record-high-rising-petrol-prices  
And here’s the link to the survey report: http://www.ons.gov.uk/ons/rel/family-spending/family-spending/family-spending-2012-edition/index.html


December 5, 2012

Top 10% of households are 850 times wealthier than the bottom – ONS

The Guardian has reported the latest results from ONS’s Wealth and Assets Survey:
& here’s a link to the ONS website:


November 23, 2012

2011 Census latest release – first estimates (Population & Households) at Output Area level for England and Wales

ONS has just sent an alert announcing the publication of population and household estimates at Output Area and ward level for England and Wales. The release is available on the ONS website at: http://www.ons.gov.uk/ons/guide-method/census/2011/index.html

You’ll see that it’s region-by-region, and in Excel format (DUG has been pressing for csv for future releases, and ONS has promised it). This follows the earlier publication of geography products such as OA boundaries (see DUG News 31 October)

ONS’s announcement continues:

“The first phase of the second release is planned to commence on 11 December with a full package of key statistics and a selection of quick statistics tables, for England and Wales. This will be supported by statistical bulletins for England and Wales, for Wales (in English and Welsh), with commentary and short stories on international migration, ethnic group and religion, and Welsh language. The release will be accompanied by data visualisations and by a fact pack product that provides quick reference 2001-comparability information for each local authority. Table layouts will be published prior to the release.

On 30 January 2013, we will issue all of the Key and Quick Statistics tables at output area, ward, parish and parliamentary constituency geographies. These outputs will also be accompanied by a statistical bulletin and census analysis. By the end of February 2013, we will issue the Key and Quick Statistics for the remaining geographies such as Welsh Government devolved constituencies, National Parks and health areas together with postcode estimates by [*gender*] and for the number of households. We will announce the precise date in February as soon as we can and at least four weeks prior to that publication.”


November 23, 2012

Open Data chief says postcodes should be free

Just in case you haven’t spotted this in “Civil Service World”, there’s more support for the PAF being made free…….

“The UK’s postcode address database should be released for free use by private companies, the chief executive of the new Open Data Institute (ODI) has told CSW”.

Speaking in an interview with CSW, Gavin Starks argued that the UK’s Postcode Address File (PAF) is “such a fundamental building block for pretty much every business that needs an address that it’s kind of crazy that we don’t have that as a properly open file.”

You can read more here: http://network.civilservicelive.com/news/csw/read/711763/open-data-chief-says-postcodes-should-be-free  
& here’s a bit more about the ODI’s recently-appointed chief, Gavin Starks: http://www.theodi.org/odi-top-jobs


November 21, 2012

GeoLytix release full Postal Boundary Set as Open Data

Here’s news of a really encouraging step forward with more (free) Open Data. The announcement reads:

“Today, GeoLytix are pleased to release our full set of Autumn 2012 GB postal sector boundaries under an Open Data license. Postal boundaries are a critical component of our countries spatial infrastructure. They are used in everything from defining media boundaries and store delivery areas to web map searching, analysing customer data and producing a wide range of maps for the public.
You are free to use and exploit these data in any way you see fit with the single requirement that you attribute GeoLytix, the Ordnance Survey, the National Statistical Agencies and the Royal Mail as the copyright holders. Postal Geography constantly changes and this Open Data version is frozen. Updates and support are available under a paid-for GeoLytix license.”

The story behind their creation is at: http://geolytix.co.uk/blog/ . You can access the product documentation including the license, an FAQ and the user guide together with the full data here http://www.geolytix.co.uk/data-and-software/postal-boundaries  (no registration required)


November 12, 2012

APPSI paper on a National Information Framework for Public Sector Information and Open Data – Call for comments

Government’s Advisory Panel on Public Sector Information (APPSI) has just published this paper, which supports the case for a strategic approach to getting better access to government information such as address files, property information, and map data:


APPSI’s views should help create more momentum for Open Data. You’ll see some more background below. If you have comments on the ideas and proposals, please email them to secretariat@appsi.gsi.gov.uk


November 10, 2012

George Osborne champions science, & The Data Revolution is first on his list

Here’s some encouraging news from a meeting yesterday at the Royal Society.

George Osborne, Chancellor of the Exchequer, spoke of both his belief in the value of science as a driver of the UK economy and his commitment to science funding into the future. He identified eight areas that the government believes that Britain is a world leader.

The first on his list was “The Data Revolution and energy-efficient computing”. His comments included:
“We have some of the world’s best and most complete data-sets in healthcare, demographics, environmental change and food.”

“We have also created the open Data Institute in the East of London in Tech City, to bring together study of all the data from our whole environment. And as a government we are making more and more of that data available. Business will invest more as they see us invest more in computational infrastructure to capture and analyse data flows released by the open data revolution.”

It’s good to see data at the top of the agenda. You can see the full detail at: http://royalsociety.org/news/2012/osborne-at-royal-society/


November 1, 2012

Open Data User Group calls for address files to be made free of charge

It’s great that the Open Data User Group has taken up the cause to make address files to be made free of charge. Here are the headlines:

• ODUG calls for the publication of the Postcode Address File (PAF) as open data under the Open Government Licence
• Charging for all Postcode Address File (PAF) licencing should end
• The Ordnance Survey AddressBase Plus and the National Street Gazetteer (NAG) should be released as open data in conjunction with the Postcode Address File (PAF)

……and you can see more at: http://data.gov.uk/odug 

Many thanks indeed to Sarah Hitchcock, and to other members of DUG who have helped make the case.


October 31, 2012

2011 Census Geography products now available (in advance of statistics next month)

Here’s news from ONS about the geography products that they have just published in advance of the Output Area level statistics that start to flow (with counts of population and households) on 23 November:


You’ll see from the links that they comprise:
• 2011 Census boundary files (.shp) for OAs, LSOAs and MSOAs.
• 2001 Census boundary files (.shp) for OAs, LSOAs and MSOAs.
• 2001 to 2011 Census geography OAs, LSOAs and MSOAs lookup files
• 2011 OA to higher area lookup files
• Postcodes to OAs, LSOAs and MSOAs
• 2011 Census median population weighted centroid files (.shp) for OA, LSOA and MSOAs datasets

We learn that only 2.6 per cent of 2001 OAs have changed as a result of the 2011 Census, with the average population in an OA increasing from 297 in 2001 to 309 in 2011.


October 24, 2012

The DUG Conference “Retail Issues, Big Data, and Research”

and the winner of our 2012 Award for “Better information from government” is ................................The Department for Work and Pensions

The DUG Conference was held on 10 October at the Royal Society, and you can see the programme, presentations, and a list of attendees at the tab on the left.
The Department of Work and Pensions, represented by its Director of Information David Frazer, was present to collect the DUG Award to the Department “for proactively collecting, re-using, and publishing more and better data”.

A DWP Press Release http://www.dwp.gov.uk/newsroom/press-releases/2012/oct-2012/dwp105-12.shtml  includes:
Minister for Welfare Reform Lord Freud said:

 “The Government has a responsibility to be open with data, so we can be more accountable to the public as well as provide the public and private sectors with the tools to plan and improve services.
“Better data also gives companies the information they need to find opportunities for growth, which benefits us all.
“I want to thank the DWP teams working to improve our data and for allowing greater public access of our information resources and to congratulate them on winning this award.”

Our citation reads:
“Commercial companies, when making decisions about targeting investment towards particular types of customers and local markets, welcome the availability of government statistics across the United Kingdom. The Census of Population is one major source, but statistics derived from government departments’ administrative files present a growing opportunity.
The Department for Work and Pensions has led the way, using the information that it collects from the UK’s citizens to create anonymous counts of various populations for very small areas, and making these easily accessible. It is also encouraging the wider use of government surveys, with the Wealth and Assets Survey benefiting from the addition of a neighbourhood classification.
Members of the Demographics User Group (DUG) strongly welcome these developments, and the future plans outlined in the DWP Open Data Strategy, which will enable better understanding of customers and increase the efficiency and growth of businesses”


October 24, 2012

Government review of open data to explore growth opportunities – Chaired by Stephan Shakespeare

Here’s news of a new government review which “aims to improve access to data and help promote economic growth by examining the market for public sector information.”

It’s to be led by Stephan Shakespeare, Chair of the Data Strategy Board, who was guest speaker at DUG’s September meeting.

At the end of the review, recommendations will be made to Ministers on how to widen access and consider new and innovative opportunities for open data. In particular, the review will look at the following areas:
• the current use of public sector information within government, the private sector and by the general public;
• the opportunities for innovation and developing new data services with public sector information;
• the constraints on developing new services with public sector information and the role that current funding models play;
• the impact of competitiveness on existing and new businesses entering the market, as well as the way in which organisations secure access to data; and
• an audit of current and past studies, as well as taking stock of the current use and re-use of public sector information.

There’s more information at: http://news.bis.gov.uk/Press-Releases/Review-of-open-data-to-explore-growth-opportunities-68203.aspx 


October 23, 2012

2011 Census stats on second address / second homes

ONS has just published stats from the 2011 Census on the number of people with second addresses in local authorities in England and Wales.

The BBC has run the story http://www.bbc.co.uk/news/uk-20026773 together with mapping http://www.bbc.co.uk/news/uk-19759650  

The data can be accessed here: http://www.ons.gov.uk/ons/guide-method/census/2011/index.html

The release includes two separate tables which provide, down to local authority level:

 • the number of people who spend more than 30 days a year at a second address in a local authority where they do not usually live, and
• the number of people usually resident in each local authority who had a second address elsewhere


October 13, 2012

Public Administration Select Committee – Study 4: Communicating and publishing statistics – Submission on behalf of the Demographics User Group

Following our response to the first of House of Commons Public Administration Select Committee’s studies on statistics and their use in Government (see August 4), another topic is “Communicating and publishing statistics”:


You can see DUG’s submission here. The key themes are summarised as the need to:

• Put more emphasis on “the last 100 yards”, making it easier to find and use data.
• Shift some resources from statistical production to marketing (identifying needs and meeting them) within the various parts of the Government Statistical Service.


October 13, 2012

ONS's Beyond 2011 Project – papers, & free seminars

Following the news about ONS’s Beyond 2011 consultation, looking at alternatives to another Census (see September 26), here’s the latest:

1) News of research work being undertaken, & the evaluation of options: http://www.ons.gov.uk/ons/about-ons/what-we-do/programmes---projects/beyond-2011/index.html

2) Seminars. Here’s a message from ONS:

“You may be interested to know that we will be running a series of seminars during November as part of our ongoing engagement with our stakeholders.

Seminars are planned for the following dates and locations and will run from 10.30 -15.45:
• 26 November 2012 – Manchester
• 28 November 2012 – London – Royal Statistical Society

During the day we plan to cover developments in the Programme, including our investigations into the options, research progress and how we are taking forward the business case and quantification of benefits. There will be the opportunity to participate in group work and a panel session in the afternoon. If you wish to attend one of these seminars please email beyond2011@ons.gov.uk  stating the date and location you would like to attend and the organisation, if any, you will be representing. Please note that there are limited places at these events and places are allocated on a first come first served basis.” 


September 26, 2012

Will there be another traditional Census? News of two government reports

Here’s news of two separate government investigations into whether there should be another traditional Census in 2021.

1. House of Commons Science and Technology Committee:

“MPs have warned that a decision to discontinue the census would have a negative impact on the Government's social science analysis, as well as social science research in the UK, and that alternative systems for collecting population data may not prove to be any cheaper.” You can see the committee’s report at:

2. ONS’s Beyond 2011 User Requirements Consultation

This report provides the results of the first Beyond 2011 public consultation which ran from October 2011 to January 2012 (you can see DUG’s submission below on 17 January 2012). A total of 266 organisations and individuals provided responses to the consultation and 207 people attended the supporting workshops.

The consultation focused on two broad areas:
• the current and future requirements for population and socio-demographic statistics on different topics; and,
• the trade-off between accuracy, geography and the frequency at which the statistics are produced.
You’ll find the report at: http://www.ons.gov.uk/ons/about-ons/what-we-do/programmes---projects/beyond-2011/news-and-events/reports-and-publications/index.htm

An ONS statement also includes:
“In October 2012 we plan to release a whole range of papers on this research work undertaken over the past 6 months. These will explain much more about the approach we are taking to evaluate the options and what the next steps are. We will be reporting much more regularly from now on with further releases later in 2012 and early in 2013”. This will give us the opportunity to learn much more. 


September 7, 2012

ONS's 2011 Census Newsletter – September 2012

Here is the latest from ONS about the release of 2011 Census statistics, including news of the first results, details of the next Census data release - planned for between November 2012 and February 2013 and available via the NeSS website – and a summary of the Census Statistical Disclosure Control methods applied to protect the confidentiality of individuals and entities such as businesses.


August 30 , 2012

Marks & Spencer customer poll suggests better times ahead – Guardian report quotes Steve Bond

Here’s news from yesterday’s Guardian:

“Marks & Spencer said on Tuesday that the latest poll of confidence among its shoppers had delivered its most positive score in more than two years, providing some good news for its embattled chief executive, Marc Bolland. The high street chain is regarded as a barometer of consumer spending and its authoritative monthly poll involves interviews with 4,000 customers.

M&S's director of customer insight and loyalty, Steve Bond, said the latest reading for the four weeks to 17 August was 12 points higher than a year ago. Optimism was riding high, he said, thanks to the feelgood effect of events such as the Olympics, although he cautioned that the poll, launched by Bolland in May 2010, had only ever been conducted in a downturn.

The "most subdued group" in the marketplace remained families battling higher food and fuel prices, added Bond”.

You can read more at: http://www.guardian.co.uk/business/2012/aug/28/marks-and-spencer-increased-consumer-confidence 


August 21 , 2012

ONS: Glen Watson appointed as Director General

Following the retirement of Steve Penneck, it’s good to see that Glen Watson has been appointed as the new Director General of ONS.

Glen is currently Director of Census at ONS, having previously been appointed Director of Neighbourhood Statistics in 2004 (when he spoke at the first DUG conference).

Here’s ONS’s announcement: http://www.ons.gov.uk/ons/media-centre/statements/new-director-general/index.html?format=print 


August 15 , 2012

ONS/BIS Meeting on "The Changing Shape of UK Business" – Tuesday 6 November 2012

For those of you interested in stats about businesses, here’s advance news of an all-day event, coordinated jointly by the ONS and the Department for Business, Innovation and Skills, whose objectives are to:

• “share what we know about the structure, content and performance of UK business;
• raise awareness of the importance and coverage of our statistics;
• hear from users about the range of uses to which our statistics are put; and
• discuss future plans and priorities with a range of users.”

The event is free of charge, but please note that numbers will be limited. You’ll find more detail at:


August  8 , 2012

What is a data scientist worth? Read all about it in the Guardian

A DUG member has spotted this nugget:


It’s clear that if you are a “Data Scientist” you can earn more than if you are a “Data Analyst”…….


August 4, 2012

Public Administration Select Committee – Statistics and Registration Service Act 2007 – Submission on behalf of the Demographics User Group

Following the session with Bernard Jenkin, MP, Chair of the House of Commons PASC, at the DUG meeting in March, and our letter of 1 May (see below), the committee is launching a programme of work on statistics and their use in Government. It is conducting its scrutiny by means of ten short “studies”:

The first of these is on the Statistics and Registration Service Act 2007. DUG has made a submission, with the key themes being the need to:
• Understand and meet the needs of users outside government
• Divert some resources from production into improving access to statistics, and hence increasing use, and maximising value: “the last 100 yards”
• Make much more use of administrative files to create statistics at the lowest Output Area level
• Put more emphasis on speed of delivery, rather than marginal gains in quality 


August 2, 2012

Public Accounts Committee publishes report on Transparency /Open Data

The House of Commons Public Accounts Committee has just published a report 'Implementing the Transparency Agenda'. The Rt Hon Margaret Hodge MP, Chair of the Committee of Public Accounts, said:

"This Committee fully supports the principle of greater openness and its potential to strengthen accountability and drive improvements in public services. But the Government has a lot more work to do before that potential is realised.

It is simply not good enough to dump large quantities of raw data into the public domain. It must be accessible, relevant and easy for us all to understand. Otherwise the public cannot use it to make comparisons and exercise choice, which is the key objective of the transparency agenda”.

There’s more comment, plus a link to the full report, here:


July 27, 2012

Republic of Ireland – 2011 Census results – Part 2

Following the first publication summarising the results of Ireland’s 2011 Census (see 3 April below), here’s the second, which examines social and economic topics such as employment, occupations, education and skills and health related issues:

Again, you’ll see that the report has lots of eye-catching maps & charts. 


July 26, 2012

DUG Members’ needs for data from government – our Manifesto (Updated July 2012)

DUG’s Data Manifesto, which was initiated by Mike Whitelegge at M&S, has made an impact in Whitehall, and is proving a very useful summary wishlist / reference document.

Here is the current version, which you might find useful if you are challenged with “What more data would you like to see made freely available?” 


July 26, 2012

First meeting of the Open Data User Group (ODUG) – Match Report

The Cabinet Office’s Open Data User Group (ODUG) – see the earlier news on 10 July – has held its first meeting.

Heather Savory, the Chair, has written a blog http://www.guardian.co.uk/public-leaders-network/blog/2012/jul/18/open-data-group-clear-agenda and Sarah Hitchcock of Sainsbury’s tells me that the group got off to a good start.

The note of the meeting and slides are on data.gov.uk at: http://data.gov.uk/library/minutes-odug-10072012 and: http://data.gov.uk/library/slides-odug-10072012 


July 23, 2012

2011 Census – ONS's plans for the 2nd release (November – February), including lots of stats at Output Area level

ONS has now published its plans (subject to any redesign as a result of the application of statistical disclosure control) for its Second Release, which gives 36 tables of Key Statistics, and 71 tables of Quick Statistics, at Output Area level.

The second release will also include postcode estimates (numbers of people by gender in each postcode), and estimates of occupied households in each postcode will be produced.

You can see the detailed tables here:

“ONS consulted extensively with census users about requirements for the output of census statistics and consequently the vast majority of 2001 Census outputs will be repeated in 2011. This is for comparability and continuity purposes, and takes account of user satisfaction with the design of the 2001 statistics.

The statistics in the second release will correspond to the 2001 Census key statistics tables produced for univariate information (counts and as percentages), univariate tables and headcounts. A look-up file is available to download. The look-up file will be developed to be relevant to every release of 2011 Census statistics.” 


July 16, 2012

2011 Census – First results for England and Wales – Largest population increase shown by any Census since 1801

The ONS has just published the first (Local Authority level) results from the 2011 Census for England and Wales: www.ons.gov.uk/census  

There’s some good summary coverage on the BBC website, which starts:

“The population of England and Wales has reached 56.1 million, up by 3.7 million in a decade, the 2011 Census shows.
It is the largest growth shown by any census since they began in 1801.
The population in England was 53 million on 27 March 2011 when the census was taken, about 400,000 more than official estimates. In Wales the population was 3.1 million.
Around 55% of the 3.7m increase was attributed to net migration.
The census showed that the population for England and Wales was greater than official estimates rolled forward from 2001 - by just over 500,000.”

You can read more at: http://www.bbc.co.uk/news/uk-18838540  


July 14, 2012

Data Strategy Board (DSB) – Department for Business Innovation & Skills lists its members

Following hard on the heels of the announcement of the membership of the Open Data User Group (see 10 July), the Department for Business Innovation & Skills has news of the new Data Strategy Board: http://www.bis.gov.uk/transparency/data-strategy-board

If you follow the link to the note of the meeting on 11 July, you’ll see who has been appointed. The three places reserved for “data users outside the public sector” have gone to:

Chris Yiu, Head of Digital Government Unit, Policy Exchange
Bill Roberts, CEO, Swirrl IT Ltd
Steve Thomas, MD Strategic Development, Experian Plc

It is striking that (unlike Sainsbury’s representation on ODUG), the Department for Business has not included any representative of the huge existing community of business end-users. Rather, we have a Think Tank, and two Value-Added Resellers / Publishers – indeed, you might have spotted that Experian has seats on both ODUG and the DSB. 


July 10, 2012

Open Data User Group (ODUG) – Cabinet Office announces members – including Sarah Hitchcock of Sainsbury's

Following the earlier news (June 17) about the launch of ODUG, it’s really good to report that Sarah Hitchcock, Head of Network & Planning at Sainsbury’s, has been appointed as one of the 12 members.

Here’s the Cabinet Office’s announcement:

You’ll see that other members of the Group also include Bob Barr (Manchester Geomatics), and Mick Cory (Chair, UK Location User Group), plus a representative from Experian.

ODUG will add to the pressure that we’ve been applying since 1998 for more Open / Free Data. 


June 29, 2012

The Government's new Open Data White Paper: Unleashing the Potential

The Government has just published its “Open Data White Paper: Unleashing the Potential”

“……..which sets out how we’re putting data and transparency at the heart of government and public services. We’re making it easier to access public data; easier for data publishers to release data in standardised, open formats; and engraining a ‘presumption to publish’ unless specific reasons (such as privacy or national security) can be clearly articulated. From the Prime Minister down, central Government is committed to making Open Data an effective engine of economic growth, social wellbeing, political accountability and public service improvement.”

At the launch event yesterday (our DUG Award winner) Francis Maude gave an evangelical speech – the White Paper has a big ambition, and there’s no doubt that he means every word of it.

The link http://www.cabinetoffice.gov.uk/resource-library/open-data-white-paper-unleashing-potential leads to both the White Paper, and a document which gives each Department’s plans for releasing more data. 


June 28, 2012

Job vacancies

Several new job vacancies in DUG member companies can be seen at the tab to the left. 


June 25, 2012

2 reports: Deloitte's on Open Data, and the Royal Society on Open Science

The momentum in favour of open data continues to grow, and this is illustrated by two new reports:

1. Deloitte’s “Open Data – driving growth, ingenuity and innovation”

“Welcome to our first white paper in a series of research reports and white papers from Deloitte Analytics on the topic of ‘competing on insights’.
Open data – Driving growth, ingenuity and innovation has been researched and written in collaboration with Professor Nigel Shadbolt, head of the Web and Internet Science Group at the University of Southampton and co-director of the Government’s new Open Data Institute.
This publication presents Deloitte’s vision for open data – a subject that has, until now, been overshadowed unhelpfully by considerations of ‘big data’. Open data is about much more than improving government transparency and it is now emerging from its silent revolution……….”
You can see more at: http://www.deloitte.com/view/en_GB/uk/market-insights/deloitte-analytics/de0d882044ad7310VgnVCM3000001c56f00aRCRD.htm  

2. The Royal Society’s “Science as an open enterprise”

This report “aims to identify the principles, opportunities and problems of sharing and disclosing scientific information and asks how scientific information should be managed to support innovative and productive research that reflects public values.” The blockbuster 105-page report can be downloaded at:
Alongside this project, Research Councils UK in partnership with JISC, the Royal Society and Sciencewise-ERC, commissioned a public dialogue exercise on open data, aiming to engage the public with issues around open data to potentially inform future policies in this area. The report of this exercise is can be downloaded from the same link. 


June 22, 2012

Scotland – 2011 Census – News of plans for outputs, & timing

Following the news from ONS about its plans for outputs for England & Wales, and their timing (see news on April 3), National Records for Scotland have now released their 2011 Census Output Prospectus and set out a timetable for a phased release beginning with the first outputs in December this year. The dataset release phases are similar in content to those in England & Wales but will follow six months behind those planned by ONS.

You can see the full detail at: http://www.gro-scotland.gov.uk/files2/the-census/progress-reports/census-2011-outputs-prospectus.pdf 


June 20, 2012

ONS and UCL work on an updated 2011 Output Area Classification

Following DUG’s response to the consultation on plans for a new UK Output Area Classification (see news on 23 March), ONS has now published a summary of the responses: http://www.ons.gov.uk/ons/guide-method/geography/products/area-classifications/ns-area-classifications/new-uk-output-area-classification/index.html

You’ll see the conclusions & next steps which, taken in the context of DUG’s views, look very encouraging. 


June 18, 2012

ONS: "First results from the 2011 Census will be released on Monday 16th July"

Hot news from ONS………

These stats will be populations by age and [gender], and occupied households estimates for England and for Wales (just at Local Authority level).

Key and Quick statistics, including many simple counts at Output Area level (NB), plus postcode populations, are due to follow on between November 2012 – February 2013. 


June 18, 2012

New statistics from DECC on fuel poverty for small areas

Here’s a link to new data on fuel poverty in England from the Department of Energy and Climate Change: http://www.decc.gov.uk/en/content/cms/statistics/fuelpov_stats/regional/regional.aspx

You’ll see that you can request data right down to Output Area level.


June 17, 2012

Open Data User Group (ODUG) – Cabinet Office Introductory Event 14 June 2012

The Cabinet Office held an event last Thursday (14 June) to introduce the Open Data User Group (ODUG). The event attracted a big crowd, and you can see the presentation slides here

ODUG will add to the pressure for more Open / Free Data that we’ve been applying since 1998. 


June 8, 2012

ESRC's Retail Research & Data – 17 Master's projects now underway

The ESRC is seeking to increase its engagement with the retail sector (http://www.esrc.ac.uk/funding-and-guidance/collaboration/working-with-business/  ) and an element of this is the Retail Research and Data project that Professor Paul Longley of UCL and Keith Dugmore are running.

We are really pleased to report that 17 Master’s projects which have been specified by retail companies (most of which are DUG members) now have students working on them. You’ll see it’s a fascinating list of topics: http://www.esrc.ac.uk/funding-and-guidance/collaboration/working-with-business/retail-masters-student-projects.aspx

We are hoping that the ESRC will be asking retailers to specify more projects towards the end of this year. 


June 1, 2012

News from ONS about 2011 Census Geography Products – Boundaries and Lookup Files

ONS has just published its 2011 Census Geography Prospectus. This contains details of what and when geographic products (digital boundaries and lookup tables) will be released. When available, these will be available to download from the ONS website or supplied on hard media from ONS Customer Services.

The first products will be available in October 2012 (subject to the Census outputs for Output Areas being published on time in November) and the second will follow in January 2013.

There’s a summary here:
and you can download the Prospectus itself, and also a FAQ document, e.g:

Q: Will the output area (OA)/super output area (SOA) and workplace zone (WZ) boundaries be free, including for commercial re-use?
A: The Office for National Statistics (ONS) has submitted a request that these boundaries will be available as Open Data, under Open Government Licence (OGL) terms and conditions. This would make them available for free including for commercial re-use. We are awaiting the decision from this process, expected at the end of June 2012.

& “We don’t anticipate that more than 5% of the OAs/SOAs overall across England and Wales will change”.

The anoraks amongst us will also be pleased to see that there’s the option to have OA boundaries clipped to the coastline. 


June 1, 2012

Job vacancies

Several new job vacancies in DUG member companies can be seen at the tab to the left.


June 1, 2012

Information Commissioner's Office consults on new anonymisation code of practice

“The Information Commissioner’s Office (ICO) has begun a public consultation on a new anonymisation code of practice. The code will provide guidance on how information can be successfully anonymised and how to assess the risks of identification. The ICO has also launched a tendering process to establish a network of experts to share best practice around the release of data in an anonymised form. Anonymisation techniques can convert personal data into a form so that individuals are no longer identifiable. The consultation will be relevant to any organisation that wants to release anonymised data, for example under the government’s open data agenda.”

See more at: http://www.ico.gov.uk/news/latest_news/2012/ico-consults-on-new-anonymisation-code-of-practice-31052012.aspx 


May 23, 2012

News of the Open Data Institute from Nigel Shadbolt & Tim Berners Lee

The Open Data Institute will “study the best ways to exploit the growing amounts of data being made public by the government, & train people to use the data in business and advise on what kinds of information should be released.”

The ODI was first announced in the Autumn Statement and now its business plan has been given the go-ahead by the government. A total of £10m of public funds have been pledged over five years to get the ODI going, but the cash is contingent on a similar sum being pledged by businesses.”

You can find out more at: http://epsiplatform.eu/content/shadbolt-tbl-outline-open-data-institute-uk 


May 23, 2012

Chair of the Open Data User Group (ODUG) appointed – Heather Savory

Following up the news on 19 March about ODUG:

“The Minister for the Cabinet Office, Francis Maude, has appointed Heather Savory as the chair of the Open Data User Group (ODUG). Heather has a background in high-technology businesses and strong strategic and governance experience of central government, entrepreneurial businesses and consulting.”

You can read more at: http://www.cabinetoffice.gov.uk/news/chair-open-data-user-group-appointed 


May 15, 2012

Generating Value from the 2011 Census – MRS Census Geodemographics Group seminar – afternoon, 19th September 2012

Here’s news of a seminar “Generating Value from the 2011 Census – How business can benefit from the new free Census data” which will follow hard on the heels of the first release of 2011 Census stats (scheduled for July).

You’ll see that the speakers include the ONS, and that the price is very reasonable (& even lower if you book before 30 June). 


May 15, 2012

ONS 2011 Census – News of plans for outputs, & timing........... detailed comments from the GLA/SASPAC team

Following ONS’s publication of its plans for Census Outputs, the Greater London Authority / SASPAC team has pulled together some detailed comments.

The issues raised include postcode headcounts, urban & settlement areas, flows data, geographical products, and workplace geographies – all techie stuff, but they might come back & bite us in future unless they are discussed with ONS soon. 


May 5, 2012

Retail Sector growth April 2012 – presentation by the British Retail Consortium & the Association of Convenience Stores

The Department of Business Innovation & Skills held a Retail Growth Strategy workshop on 17 April 2012.
Here is a presentation by the BRC and ACS on the shape of, opportunities for, and challenges to the UK retail sector – you might find some of the stats & graphics useful. 


May 4, 2012

Job vacancies

Several new job vacancies in DUG member companies can be seen at the tab to the left.


May 1, 2012

Open Data – Report by the National Audit Office: "Implementing Transparency

The National Audit Office’s newly published report “recognises the strength of the strategic case for greater transparency, and highlights progress across government in fulfilling most of its initial commitments. However, government needs a better understanding of costs, benefits and use to assess whether transparency is meeting its objectives of increasing accountability, supporting service improvement and stimulating economic growth……”

Some key facts:

• £16bn (NB!) is the Government’s estimate of the value of public sector information to the UK economy in 2011
• 7,865 data sets were linked to the www.data.gov.uk website in December 2011
• 47m estimated number of visits made to the police crime map website between February and December 2011

You can find more here: http://www.nao.org.uk/publications/1012/implementing_transparency.aspx  


May 1, 2012

DUG’s letter to Bernard Jenkin MP: the private sector’s need for government statistics and data – potential issues for Inquiries

Following DUG members meeting Bernard Jenkin, MP, Chair of the House of Commons Public Administration Select Committee, here is a follow-up letter.

It highlights 3 areas for potential inquiries by the committee:

1. The government’s Trading Fund business model severely limits the use that can be made of existing data, and needs fundamental reform. (This is referring to Ordnance Survey, Valuation Office Agency, etc.).
2. The use of government administrative files to create new statistics needs to be accelerated. (e.g. HMRC data on income).
3. Government suppliers of data need to put more emphasis on “the last 100 yards”, making it easier for new users, particularly small businesses, to find and use data. 


April 30, 2012

UCL's Centre for Advanced Spatial Analysis – Conference highlights using scoop.it

UCL's Centre for Advanced Spatial Analysis held its Smart Cities conference on 20 April. Here’s a report:

& media coverage included stories on:
• Crunch commuter data to track changing communities
• Urban analysis specialists launch the City Dashboard, a hub for real-time data on major UK cities
• How data could help fix London's transport woes
• Meticulous map shows every London bus journey

You can see more at: http://www.scoop.it/t/spatial-analysis 


April 30, 2012

Scotland's 2011 Census – Update on Statistical Outputs

Following the news from ONS about its plans for outputs for England & Wales, and their timing (see news on April 3), National Records of Scotland has published the first in a series of quarterly progress notes designed to keep users up to date on the statistical outputs from the 2011 Census:

The headlines are:
• Table outlines will be published very shortly together with a note of any significant changes made since they were first published in June 2011
• First 2011 Census results are planned for late 2012 (NB: delayed from the original July date) – a full output prospectus will be published around the end of May
• A lot of effort is being put into producing compatible statistics across the whole of the UK wherever possible 


April 12, 2012

Government seeking a Chair of the Data Strategy Board

Following up the news of 19 March about the government’s plans for open data, it is now seeking a Chair for its new Data Strategy Board:

“Outstanding Chair to deliver better access to public data
The Government has taken significant steps towards extending access to public sector information. The new Data Strategy Board (DSB) will advise Ministers on what public sector data should be released, initially from the four Trading Funds that make up the Public Data Group (PDG): the Met Office, Ordnance Survey, Companies House and the Land Registry. The data from these organisations and others across the public sector has the potential to unlock growth and create new opportunities for high-value businesses of all sizes across the UK.
.……………. To help realise our ambitions we are now seeking an experienced Chair to lead DSB in its formative years……..” 

April 11, 2012

Centrica joins the Demographics User Group

Great news that Centrica has announced its joining DUG. Here’s the press release:

“Centrica is the latest company to join the Demographics User Group (DUG), whose membership already includes major companies across a range of sectors.

Centrica is a top 30 FTSE 100 company with energy businesses in the UK and North America. Upstream it sources, generates, processes, trades and stores energy. Downstream, it supplies gas and electricity to millions of homes and businesses and offers a distinctive range of home energy solutions and low carbon products and services.

Centrica’s Customer Insight team sits within British Gas’s Commercial Marketing directorate. Their activity spans a range of disciplines from Analytics & Modelling to Market Research and it strives to help the business make great commercial decisions in its role at the heart of the business.

Senior Analyst, Harry Harrison stated "We are passionate about putting the customer at the heart of our activity and the more we understand them, the better the chances of us being able to achieve this aim. We are very pleased to be joining DUG and we look forward to sharing our own experiences and learning from the experience of others across different industry sectors to facilitate this aim." 


April 3, 2012

Republic of Ireland – 2011 Census results

“This report is the first of two publications summarising the results of the 2011 census. This first one, This is Ireland - Part 1, looks at overall population change by county; it also examines age, marriage, households and families as well as including first results on nationality, foreign languages, the Irish language, religion and housing……..”

You’ll see that the report has lots of eye-catching maps & charts:

April 3, 2012

ONS 2011 Census - News of plans for outputs, & timing

ONS has now published its Outputs Prospectus, which sets out the release plans for the 2011 Census statistics in England & Wales:

You’ll find the detail at the tab on the left: “Release plans for 2011 Census statistics”. There are four releases
First release July 2012: Age and sex, and occupied households estimates for England and for Wales (just at Local Authority level)
Second release November 2012 – February 2013: Key and Quick statistics (including many simple counts at Output Area level (NB), plus postcode populations)
Third release March 2013 – June 2013: Local Characteristics (including more detailed OA tables)
Fourth release July 2013 – October 2013: Detailed Characteristics, Detailed Theme and Armed Forces
“Subsequent releases” will include Workplace (& possibly Daytime) population.

There’s a tab on the right “Prospectus tables and schedule” to see a spreadsheet which gives lots of detail.

Scotland and Northern Ireland will be releasing their own prospectuses later this month.

March 30, 2012

Mary Portas High Street plans get government go ahead

“The government has announced it has accepted "virtually all" 28 recommendations from Mary Portas in its bid to rejuvenate UK High Streets. These will include creating dedicated "town teams" to manage High Streets and making parking more affordable. It has also unveiled additional measures, including funding schemes and cutting back on red tape……..”.

You can read more at: http://www.bbc.co.uk/news/business-17554053

& the official source: http://www.communities.gov.uk/news/corporate/2120114   

March 29, 2012

Ofcom publishes more detailed Broadband Availability data for England

DUG’s Data Manifesto (see February 6 below) includes the wish for data on Broadband coverage, & here is some news:

“Following a request from the UK governments Department for Culture, Media and Sport (DCMS), Ofcom, the communications regulator, has today published some of its “early” 2011 Infrastructure Report broadband data at a “more detailed geographical level“. It’s for England, and at Local Authority level. Here’s the link:

And the Ofcom (July 2011) report is here:

March 23, 2012

Output Area Classification (OAC) for 2011 – User Consultation – DUG's Response

ONS and University College London (UCL) have been consulting on plans for a new UK Output Area Classification:


The original classification was created for ONS by Dr Daniel Vickers at the University of Leeds using 2001 Census data (it won DUG’s first Award in 2005). Our response to ONS’s consultation is here

March 19, 2012

"Open Data – resist Sir Humphrey’s last stand" – article by Michael Cross

Michael Cross, founder of the Guardian’s “Free Our Data” campaign (& winner of our Award in 2006), is continuing the battle:

“You wait all year for a think tank to report on how government should embrace Big Data, then two do at once. Over the past week, both Demos and the Policy Exchange have published studies on the opportunities created by David Cameron’s pledge to create the world’s most transparent government.
One urges the government to get on with it; to create its promised “right to data” and let the private sector and concerned citizens do their best with it.
The other counsels caution - the transparency revolution will happen only when government departments are equipped with new IT systems to collect and disseminate accurate data, and civil servants and the general public educated to make sense of it.
Wonk-watchers will guess which philosophy emerged from which think tank. Anyone who has worked with government will guess which approach the civil service and wider public sector will find more congenial……..”

Read on at: http://www.thecommentator.com/article/988/open_data_resist_sir_humphrey_s_last_stand


March 19, 2012

Government response to Consultation on Public Data Corporation; also Terms of Reference for Data Strategy Board, Public Data Group, and the Open Data User Group (ODUG)

The Government has now published its response to ‘A Consultation on Data Policy for a Public Data Corporation'. The views that we expressed on 11 October (see below) were shared by many of the 412 respondents.

 The Government has also published the terms of reference for the new Data Strategy Board (DSB) and the Public Data Group (PDG); “This marks a significant step towards embedding a culture of transparency and better access to Government data”.

You can see details of both at:

The Cabinet Office is also setting up Open Data User Group (ODUG). “This Group will give data users and re-users an opportunity to influence the release of Government data”: http://www.cabinetoffice.gov.uk/content/open-data-user-group  

All this will require continuing effort on our part, but it’s good to see that the tide is still flowing strongly in the direction of more open / free data.

March 14, 2012

More free data, signposted by Data4nr

The latest update from http://www.data4nr.net/introduction/  includes business stats for small areas; various health indictors; and road distances to food shops, GPs, Post Offices, etc.


March 14, 2012

2011 Census Evaluation

Here’s a link to information about ONS’s evaluation programme in general which also links to a download about internet completion take-up:


And another relates more to the 2011 Census campaign itself, including an insight into how ONS selected the languages for the translation leaflets:



March 7, 2012

Francis Maude speech to Information Commissioner's conference

Francis Maude gave a speech to the Information Commissioner’s conference yesterday, 6 March 2012. The draft text is on the Cabinet Office web site at:

He spoke on: Transparency; Data Sharing; Fraud, Debt and Error; Identity Assurance; Census; and Privacy. In particular:

Data Sharing:
“In May we will publish proposals that will make data sharing easier – and, in particular, we will revisit the recommendations of the Walport-Thomas Review that would make it easier for legitimate requests for data sharing to be agreed with a view to considering their implementation.”

“And the census is another area where I want to bust the myths around the complexities of data sharing.
I find it incredible that we haven’t fundamentally changed the way we approach the census since it began in 1801.
We are currently supporting the work by the UK Statistics Authority to look at census alternatives that will be cheaper and easier for the customer without compromising the privacy of the information. Technology has a critical part to play in enabling this.”

“I want us to reach a position where the citizen is driving the disclosure of their personal data wherever possible – with effective mechanisms so they can truly decide what they want to consent to and be able to change their mind.”


March 3, 2012

ONS's 2011 Census Newsletter – February 2012

Here is ONS’s latest Census Newsletter. You’ll see that it includes items on Outputs, Analysis, Web Data Access (including an unexpected mention of the Retail Sales Index) an application programming interface, and also some Roadshows in April:

 “ONS will hold a series of roadshows in April 2012 to keep stakeholders informed of progress to deliver the 2011 Census outputs and the planned web data access channels. There will be a demonstration of the planned tools and how these will help users to access and exploit ONS data. There will also be a workshop session to seek your feedback about the opportunities and challenges presented by the planned web dissemination tools. Further information about these events and how you can register, will be issued in March 2012.

Locations & Dates:
• Manchester 17 April
• York 18 April
• London 19 April
• Birmingham 25 April
• Newport 26 April
• London 27 April”


March 2, 2012

Press Coverage of ONS's "Measuring Well-Being"

Wednesday’s papers gave extensive coverage to ONS’s efforts to measure well-being, for example:

“This must be a heartening week for those wallowing in self-pity. Suddenly everyone seems to care how happy we are. Just as a campaign was launched yesterday to dissuade urban elderly people from retiring to the countryside, lest among other horrors they find themselves with neighbours resembling a cross between Nick Griffin and Margo Leadbetter, the Office for National Statistics also published a comprehensive report on who the happiest people in Britain are, and where they live”

Guardian: http://www.guardian.co.uk/commentisfree/2012/feb/29/ons-happy-scots-northern-irish

& the BBC: http://www.bbc.co.uk/news/uk-17192780

You can find ONS’s detailed report as a pdf at:


February 18, 2012

"The Changing face of poverty" – The Daily Mail publicises Charles Booth's famous map of London

A DUG member (who wishes to remain anonymous) tipped me off about an article in the Daily Mail:

“The changing face of poverty: Updated maps of London's poorest areas show epidemic of 'junk food' diabetes in same streets where Victorians died of malnutrition”:

It features Charles Booth’s famous map, which has been the regular Award at our annual Conferences
If you are interested in Booth’s maps, there’s more at the LSE’s Online Archive: http://booth.lse.ac.uk/


February 6, 2012

DUG’s Data Manifesto

Here is a manifesto of DUG members’ needs for data from government, which has been prepared for the UK Location Programme.


February 4, 2012

Tim Kelsey appointed as the new Executive Director for Transparency and Open Data

The Cabinet Office has announced the appointment of Tim Kelsey as the new Executive Director for Transparency and Open Data within its Efficiency and Reform Group (ERG).

Tim gave the Keynote address “Open Data: Making it Real” at our Conference in October: http://www.demographicsusergroup.co.uk/resources/Tim+Kelsey+DUG+2011.pdf  


February 4, 2012

Job Vacancy – Boots

The latest job advertisement – Boots in Nottingham seeking a Portfolio Analyst – can be seen at the tab to the left.


February 4, 2012

National Statistical Offices: Independent, Identical, Simultaneous Actions Thousands of Miles Apart – a view from the USA

The Director of the US Census Bureau has written about initiatives that are common across Anglophone statistical offices around the world:

1. The volume of data generated outside the government statistical systems is increasing much faster than the volume of data collected by the statistical systems; almost all of these data are digitized in electronic files.
2. As this occurs, the leaders expect that relative cost, timeliness, and effectiveness of traditional survey and census approaches of the agencies may become less attractive.
3. Blending together multiple available data sources (administrative and other records) with traditional surveys and censuses (using paper, internet, telephone, face-to-face interviewing) to create high quality, timely statistics that tell a coherent story of economic, social and environmental progress must become a major focus of central government statistical agencies.
4. This requires efficient record linkage capabilities, the building of master universe frames that act as core infrastructure to the blending of data sources, and the use of modern statistical modeling to combine data sources with highest accuracy.
5. Agencies will need to develop the analytical and communication capabilities to distil insights from more integrated views of the world and impart a stronger systems view across government and private sector information.
6. There are growing demands from researchers and policy-related organizations to analyze the micro-data collected by the agencies, to extract more information from the data.”

You can see the full article at: http://blogs.census.gov/directorsblog/2012/02/national-statistical-offices-independent-identical-simultaneous-actions-thousands-of-miles-apart.html


February 2, 2012

Land Registry releases data free of charge

Following the mention of Land Registry Data in the Autumn Statement (see December 6 below), here’s some more news:

“The UK Land Registry as of 30 January is giving access to their monthly transaction data free of charge. This is to be followed by the release of 'Price Paid' data in March 2012”.

There’s more detail at: http://epsiplatform.eu/content/uk-land-registry-releases-data-free-charge  


February 2, 2012

Cabinet Office Open Data consultation – summary of responses

Following our submission to the Open Data consultation (see October 11 below), the Cabinet Office has now published a summary of responses:

“There was a very positive response to the consultation. Launched in August, there were nearly 500 responses submitted before its closure in October.

The consultation set out a series of questions aimed at informing Government’s strategy to embed transparency and open data as core operating principles of the public services. Many of you expressed your support for the principles of open data, your confidence in its ability to bring about a positive change and the benefits it can deliver, including enhancing the performance and delivery of public services and fostering economic growth.”

In short, encouraging stuff.

You can read the detail at:


January 29, 2012

Living Costs and Food Survey – Users' meeting – Tuesday 20 March 2012

Various DUG members are interested in the ONS’s Living Costs and Food Survey (formerly the Expenditure & Food Survey), which is now coded with ONS’s OAC geodemographic classification.

Here’s news of a meeting:
“…….organised by the Economic and Social Data Service, which will provide a forum for data users and producers to meet and discuss new developments and exchange information about the Living Costs and Food Survey. The programme contains a mixture of papers from data producers and researchers. The meeting is free to attend and lunch is provided.

To view the programme and book a place please go to www.ccsr.ac.uk/esds/events/2012-03-20/  ”


January 24 , 2012

Eurostat's new vision for European Statistics

The National Statistician organised a meeting on 18 January which featured Walter Radermacher, Chief Statistician of the EU, talking about “A new vision for European Statistics” (here) to a large audience from ONS and departments such as DWP, HMRC, etc.

Keith Dugmore was invited to give the user view on the panel session, & had a chance to:

• Warn of the danger that EU officials might get an inside track on priorities in the same way as Government Departments do in the UK.
• Welcome the focus on making more use of admin data, better dissemination, and the importance of political support for Open Data in changing culture.
• Make the case for more stats at small area level, plus the necessary boundaries & mapping being freely available.


January 17 , 2012

Mapping of Urban Sprawl using open data from www.data.gov.uk

Blair Freebairn is exploring Open Data, generating new urban sprawl boundaries.

Here is a presentation which defines the current problem, shows what he’s done & why it’s better, and gives some examples.

If you would like more information, email blair.freebairn@geolytix.co.uk  


January 17 , 2012

ONS's "Beyond 2011" User Requirements Consultation – DUG’s Response

Following the earlier news item (October 27) here is DUG’s response to ONS.


January 11 , 2012

Sub-regional public and private sector employment in the UK

Here’s a new article on the rarely-discussed topic of analysing public and private sector employment in different sub-regions of the UK.
“The level of public sector employment in the UK economy has increasingly become a policy issue over recent years. To help inform the debate, ONS produces quarterly public sector employment data that measure the total level of public sector employment at the national and regional level. At present, however, there have been relatively few data releases or articles examining the sub-regional distribution of public sector employment. Furthermore, this distribution is of interest to policy makers both nationally and locally. This article seeks to fill this gap by providing an overview of some of the data available from ONS that can be used to examine the sub-regional distribution of public sector employment and also the sub-regional distribution of private sector employment.”


January 8, 2012

Letter to the Rt. Hon. Francis Maude MP - "The National Address Gazetteer: being run in the interests of Ordnance Survey, to the detriment of citizens and the private sector"

You can see this letter from DUG to Francis Maude here

In particular:
“...........For anyone outside the public sector, the prices for GB coverage and 100+ users range between £130,000 p.a. for the basic version to £189,000 p.a. for the full product. Even a single user would have to pay £24,000 p.a. These prices are prohibitive for almost all potential customers, and appear designed to protect OS’s existing policy of setting high prices for a small captive market.”


January 7, 2012

ONS is seeking users' involvement in designing its website for delivering 2011 Census outputs

ONS is currently developing its website to provide access to 2011 Census outputs, which are planned to start in July. It is seeking users’ views during January & February.

If you would like to learn more, please contact julie.peters@ons.gsi.gov.uk before the end of January.


January 6, 2012

RSS Open Meeting "Measuring Well-Being" - 8th March 2012, 5PM- c.7PM

Several DUG members have expressed interest in this topic: here’s a chance to catch up on developments at a 2-hour meeting after work – see the details here


January 5, 2012

Job Vacancies – Sainsbury's (closing date 22 January 2012)

Here's the latest job advertisement, with Sainsbury’s seeking Senior Location Analysts:

Earlier Job Vacancy announcements can also be seen at the tab to the left.


January 2, 2012

Happy New Year........... & here's a summary of our all activities in 2011

Looking back on the past year, you’ll find a summary of DUG’s activities during 2011 here

As you’ll see, it’s been another busy and successful year, with members sharing their experience in numerous ways, supporting the ONS in its case for what proved to be a successful Census, and the government both investigating the alternatives to another traditional Census, and continuing to promote the policies of Open Data: these initiatives led to Francis Maude, Minister for the Cabinet Office, receiving this year’s DUG Award at the Royal Society in October.


December 17, 2011

Andrew Dilnot confirmed as the new Chair of the UK Statistics Authority

Andrew Dilnot – a great advocate of making statistics accessible to wider audiences – has been appointed as the new Chair of the UK Statistics Authority, succeeding Sir Michael Scholar.

His “brilliant performance” before the Public Administration Select Committee can be seen at:

The appointment has now been endorsed by the House of Commons, and Andrew will take up the post in April.


December 17, 2011

Google Public Data Explorer

If you are not already aware of Google’s Public Data Explorer, here’s some news:

“The Google Public Data Explorer makes large, public-interest datasets easy to explore, visualize and communicate. As the charts and maps animate over time, the changes in the world become easier to understand. You don't have to be a data expert to navigate between different views, make your own comparisons, and share your findings.

Students, journalists, policy makers and everyone else can play with the tool to create visualizations of public data, link to them, or embed them in their own webpages. Embedded charts and links can update automatically so you’re always sharing the latest available data.”

You can find out more at: http://support.google.com/publicdata/?hl=en

And some examples from around the world at: http://www.google.com/publicdata


December 14, 2011

News of further useful statistics from DWP

Amongst all the free datasets flowing from government, and publicised on the very useful http://www.data4nr.net/  website, the following may be of particular interest:

National Insurance Number (NINo) allocations to overseas nationals entering the UK. These are only at Local Authority District level, but do give the countries of origin for the c.700,000 people who arrived in 2009. http://www.data4nr.net/resources/1112/

Workless benefit claimants at Output Area level (Updated for quarter 2 2011). These are unusual, in being down at Output Area level. http://www.data4nr.net/resources/1408/


December 6, 2011

M&S – Steve Bond appointed as Director of the Customer Insight Unit and Loyalty

DUG members will be delighted to learn that Steve Bond has been promoted to Director at M&S.

As many of you will know, Steve was one of the founders of the group in 1998, and has been an active in encouraging and guiding our development ever since.

His new role will further add to DUG’s influence, especially within government.


December 6, 2011

Autumn Statement – "The Government is making available for free a range of core reference data sets" – further news

Following up the news (29 November) about the Chancellor’s Autumn Statement, you’ll find most detail about the new free datasets in this 13-page document:



November 29, 2011

Autumn Statement announcement - "The Government is making available for free a range of core reference data sets"

Here below is news about government moves on open (free) data – including Land Registry, Companies House, & the Met Office – and also the Public Data Corporation.

The tide is still flowing in favour of making more data free (although it’s not yet a total victory).

“The Government has today announced that:
To support the growth of high-value data businesses and make access to data easier for startups, the Government is making available for free a range of core reference data sets. In addition it is announcing the creation of a Data Strategy Board and a Public Data Group which will maximise the value of data the public sector buys from the Met Office, Ordnance Survey, Land Registry and Companies House.

Delivering on its commitment to establish a Public Data Corporation, Government has announced the establishment of a Data Strategy Board (DSB) which will seek to maximise the value of data from the Public Data Group (PDG) of Trading Funds for long-term economic and social benefit, including through the release of data free of charge.

Sending a clear signal of the DSB’s mandate, Government is announcing the release of additional core reference datasets for unrestricted use from the PDG, including, for the first time, weather observation and detailed weather forecast data and core data from the Companies Register.

The PDG currently includes Ordnance Survey, Met Office, HM Land Registry and Companies House. The Group will identify and deliver efficiencies and synergies to reduce the cost of data for users and re-users of data and provide additional funding for making data freely available.

This change clearly separates the commissioning and provision functions of public data, rebalancing the incentives to release more data for free, as well as strengthening the capability of Government to commission data for its own needs.
This announcement signals a significant step towards making additional core reference data from the Met Office, Ordnance Survey, HM Land Registry and Companies House available and free at the point of use

For further information on the wider announcement please see the following link:

http://www.cabinetoffice.gov.uk/news/open-data-measures-growth-review  “

November 12, 2011

Open Data initiative moves into the world of consumers' personal data – "Midata"

The Government has announced a ground-breaking partnership with 26 organisations to deliver a new era of consumer empowerment. The businesses, consumer bodies and regulators involved are all committed to working with Government to achieve its vision for “midata”, launched today. And all are endorsing the key principle that data should be released back to consumers. “midata” is a voluntary programme the Government is undertaking with industry, which over time will give consumers increasing access to their personal data in a portable, electronic format..........”

You can read more about it in a press release from the Department for Business Innovation & Skills: http://www.bis.gov.uk/news/topstories/2011/Nov/midata  which includes a list of 19 companies (including E.ON) who have committed so far.

November 2, 2011

DUG’s annual conference: presentations, & the winner of our 2011 Award for “Better information from government” is…………Francis Maude, Minister for the Cabinet Office

DUG’s annual Conference, held at the Royal Society on 12 October, was a great success.

The presentations are now on the website (see tab to the left).

Francis Maude, Minister for the Cabinet Office, came to collect our Award “for his enthusiastic promotion of the policies of Open Data, and investigating alternatives to another traditional Census”. The citation reads:

“Commercial companies have been making the case for better access to government datasets for more than a decade. Public pressure has grown, but it is vital to have strong ministerial direction to make progress. Francis Maude is giving this, vigorously promoting both better access to existing government data, and also the exploration of new sources which would make another traditional Census unnecessary. Members of the Demographics User Group strongly welcome these developments, which will increase the efficiency of services to their customers, and fuel innovation.”

The event was reported in a Cabinet Office Press Release:


November 1, 2011

Neighbourhood Statistics Update (Datasets, New Atlas of Deprivation, Local Profiles, plus Rural/Urban info)

Here’s the latest news from ONS’s Neighbourhood Statistics team:

New Datasets
• Tax Credit Claimants 2006, 2007, 2008 (Lower Layer Super Output Area)
• Resident Population Estimates by Broad Age Band, 2010 (Middle Layer Super Output Area)
• Benefits Data, February 2011: Working Age Client Group; Incapacity Benefit/Severe Disablement Allowance Claimants (Lower Layer Super Output Area)

News and Information
• The Atlas of Deprivation (based on the 2010 Indices of Multiple Deprivation) is now available as a download from the NeSS website.
• An update to the Local Authority Profiles is now available. These profiles help authorities better understand the economic, social and environmental picture of their area.

Also, on urban / rural matters:
• Podcasts – two video presentations on rural and urban areas have been released: one summarises an article on measuring the differences, the other gives guidance on using and mapping rural/urban classifications.
• A google map showing the Rural Urban Definition for England and Wales is now available. The map is based on the definition used in a recent Regional Trends 43 article and can be navigated by typing in an address or postcode.
• A user survey on the proposed options to update the urban areas dataset and rural-urban definition classification for England and Wales has been launched and will run until 25 November. [see KD’s next message below]



October 27, 2011

ONS's plans to revise urban area boundaries and the rural-urban definition/classification following the 2011 Census

A project is considering how the urban areas dataset and rural-urban definition/classification for England and Wales might be updated following the 2011 Census to ensure that they remain relevant and fit for purpose. ONS has a “user engagement document”:


.............which they suggest reading before filling in their survey.


October 27, 2011

ONS's Launch of the "Beyond 2011" – alternatives to another Census – User Requirements Consultation

As we await the first results of the 2011 Census next July, here's news of "Beyond 2011" – ONS's investigation of whether other data sources might be used instead of holding another traditional Census.

This consultation is focusing on the needs of users of information (rather than possible sources of data). You'll see that there's a consultation document, & also some workshops.

Also, by way of background, you might be interested in the attached paper "Beyond the 2011 Census in the United Kingdom, with an international perspective", written by four members of the MRS's Census Geodemographics Group.


October 26, 2011

The demand for, and feasibility of, a UK-wide Index of Multiple Deprivation – UKSA report

This topic had arisen at DUG meetings – with interest in not only those areas that are most deprived, but also implicitly those at the other end of the spectrum too.

I’ve just spotted that it has recently been investigated by the UK Statistics Authority, which has published this Brief.

The UKSA’s headline conclusion is “At the current time, the Authority does not see a sufficiently strong case for the development of a UK-wide IMD by government to justify the additional resources needed, but we have concluded nonetheless that some further steps would help meet the needs of users with an interest in deprivation across the UK.”

October 23, 2011

Presentation slides from ONS's 2011 Census September roadshows

Here's a message from ONS which starts:
“I am confident that we have achieved a response rate of 94 per cent nationally and over 80 per cent in every local authority. Across London, the estimated response rate in most London boroughs is between 5 and 15 percentage points better than in 2001, and across the capital as a whole I am also confident we have reached a response rate of over 90 per cent.” - Glen Watson, 2011 Census Director - July 2011

How this was achieved, what happened in the field operations, and what happens between September 2011 and the delivery of the statistics were all questions answered in a series of nationwide roadshows held in September 2011. Topics included:
• the field operations challenges and successes (this includes info on the address register)
• from questionnaire to datasets - ensuring the quality of census information
• delivering the census data: how, when and what; what's next?

Here’s a link to the presentations:


October 23, 2011

Retail research: (1) Geography of e-commerce; (2) Deep discount food retailing

Following DUG’s September meeting, Professor Martin Clarke at the University of Leeds has sent news of retail research: (1) Geography of e-commerce; (2) Deep discount food retailing


October 11, 2011

Cabinet Office Consultations – Open Data, and the proposed Public Data Corporation - DUG's responses

Following the earlier news (see August 4 news) here are DUG’s responses:
Open Data
Public Data Corporation

October 9, 2011

"There's a wealth of data out there – why not let us use it?" – article in the Guardian

It’s good to see yet more support for freeing up government data (especially postcodes) from Ben Goldacre in his latest article in the Guardian (8 October):

“Bad things happen when problems are protected by a forcefield of tediousness. Here is an example. Data is the fabric of the modern world: just like we walk down pavements, so we trace routes through data, and build knowledge and products out of it. The government has lots of data that has already been collected, because it has needed it to run the country properly: simple stuff like maps, postcode areas, land ownership, procurement data, endless weather readings, and so on.

Right now a fight is happening in Whitehall, with two factions in government: one group thinks we should give this data away for free, as a matter of principle, because it will make good things happen; the other thinks we should restrict access, and sell it. A consultation is under way. Despite a positive ministerial introduction, each of the three options it gives for releasing data is foolishly restrictive. Here's why that's a problem.......”

Read more at: http://www.guardian.co.uk/commentisfree/2011/oct/07/wealth-of-data-locked-away


October 3, 2011

"The time is right for a single national address database" – Opinion in "Database Marketing"

Peter Sleight, Chair of the Association of Census Distributors, is pushing the case for this in an excellent article in September’s Database Marketing.


& click through to page 12.


September 21, 2011

ONS Census News Alert - Workshops on 2011 specialist outputs

Here’s news from ONS about four workshops taking place in October & November:

To date consultations on the design of 2011 Census Outputs have concentrated on the main statistical series of outputs. The 2011 Census design team are now planning to hold four workshops on more specialised products and services. These are:

Small populations (concentrated in particular areas) – 24th October 2011

Analytical products – 25th October 2011

Origin and Destination outputs – 26th October 2011

Microdata – 11th November 2011

If you interested in attending one or more of the events, please see:
http://www.ons.gov.uk/ons/guide-method/census/2011/the-2011-census/census-consultations/planned-events/index.html u


August 30, 2011

Launch of ONS's new website

ONS’s has gone live at http://www.ons.gov.uk/ons/index.html

The headlines are:

• Information published on the new website. There is a 'Welcome to the new ONS website' section in the carousel at the centre of the website home page. This includes tips on finding content, plus information on how the website has changed and why ONS made those changes.

• Tutorials on YouTube. Tutorials on finding information on the new website are available at www.youtube.com/ONSstats  . These include tips on searching, finding publications, using Key Figures and browsing to find statistics on a particular topic.

• Census. You can access census information from the Guidance and methodology tab and also from the Related links panel on the home page.

• Support for users – Customer Contact Centre. Help is available by emailing info@statistics.gov.uk  or phoning 0845 601 3034. If you wish to send any feedback on the new site, this should also be sent to info@statistics.gov.uk .


August 18, 2011

"Big data: The next frontier for innovation, competition, and productivity" – report by McKinsey

You may find this report a heartening read:

“The amount of data in our world has been exploding and analyzing large data sets—so-called big data—will become a key basis of competition, underpinning new waves of productivity growth, innovation, and consumer surplus, according to research by MGI and McKinsey's Business Technology Office. Leaders in every sector will have to grapple with the implications of big data, not just a few data-oriented managers. The increasing volume and detail of information captured by enterprises, the rise of multimedia, social media, and the Internet of Things will fuel exponential growth in data for the foreseeable future........”


“Analyzing large data sets—so called big data—will become a key basis of competition, underpinning new waves of productivity growth, innovation, and consumer surplus as long as the right policies and enablers are in place. Research by MGI and McKinsey's Business Technology Office examines the state of digital data and documents the significant value that can potentially be unlocked. However, companies and policy makers must tackle significant hurdles to fully capture big data's potential – including a shortage of skilled analysts and managers.....”
You can download the report at: http://www.mckinsey.com/mgi/publications/big_data/index.asp


August 11, 2011

AGI’s conference, 20-22 September 2011, Nottingham

Here’s news of the Association for Geographical Information’s GeoCommunity '11 event:

“AGI GeoCommunity is the largest and most comprehensive independent conference in the UK digital mapping and data calendar. The conference provides tangible insight and leadership in current geographic information and location based issues via a range of keynote addresses and conference papers, a showcase exhibition, as well as by hands on training and face to face delegate networking. The event is already a 'must attend' for decision makers, practitioners and thought leaders from all aspects of the public and private sectors.”

You can find out more at http://www.agigeocommunity.com/  which includes the Conference Programme and its multiple streams, and also Delegate Fees and Registration. 


August 8, 2011

MRS seminar on 2011 UK Census – 4 July 2011: presentations now available

You can now see the presentations from this seminar (see earlier details below, 20 April) at: http://www.mrs.org.uk/networking/cgg/july2011_census.htm

They include news from ONS, Annette Dellevoet’s presentation on location planning at Sainsbury’s, and also Martin van Staveren’s detailed review of 2011 Census questions and the changes since the Census in 2001 (which will be very useful for reference when we start to get results later next year). 


August 4, 2011

Two Cabinet Office Consultations launched today – Open Data, and the proposed Public Data Corporation

Today the Cabinet Office has launched two consultations relating to government data.

1. Open Data policy in UK Government: see www.data.gov.uk/opendataconsultation

2. Data policy for a Public Data Corporation (from the Cabinet Office & BIS): see http://discuss.bis.gov.uk/pdc/

You can see a video of a short upbeat presentation by Francis Maude in the Cabinet Office press release:

It’s great to see such momentum behind this, although the minister does mention that there will be charging for some data. 


July 29, 2011

Google maps adds London Transport directions

“Google Maps has now included public transport directions for London travel on its website and mobile app. The directions on the web will help guide you from A to B in London with step by step directions, including which Underground station or bus stop to go and how long each step of your journey will take. Using the mobile app you are able to get directions from your current location using the GPS on your phone, meaning you can constantly stay on track in the British capital..............”.

See more at:


July 29, 2011

ONS's Census News Alert – Latest Update, + roadshows in September

Here’s the latest news from ONS about its plans for 2011 Census Outputs:

http://www.ons.gov.uk/census/2011-census/produce-deliver-data/2011-census-outputs  It includes:

* Analysis of the recent consultation on Outputs. We had expressed our views on May 12 – see below.

* The proposed running order for outputs. The first statistics – at Local Authority level – are expected in July 2012, followed by univariate counts at Output Area level as the next priority, which is good news.

ONS is also planning a series of roadshows throughout England and Wales in September 2011:
"To give all our stakeholders an opportunity to ask questions, find out what happens next and obtain details of the release of census data. Roadshow presentations will be published after the events have been completed. The attached flyer gives full details, including locations of what should be a series of very informative events. Book now to ensure your participation." 

July 21, 2011

News from Data4nr of more free datasets – Accessibility (to Supermarkets, GPs, etc)

Data for Neighbourhood Renewal continues to announce more new datasets:

The ones which are likely to catch the eye of DUG members are those on Accessibility from the Department of Transport, e.g. http://www.data4nr.net/resources/836/ 


July 14, 2011

Census 2011: Conference on its impact and potential, Manchester, 7- 8 July 2011 - Presentations

Last week’s Census conference in Manchester had lots of speakers across a wide range of topics.

Most of the presentations are now on the web at: http://www.ccsr.ac.uk/sars/conference2011/


July 13, 2011

ONS Income Estimates – Map mashup - enter a postcode.........

ONS is now presenting its small area Income estimates as a map mashup: http://www.neighbourhood.statistics.gov.uk/HTMLDocs/incomeestimates.html

There’s a further link to “Download data and more information”, which includes “These estimates provide the average household income for small areas within England and Wales. It is the income a household receives from wages and salaries, self-employment, benefits, pensions, plus any other sources of income. These estimates are produced at Middle Layer Super Output Area (MSOA) level.”

Our ex-ONS friend Chris Denham has commented: “This map is a breakthrough for ONS as a mash up of National Statistics with Google maps – with an underlay of streets, plus Street View for a 'walk through' of any locality (drag the yellow pegman), or satellite imagery. The map is searchable by postcode or place name, and has a choice of four measures of household income. Clicking on an MSOA will create a pop up of data for the area.”



July 10, 2011

Government to publish new data on health, schools, courts and transport

The Cabinet Office has announced (7 July) the release of more key data on the National Health Service, schools, criminal courts and transport:

“This represents the most ambitious open data agenda of any government anywhere in the world. The new data will reveal clinical achievements and prescribing data by individual GP practices, the performance of hospital teams in treating lung cancer and other key healthcare conditions, the effectiveness of schools at teaching pupils across a range of subjects, criminal sentencing by each court, and data on rail timetables, rail service performance, roadworks, current road conditions, car parks and cycle routes in an open format for use by all.”

More details are given at: http://www.cabinetoffice.gov.uk/news/government-publish-new-data-health-schools-courts-and-transport

Amongst the list of datasets, the following caught my eye as of potential interest to various DUG members:

• Prescribing data by GP practice to be published by December 2011.
• Information about schools & their pupils.
• Data on current and future roadworks on the Strategic Road Network to be published from October 2011, and subject to consultation to extend this during 2012 to Local Authority Streetworks Registers maintained under statute.
• All remaining Government-owned free datasets from Transport Direct, including cycle route data and the national car park database to be made available for free re-use from October 2011.
• Real time data on the Strategic Road Network including incidents, speeds and congestion to be published from December 2011. Office of Rail Regulation to increase the amount of data published relating to service performance and complaints by May 2012.
• Rail timetable information to be published weekly by National Rail from December 2011.
• Government procurement card spend data

…… and there may well be other nuggets of value to you.


July 10, 2011

MP's turn down the nomination for the next Chair of the UK Statistics Authority

Here’s news from the Financial Times on 7 July of the House of Commons Public Administration Select Committee rejecting the civil service’s nomination of a successor for Sir Michael Scholar as the next Chair of the UK Statistics Authority:

Bernard Jenkin, the Conservative chairman of the committee, said: “The panel appointed to make the selection seemed to be very much like the regulated choosing their regulator.” The Independent reported that MPs had questioned whether the nominee – Dame Janet Finch – would be a "stooge" for the Government.

It’s the first time such a thing has happened, and could signal that the new Chair will have more independence, and greater influence over the producers of statistics across government.


July 5, 2011

Good news of 2011 Census response rates / coverage in England & Wales

ONS’s Glen Watson, Census Director, spoke in public for the first time yesterday about response rates:

• Confident of hitting the 94% national response rate target [similar to 2001]
• Over 80% in every Local Authority [important both technically & politically – Kensington & Chelsea was only c.65% in 2001]
• > 90% in Greater London as a whole [better than last time, reflecting the benefits of targeting]
• The voluntary (NB) Census Coverage Survey has a c.90% response. [Amazing, and really important when ONS comes to estimate & impute those people who were missed].

All in all, very encouraging for ONS coming up with plausible final numbers.


July 2, 2011

2011 Census Outputs – Scotland

Following our response to the Scottish Census Office’s outputs consultation (see below, 3 April), here below is news of the detailed tables that they are planning to produce:

You’ll also see that the “General Register Office” has now been joined into the new “National Records of Scotland”.


July 2, 2011

2011 Census – successful data collection - & ONS's planning for outputs

With the Census data collection apparently having been carried out successfully (we should get more news next week),

ONS has just announced its current plans for outputs:

A proposed running order for outputs is due this month, with a full outputs prospectus & release timetable in March 2012.

The first statistics – at Local Authority level – are expected in July 2012, followed by Key Statistics at Output Area level as the next priority (which we’ll welcome) later in 2012.

Meanwhile, consultation continues on some of the more specialist outputs, including workplace statistics, and commuting & migrant flows.


July 2, 2011

Francis Maude's speech to the CBI annual business summit – further commitment to Transparency & access to public data

The text of a recent speech by Francis Maude, minister for the Cabinet Office, to the CBI, includes:

“The other side to the Transparency agenda is about boosting the economy. Firms in areas as diverse as life sciences and insurance have told us that better access to public data would help them grow their businesses. And dozens of bright new apps and websites have sprung up on the basis of public information published so far. A new study from McKinsey estimates that across Europe the enterprise and productivity benefits of open public data assets could be worth 250bn euros a year. Look across the Atlantic at the multi-billion dollar American health informatics industry, and that doesn't seem far-fetched.”

“The Prime Minister will shortly be publishing a letter outlining our open data philosophy, and detailing several extremely significant - and potentially controversial - new datasets that we will be publishing from next year, in the areas of health, education, transport and criminal justice. I can't pre-empt him by saying more here, but urge you to believe, as I do, that they are part of a fundamental change for the better in the way we use, scrutinise and provide all our public services.”

Watch this space....


June 22, 2011

Cabinet Office minister's second seminar on “Developing Census Alternatives”

Following the meeting held in November, Francis Maude was as good as his word and held a second meeting yesterday.

ONS gave a presentation which focussed on their comparisons of DWP and NHS data with their estimates – the results were encouraging, and well received.

There was lots of discussion, with the key points from Francis Maude including:

• Recognition that the future of PAF in the National Address Gazetteer is an important issue
• “If data sharing between government departments is bogged down by legislation, we can change the law”
• The importance of looking at commercial data sources as well as government (although I’m not sure that the distinction between VARs such as Experian, and companies such as DUG members with customer data is always realised)
• This isn’t simply looking towards an alternative to a 2021 Census – it needs to be incremental, testing data on the way.

All in all, another encouraging session – but ONS will have to push ahead fast if they are to make real progress.


May 28, 2011

Open data / LinkedData at DCLG – the English Indices of Deprivation and more… & the 2011 Census?

If you are interested in the potential of Open data / LinkedData might like to know that the 2007 and 2010 Index of Multiple Deprivation datasets have now been released in full LinkedData form, albeit on a trial/prototype basis. You can read an article about it at:

Perhaps such an approach help with our access to big files of 2011 Census stats later next year….


May 27, 2011

Data4nr updates – some new free datasets for small areas

“Data for Neighbourhoods and Regeneration” http://www.data4nr.net/introduction/  “identifies and signposts the datasets available for targeting, monitoring, priority setting and performance management at a neighbourhood level.”

Some recent ones which might be of interest to DUG members for targeting local areas, or perhaps for modelling (they are all at Lower Super Output Area level), are:

Income Support (IS) (Updated for November 2010) http://www.data4nr.net/resources/370/  
State Pension (Updated for November 2010) http://www.data4nr.net/resources/406/
Fuel poverty sub-regional statistics (New, Available for 2008) http://www.data4nr.net/resources/1457/  
Population estimates by broad age band http://www.data4nr.net/resources/527/  
Child Benefit (Updated for 2010) http://www.data4nr.net/resources/405/  


May 27, 2011

Census 2011: Conference on its impact and potential, Manchester, 7- 8 July 2011

If you really want to get immersed in preparing for the arrival of the 2011 Census results next summer / autumn, here’s the event for you – a 2-day Conference in Manchester, organised by the ESRC (with Professor Dave Martin taking the lead).

For more information and booking please go to http://www.ccsr.ac.uk/sars/conference2011/ 


May 18, 2011

"Chancellor of the Exchequer wants businesses to take advantage of big data", + news of National Address Gazetteer

It’s good to see that the Government is still committed to pushing ahead in opening up more data:

George Osborne, at the Google Zeitgeist meeting on Monday, spoke of the great social & economic benefits that could come from open data, and “We will have a specific focus on open data over the coming months, to ensure that we maximise the business opportunities at hand”.

Read more at:

Also, at a meeting last night, Francis Maude (Minister for the Cabinet Office) when questioned by DUG about the new National Address Gazetteer being made freely available to all – not just the public sector – stated that “This is unfinished business”. 


May 12, 2011

ONS 2011 Outputs Consultations, 2nd Round – DUG's responses

Following the News on February 8 below, here are the responses now sent to ONS on behalf of DUG:

1. Outputs. Please see the document attached

2. Geography. Our reply simply:
- expressed support for ONS's Geography policy, subject to Workplace Zones being an additional option, rather than a replacement for OAs for workplace stats
- welcomed the proposed geography products, and confirmed that we would be content with best-fit wards. 


April 25, 2011

AGI / DUG User Needs Workshop on the UK Location Programme - 9 June

Here’s news of a joint Association for Geographical Information / DUG event, and we would really appreciate your support by attending if at all possible.

Title: Growing Location-based Applications in Retail, Financial Services and Transport sectors: How can the UK Location Programme help?
Time: Thursday 9th June, 9.30-16.00
Place: School of Economic Science, 11 Mandeville Place, London W1U 3AJ
Cost: NOW FREE (due to generous financial support from the UK Location Programme)

The event will provide a golden opportunity for commercial companies to provide input to the direction of the UK Location Programme http://location.defra.gov.uk/ . A lot of Government effort is being put into developing powerful web services for geographical data discovery, geospatial data view and download, but there is an awareness by the UKLP, and its Location User Group – where DUG is represented by Mike Whitelegge of M&S – that such initiatives should benefit users not only public services, but also the private sector.

Details of the programme and registration are at:

April 21, 2011

Making data accessible – Guardian Chalkboards

Following the discussion at our most recent meeting about taking data and making it easily accessible for the public, here’s a great (football) example provided by Lee Madden at Barclays: http://www.guardian.co.uk/football/blog/2011/mar/07/liverpool-manchester-united-chalkboard-analysis  

Note the "make your own" button. Lee’s done just that – see attached.


April 20, 2011

MRS seminar on 2011 UK Census – afternoon, 4 July 2011

Here’s news of a seminar on the 2011 Census.

You’ll see that the great line-up of speakers includes Annette Dellevoet of Sainsbury’s.

The announcement below includes a link to a booking form.



April 6, 2011

Population 24/7 – Tracking individuals by phone

Following Professor David Martin’s presentation on Population 24/7 at the most recent DUG meeting, Norman Bekker at Whitbread has alerted me to this:

A German politician went to court to make Deutsche Telecom divulge all the information they held on him. He won and was given a 36,000 line spread sheet. The German paper Die Zeit then produced this Google earth related application using the data supplied. Open the link and see what your phone can reveal: http://www.zeit.de/datenschutz/malte-spitz-data-retention


April 3, 2011

2011 Census: GRO Scotland Consultation – DUG’s Response

Following our response to the previous consultation (see the posting on May 6,2010) GRO Scotland has been seeking further views:

Having made lots of general points (most of which were accepted) before, this consultation goes into detail about table designs. The general approach is to reproduce what was provided in 2001 whenever possible, and add new tables for the new Census questions.

Here is the response that has now been sent to GRO Scotland.

March 24, 2011

English Indices of Deprivation 2010 published today

Several DUG member companies are likely to be interested in the latest Index of Multiple Deprivation – both the highest and lowest ranking areas.

It “combines a number of indicators, chosen to cover a range of economic, social and housing issues, into a single deprivation score for each small area in England. This allows each area to be ranked relative to one another according to their level of deprivation. As with the 2007 and 2004 Indices, the Indices of Deprivation 2010 have been produced at Lower Super Output Area level, of which there are 32,482 in the country.”



March 14, 2011

Public Data Corporation – DUG's letter to the Cabinet Office

An earlier message (23 February) reported that the government has announced the idea of setting up a Public Data Corporation to “open up opportunities for innovative developers, businesses and members of the public to generate social and economic growth through the use of data”, and that there was an informal public consultation.

Following the workshop held by the Cabinet Office and the Shareholder Executive on 4 March – it was good to see some DUG members there – this letter has been sent from DUG.


March 5, 2011

Press coverage: Two stories about the 2011 Census (& DUG) in Marketing magazine, plus a 3rd in the Daily Mail

Marketing magazine (2 March) had a two stories this week on the 2011 Census which mention DUG:

“Government targets UK brands in quest for census data” (front page lead story in the printed edition)

“Census: marketing tool or obsolete?”

The Daily Mail (2 March) then picked up the news, and gave the story its own (typical) treatment:

“Could 2011 see the last-ever census? Supermarket loyalty cards may be used instead”


February 28, 2011

"The Demography of Ageing and Official Statistics" Northern Ireland 17-19 August 2011

An International Association for Official Statistics conference will be held in Belfast between 17th and 19th August 2011. The draft programme can be seen here.


February 23, 2011

National Address Gazetteer – OFT decision; plus news of a Public Data Corporation 

Following our letter (see news item 4 Jan) the Office of Fair Trading has now announced that it has “cleared national addressing database joint venture”:

Whilst it fails to recognise the potential use in the private sector, and there are lots of loose ends (such as the future of the Postcode Address File), at least it opens the way for the file to be created, even if free use is currently restricted to the public sector.

On this issue, the Department for Communities and Local Government has replied to our letter to Francis Maude (see news item 21 Dec), and includes the phrase “.....we are continuing to work with the relevant bodies to identify if there might be additional benefits to making addressing data available for free.” So we will continue to make the case.

On a wider front, the government has announced the idea of setting up a Public Data Corporation to “open up opportunities for innovative developers, businesses and members of the public to generate social and economic growth through the use of data.” There is now a public consultation, which closes on 11 March: http://pdcengagement.cabinetoffice.gov.uk/pdc/


February 9, 2011

MRS afternoon seminar "Using Data and Technology to Manage Customers across Multiple Channels" - 15 March 2011, London

Here’s news of an afternoon seminar organised by the Market Research Society’s Census & Geodemographics Group:

“In today’s fast moving, technological environment, with increasingly sophisticated consumers, marketing issues are becoming ever more complex. With consumers shopping online, whilst browsing Facebook and watching TV, there are many pitfalls for the marketer that must be avoided, and there is much to understand about the multi-tasking customer, especially in bridging the gap between online and offline activity. The Changing Interface of Marketing seminar, chaired by Barry Leventhal – Director of BarryAnalytics – aims to address these issues and potential pitfalls with presentations from an impressive line-up of speakers:

Steven Plimsoll, Acxiom – Linking Social Media to marketing activity
Heidi Thompson, CACI – The Art of Good (Customer) Conversation
Nigel Wilson, Experian – Search, online competitor intelligence and the role of segmentation
Dave Evans, Information Commissioner's Office – Data Protection, targeting, and fairness”

More details, including the booking form, are available at:


February 8, 2011

ONS Census News Alert – 2011 Outputs – Consultation, 2nd Round

The Census itself is almost upon us (27 March), and ONS is having a second round of consultation on Census Outputs.

The deadline for responses is 29 April. 


January 25, 2011

ONS's Wealth and Assets Survey – now analysed using the Output Area Classification (OAC)

You may remember that ONS published its new Wealth and Assets Survey at the end on 2009 (see news item 11 Dec 2009 below).

I'm delighted to report that not only has the survey been coded with the OAC geodemographic classification, but there's a paper "Exploring the geographical distribution of wealth using the output area classification" in the latest Economic & Labour Market Review (Jan 2011). http://www.statistics.gov.uk/articles/elmr/elmr-jan11.pdf  - pages 59-78.

You'll see some dramatic profiles of the variations in Total, Property, Pension, Financial and Physical Wealth by neighbourhood type.


January 14, 2011

Data on Car Parking from Betts Consulting

Just in case not all of you are aware of this as a data source, here is news from Ian Betts about his information on car parks.

The anoraks amongst us will be interested to see a grand total of 13,795, of which 91.58% are postcoded.


January 14, 2011

ONS's latest 2011 Census News Alert – inc. a Roadshow on plans for Outputs

Here’s an alert from ONS which includes news of its further consultation on 2011 Census outputs. You may want to book a place on one of the roadshows if you'd like to keep up with current plans.



January 4, 2011

ONS Spending Review  – User  Consultation – DUG's response

Here is our response to ONS's consultation about its spending priorities (other than the Census, which is ring-fenced).

You'll see that the answers focus very much on our primary interests, when seeking to target consumers and local markets.

"Our general view is that when seeking to make cuts ONS should seek to use statistics from administrative sources (which also offer small area statistics) rather than large sample surveys (which may be then be carried out less frequently). This may also help with another of our goals: more timely statistics."


January 4, 2011

National Address Gazetteer – letter to the OFT

Following our letter to the Cabinet Office (see 21 Dec), here is a related letter to the Office of Fair Trading.



December 22, 2010

data.gov.uk, & "Open Government Data – from a DUG user point of view"

Here is a presentation given at a TNA meeting on Transparency on 7 December. Slides 1-9 give the context, & 10-13 focus on the website, a shopping list of datasets, and some current issues.


December 21, 2010

Letter to the Rt. Hon. Francis Maude MP: Transparency, and the new National Address Gazetteer – free use shouldn't be limited to just the public sector

With many thanks to DUG Contacts for their comments on an earlier draft, here is a letter just sent to the Cabinet Office.


December 16, 2010

Mapping the US Census

Here’s some news spotted by Pablo Mateos at the Centre for Advanced Spatial Analysis at UCL.

The New York Times has just launched a Census Atlas based on the ACS (American Community Survey 2005-2009) http://projects.nytimes.com/census/2010/explorer which illustrates the sorts of presentation that should become available for our own 2011 Census.


December 16, 2010

Neighbourhood Statistics Monthly Update – December 2010 – inc. house prices

Here’s a link to the latest update from Neighbourhood Statistics.

The Recently Released Datasets include:
"Changes of Ownership by Dwelling Price, 2009, drawn from the Land Register.
The data provides information on the total number of changes of ownership of dwellings, with statistics on the prices paid. These figures are broken down by four specific types of dwelling:
Detached / Semi-detached / Terraced / Flat
The statistics are available down to Middle Layer Super Output Areas within England and Wales."


December 5, 2010

ONS's 2009 Living Costs and Food Survey (was the EFS) – Household Expenditure analysed by OAC

ONS’s latest (2010) Family Spending report includes some analyses using its geodemographic Output Area Classification, OAC. See pages 99-112 at:


December 5, 2010

National Address Gazetteer Database announced – but only the Public Sector gets it free at the point of use

An announcement has now been made in parliament about a definitive “National Address Gazetteer Database”, see: http://www.theyworkforyou.com/wms/?id=2010-12-02a.85WS.4  
It includes: “The products created from the national address gazetteer database will be made available free at the point of use for all public sector bodies under the centrally funded public sector mapping agreement. Commercial customers will be able to license the data in the same way as they do for the Ordnance Survey and local government products they use today.”
This is a good technical step forward, and is really welcome for the public sector, BUT.... everyone else, including private sector businesses (2.3m in all) and also the public (NB) are outside the tent.
This doesn't square with the government's Transparency policy (including the mooted new Public Data Corporation’s mission "to drive the release of core reference data for free re-use”), so there are still battles to be fought.


November 1, 2010

DUG’s 2010 Conference – feedback, presentations

Here's more news of our Conference “How efficiency savings could trigger innovation in Insight” which was held at the Royal Society on 13 October:

* Feedback - this was our biggest conference so far, and to quote panellist Carol Tullo from The National Archive: “There was a tangible energy to the day, probably not unrelated to the pace and momentum of the policy and government agendas in this area and the great interest and excitement in helping shape all this.”

* The presentations are now on our website – see the tab to the left. 


October 31, 2010

Press Release – GlaxoSmithKline joins the Demographics User Group

We are delighted that GSK has joined DUG. Here’s the Press Release.


October 30, 2010

News of a government statement on a national address register

DUG gave evidence about the 2011 Census to the House of Commons London Regional Committee in the last parliament.

The new Government’s response has just been announced, and includes the following encouraging news:

“Recommendation 14 of the report was on the need for the census address register being developed by the ONS for the 2011 census to be maintained after the census. The previous Government failed to deliver a definitive address register, despite the demands for such a register and the associated costs of inefficiency in maintaining a number of similar registers. This Government are working with the parties concerned and will look to deliver a definitive register. Considerable progress has already been achieved. The work ONS has done will form part of the solution.” 


October 24, 2010

Public Spending Review – ONS & the 2011 Census, & Beyond 2011

The early unconfirmed headlines from the government’s spending review for the ONS are:

* Census funding has survived intact, and the ONS has in addition got the funding for outputs (i.e. post April 2012).

* ONS have also got funding for "Beyond 2011", and this is to be discussed at a meeting with Francis Maude at the Cabinet Office on 18 November.

* Outside these items there will be a 5% cut in ONS funding per year in real terms which they plan to meet partly through efficiency savings and partly through real cuts. 


October 15, 2010

House prices, & some data on Income from HMRC

Land Registry – Property prices (but only at County / UA level):
Property prices (Updated for August 2010)

HMRC – 5 files concerned with people on low income, down to Lower Layer Super Output Area (LSOA), + Personal Incomes from tax records (albeit at Local Authority District level). This is encouraging, given that HMRC has traditionally lagged behind as a source of data.

Child Tax Credit (CTC) (Updated for 2008)
Child Tax Credit (CTC) - Lone parent recipients (Updated for 2008)
Out of work families with children (Updated for 2008)
Out of work lone parents with children (Updated for 2008)

NI 116 - Proportion of children in poverty (Updated for 2008)
Personal Incomes from tax records (Updated for 2007/08)


October 15, 2010

DUG Award – to Professor Nigel Shadbolt and Sir Tim Berners-Lee for data.gov.uk

This year’s DUG Award for “better information from government” – which this year is in memory of Andrew Tasker – has been scooped by Professor Nigel Shadbolt and Sir Tim Berners-Lee for the innovatory government website data.gov.uk.

You’ll see in the attached Press Release comments from:

Steve Bond (who chaired this year’s Conference):
“Commercial companies have been making the case for better access to government datasets for more than a decade. Public pressure continued to grow, but it was the appointment of Professor Nigel Shadbolt, and Sir Tim Berners-Lee, to advise on data transparency that proved to be the catalyst for radical change, with the launch of data.gov.uk in this year. Members of the Demographics User Group (DUG) are delighted with the progress, which will inform society, increase the efficiency of service delivery, and fuel innovation.”

Professor Nigel Shadbolt:
”Tim and I are delighted to accept this award on behalf of all those who have helped to make data.gov.uk a reality. Government Open Data supports transparency and accountability, it promotes efficiency and innovation – in short in makes the UK work better.” 


October 11, 2010

Putting Location-Based Ads to work

Here’s some news from the USA – see below – spotted by our friends at UCL.

“Ads targeted to a person's location are an advertiser's dream. The reality is more complicated.......”


September 17, 2010

MRS CGG conference - Targeting for multi-channel marketing conference - 24 November 2010

Here’s news of a conference organised by the MRS’s Census & Geodemographics Group.

Last year’s event attracted a large audience, and you’ll see that there’s a good line-up of speakers.


September 17, 2010

OAC conference - news of free access to the OAC-coded British Population Survey, and to software for coding customer files

The OAC Conference was a great success.

Here’s a link to all the presentations, which included news of the OAC coding of the British Population Survey (an alternative to the Target Group Index) & a presentation by Prof. Martin Callingham:
Also, the team at UCL has developed OACoder, a tool for affixing OAC to postcoded files:
Do give it a try!


September 17, 2010

ONS's Revised Super Output Area Mid-Year Estimates for 2001-2008 

ONS has produced updated population estimates for small(ish) areas.

The revised Lower Layer and Middle Layer Super Output Mid-Year Estimates for 2001-2008 have been released today as experimental statistics on the National Statistics website. They are available to download from:

These estimates are also available on the Neighbourhood Statistics website at:http://neighbourhood.statistics.gov.uk/dissemination/

User feedback on these estimates is very much welcome as this may assist in making improvements to future estimates.

The following link takes you to the Small Area Population Estimates project page, where future updates about developments and releases will be made:


September 6, 2010

DWP stats - at Output Area level at last

Here’s news of Out of Work benefits at Output Area level for England & Wales.

Whilst this might not appear directly relevant to most DUG members, it’s potentially important in:
• Implicitly highlighting prosperous areas where very few people claim worklessness benefits
• Setting a precedent that central government departments can use their administrative files to create statistics down at OA level (far more useful to us than Super OAs or other higher areas such as wards or LAs): we’ve been pushing for this ever since ONS’s Neighbourhood Statistics started.

The announcement reads: “The Department for Work and Pensions have published data on the number of people receiving ‘out of work’ benefits – Jobseekers Allowance, Incapacity Benefit, Employment Support Allowance and people on low income benefits, down to Output Area level. These new data will enable the identification of very small pockets of worklessness, as well as providing more detail on the nature of worklessness in these areas (by benefit type).”


September 1, 2010

ONS User Engagement Consultation - DUG's response

ONS has sought views on its strategy for engaging with users.

The attached reply on behalf of DUG - thanks for the input from DUG Contacts - makes the case that ONS pay too much attention to government departments, and that they need to change the culture, setting customer service as their top priority. 


August 2, 2010

Department for Transport - Accessibility Indicators down to LSOA level - Food stores, etc.

The Department for Transport has published updated statistics on the Core Accessibility Indicators for 2009 for England – these include accessibility to food stores, and town centres.

The Indicators provide a number of measures of accessibility by public transport, walking, cycling and car to eight service types: primary schools, secondary schools, further education, GPs, hospitals, food stores, town centres and employment centres – and are available down to Lower Super Output Area Level.

Here’s the detail: http://www.dft.gov.uk/pgr/statistics/datatablespublications/ltp/coreaccessindicators2009


July 29, 2010

"Ordnance Survey Leading The Way On Government Transparency"

The Department for Communities and Local Government has issued a Press Release which starts:

“The national mapping agency is the pioneer in a new era of information transparency by Government, Communities Minister Baroness Hanham said last week.

Three months after launching the OpenData initiative, Baroness Hanham said Ordnance Survey is leading the race to make public data freely available and accessible........”

You can read it all at: http://www.egovmonitor.com/node/37518  

Given OS’s bitter resistance over the last decade to opening up free data, this spin is truly boggling. George Orwell would have been proud.
But, better late than never, and it’s good to see the new Coalition strongly committed to making public data freely available.


July 29, 2010

Job vacancies – Tesco & Boots

For details, see the tab to the left.


July 27, 2010

2011 Census to proceed as planned – Ministerial Statement

The UK Statistics Authority has just issued a brief Press Release:

“The UK Statistics Authority welcomes the Government’s decision to proceed with the existing plans for the 2011 Census in England and Wales, set out in an answer by the Minister for the Cabinet Office, Rt. Hon. Francis Maude MP, to a Written Parliamentary Question.
The Office for National Statistics’ preparations for the Census on 27 March 2011 will now press ahead with all speed.
The Statistics Authority is determined that, with the full support of the Government and all the other parties concerned, the 2011 Census will be the success that the country needs it to be, and will provide the information about our population which can only be derived from the full-scale Census, which has now been confirmed.”


So it appears that Francis Maude – despite previous hostility – has now been convinced. Phew.


July 22, 2010

OAC conference – PM, Monday September 6th, 2010 – free

Here’s news of this year’s OAC (Output Area Classification) conference. The programme is pasted below. The emphasis is on use in the public sector, but it includes sessions on the British Population Survey (an alternative to the TGI), and Open Data, Free Tools, and there will no doubt bee wider discussion & ideas. There is more information on the OAC User Group website: http://areaclassification.org.uk/

You can book (no charge) at: http://oac.eventbrite.com


July 12, 2010

"National Census to be axed after 200 years" – Daily Telegraph – DUG's letter published

Our letter has been published in today's Daily Telegraph:

"Counting the population
SIR – The Government has decided that next year’s Census will be the last (report, July 10). Hitherto, censuses have provided detailed counts of the population that are vital for both public and private organisations when deciding where to target investment, and for the public, to inform democracy. Census returns released after 100 years are also of interest to family researchers.
Francis Maude, the Cabinet Office minister, is right to examine different and potentially cheaper ways to count the population. In particular, the Government’s own extensive databases would be much the best starting point, learning from the experiences of several Nordic countries.
However, to announce a decision to scrap future censuses before proving that alternative sources are adequate is reckless.
Keith Dugmore
London SW1"

& can be found on their website at:


July 3, 2010

UN / Eurostat Census meeting – papers on Censuses overseas

For those of you interested in Census information for countries outside the UK, here's news of the "Joint UNECE/Eurostat Group of Experts on Population and Housing Censuses", which takes place in Geneva, 7-9 July:


You'll see papers from many countries, including one from me on "Disseminating Census information to maximise use and value", which highlights the need for a better balance between statistical disclosure risk, and utility, guided by Jeremy Bentham’s view: “It is the greatest good to the greatest number of people which is the measure of right and wrong.”

June 21, 2010

British Population Survey – coded with geodemographics, including OAC

Here’s news of the British Population Survey (a commercial survey, a bit like the Target Group Index), which is now coded with OAC and other geodem classifications.


You’ll see the word FREE (I understand that only the most recent year is charged).


June 19, 2010

News of an open license for a NSPD postcode lookup file

Here's news from the OAC User Group:
"An open license for the non PAF version of the National Statistics Postcode Directory postcode lookup file is now available. The terms of which enable a user to:
• Copy, distribute and transmit the Data
• Adapt the Data
• Exploit the Data commercially whether by sub-licensing it, combining it with other data or by including it in your own product or application
This is great news as it includes OAC as one of the variables and makes address lookups a much simpler process. We shall update the tutorial guides on the website soon to reflect this change"

The ONS website goes into more detail, starting:
"The NSPD is released quarterly in February, May, August and November and NSPD products are supplied with various elements of Postcode Address File (PAF) data depending on the level required. These extracts are available as Full PAF or Part PAF options. Additionally, an ‘Open’ version, containing no PAF elements (and not attracting Royal Mail royalties) is also available......."
See more at: http://www.ons.gov.uk/about-statistics/geography/products/geog-products-postcode/nspd/index.html  


June 19, 2010

More free data: a) Transport for London; b) Postcode grid references & lat / long

APPSI have just alerted me to more free data:

1) Transport for London - as reported in The Guardian:
"Transport for London has opened up huge swathes of its data for free re-use by developers keen to see what they can do - and build - with it. After years of lobbying by developers who have wanted to be able to exploit the data - which, being publicly funded, logically belongs to the public too - TfL today acknowledged its willingness to listen to what people want to do as the London Data Store, headed by Emer Coleman, announced the datasets that will be provided.........."
Read all about it at: http://www.guardian.co.uk/technology/blog/2010/jun/15/london-datastore-tfl-data-free 

2) Geopostcode:
"An API for retrieving geographic information (grid reference, latitude and longitude) relating to postcodes. Data is available as XML or JSON" http://data.gov.uk/apps/geopostcode  


June 10, 2010

UKSA interim report "Strengthening User Engagement" – DUG's response

Here is DUG’s response to the UKSA consultation. It makes the case for going further than just better user dialogue, with UKSA's monitoring role being more proactive, the government statistical service becoming more customer focussed - and adding a proposal to rename the Government Statistical Service the National Statistical Service – to reinforce the need to serve all customers, not primarily those in central government.


June 8, 2010

Independent newspaper – The real cost of a compromised census will be inaccurate data – DUG's views quoted

ONS has just alerted me to an article in last Saturday's Independent:

"With less than a year to go to the 2011 Census, is Francis Maude about to change the rules? In opposition he was strongly critical of the Census, describing it as burdensome, intrusive and poor value for money. Now he's in charge, as Minister for the Cabinet Office. He's also central to the programme of Government cuts that the coalition has promised. So demographers and statisticians are worried that he may make good on his promise to "scale back" the Census even at this late stage........

.....if he doesn't make any changes when he's now in a position to, he's going to have to eat a lot of words. He's already had a warning from the Demographics User Group, a private-sector body that represents heavy-hitters such as John Lewis, Marks & Spencer, Sainsbury's, Nationwide and Alliance Boots, saying that they regard the Census as of "fundamental importance" when making decisions about opening new stores, the products to be stocked, and the customers to be targeted. A late decision to cut the number of questions would disrupt existing plans and contracts, save little money – or even add to the cost – and result in reduced and inconsistent information between England and Wales, Scotland and Northern Ireland, says Keith Dugmore, director of the group........."

You can see the full article at:


June 8, 2010

Big role for supermarkets in regenerating poor communities – report by Demos

Yesterday's Guardian reported: "Demos sees big role for supermarkets in regenerating poor communities" & argues for tax breaks for supermarkets if they can turn sink areas around: http://www.guardian.co.uk/uk/2010/jun/07/demos-supermarkets-poverty-regeneration-pride

& here's the link to the Demos website, where you can download the full report Civic Streets: http://www.demos.co.uk/publications/civicstreets 

June 5, 2010

Victory for campaigners as coalition releases data deluge

Today's papers have further news of the new government releasing more data into the public domain.

The Guardian's article http://www.guardian.co.uk/politics/2010/jun/04/government-spending-database-special-report focuses on COINS (Combined Online Information System) and information on government spending, whilst Charles Arthur reports "A flood of data is on its way ... but we will need to make sense of it"

Much of this won't be immediately relevant to us, but it does confirm the new government's general policy that government information should be in the public domain. 


May 26, 2010

Experian partners with The Local Data Company

Here’s some news from Experian about The Local Data Company (who presented to our meeting last September, and are used by several DUG member companies):

You’ll see that it’s referred to as partnering, rather than continuing Experian’s recent history of buying small companies. 

May 20, 2010

Database valuation

Several members have been interested for some time in how a value might be put on their customer databases. At DUG’s most recent meeting Mark Perrett (E.ON) mentioned his recent discussions with Call Credit, who are researching the topic.

Mark has also unearthed two earlier documents from the Institute of Direct Marketing:

* White paper - Database Valuation: Putting a Price on Your Prime Asset.

* A case study in the valuation of a database prepared by Valuation Consulting. 


May 18, 2010

Ordnance Survey free "opendata" – more vector mapping released – 1:25,000 scale

Following up the news of 10 April, OS has now released another map dataset: "OS VectorMap™ District". It's nominally at 1:25,000 scale, and in both vector & raster formats.

Details are given at: http://www.ordnancesurvey.co.uk/oswebsite/products/vectormap/district/index.html


May 18, 2010

Letter to the Rt. Hon. Francis Maude MP: The 2011 Census – its importance to commercial companies

The new Cabinet Office Minister Francis Maude has in recent months actively made the case to reduce the scope & cost of the Census (& one of the options now being mentioned is to scrap it completely). We have therefore sent him this letter to get the commercial users’ case for the Census in front of the new government without delay.

May 6, 2010

2011 Census Output Consultation – Scotland – DUG’s Response

See our response to GRO Scotland here

April 10, 2010

Ordnance Survey free "opendata": what does it actually include?

Following up my message of 3 April, here's the Guardian's Data Blog answer to the above question:

If you have any obs about this treasure trove - maps, boundaries, postcode grid references, gazetteers - or, indeed, a wish-list of more government data you'd like to push to be made freely available, do get back to me. 

April 4, 2010

Job Vacancies – advertisements by DUG members

Several members are seeking staff – see the “Jobs” tab on the left.


April 3, 2010

Major Government announcement – Ordnance Survey – significant datasets now FREE

Following the consultation on Ordnance Survey the Government has published its response:

The Guardian’s Free Our Data report http://www.guardian.co.uk/technology/2010/apr/01/ordnance-survey-maps-download-free  starts off:

“It is the biggest shakeup in the Ordnance Survey's 260-year history: you can download maps as detailed as 1:10,000 scale, or collect a list of locations appearing on maps at the 1:50,000 scale, or a conversion system for postcodes to grid references – free for personal or commercial use.”


April 1, 2010

London Regional Committee's first report, London's population and the 2011 Census

Here's the report http://www.publications.parliament.uk/pa/cm200910/cmselect/cmlonreg/349/34902.htm  from the House of Commons committee to which we gave evidence. I'm pleased to see that DUG gets quoted.

You'll see strong support for the Census, but also the predictable concern about the dangers of a low 2011 Census response in London - and the need to target effort.

Also: "The Committee welcomes the creation of a national address register for the Census, which will greatly improve accurate delivery of census forms, but say it is 'barely credible' that this valuable resource is not to be maintained after 2011. The report recommends that the Government acts urgently to resolve the intellectual property issues which are preventing maintenance of the register past 2011."


March 31, 2010

Orange joins DUG

Orange is the latest company to join the Demographics User Group (DUG), whose membership already includes major companies across a range of sectors. 

Orange is one of the largest mobile operators in the UK, with a current consumer base of 17million. It has a wealth of internal data in its analytical environment, from point of sale through to individual call record - who, when, where, and how often a customer calls.

Orange’s Strategic Insight's team's main responsibilities are to provide: Customer segmentation; a behavioural view to increase value, loyalty & enhance the customer experience; bespoke deep dive analytics from a customer led approach for the whole of Orange.

Anthony Mawby, Head of Strategic Insight, stated "We are very pleased to be joining DUG, and are looking forward to sharing experiences of data, analysis, and insight with other major companies from a range of different sectors including retail, financial services, and energy."


March 27, 2010

Department of Transport's 350,000 public transport access points "NaPTAN" – available free

Following my message back in September (see below), and the DfT then saying that commercial users have to pay, I've now checked about the data being part of the (free) www.data.gov.uk initiative. Here's the answer that I've received from the DfT:

"Thanks for your email. I had a quick look at the data.gov.uk website and snapshot bus stop location data is available to download at: http://data.gov.uk/node/8473  

I've clarified the use of this data with colleagues at Transport Direct, and the snapshot data is freely available to all for use. However, this is snapshot data and so will already be out of date, as changes are made to the bus stop data almost on a daily basis. Therefore, if commercial users wanted to access more up-to-date information, then they would need to request a license and follow the procedures as set out on the NaPTAN website: www.dft.gov.uk/naptan  .

I understand that updates will be made to the data held on the data.gov.uk website, but am not sure of the frequency of these."


March 24, 2010

Prime Minister's speech: "Building Britain's Digital Future" (more news of free data, inc. Ordnance Survey)

See: http://www.number10.gov.uk/Page22897 


March 13, 2010

Google Street View - now extended to UK coverage

Google's Street View has now increased coverage for the whole UK: http://maps.google.co.uk/

To quote a geographer at UCL: "It is pretty amazing – just drag the little yellow man onto the map of the UK.....".


March 13, 2010

Consultation on Ordnance Survey – DUG's response, + extra comment

Here's DUG’s response to the Department for Communities’ consultation on OS.

APPSI has also made a heavy duty response (led by David Rhind): http://www.appsi.gov.uk/content/binary/APPSI-response-to-GI-Consultation.pdf

I'm very pleased that our own original description of the address register situation as a "national scandal" [Copyright DUG] is now ubiquitous.


March 4, 2010

Sharing customer data & insight with government – report now published by ONS

ONS commissioned Demographic Decision to carry out a project on “Information collected by commercial companies: what might be of value to ONS?”. This is to help them with both their current population updates, and also planning 2011 Census coverage.

I’m very grateful to several DUG members and other companies who have made time to discuss this. The report is now complete, and ONS have just made an abridged version public:


ONS are now considering the recommendations as part of their Improving Migration and Population Statistics programme, and their planning for the Census next March. I hope that this leads to further steps forward in sharing data and insight across the public / private divide.


February 28, 2010

DUG's new LinkedIn networking site – please make the most of it to exchange news & views!

We've now set up a community communication / social network site, and I'm really grateful to Lee Madden of Barclays for getting us started.

Here's the link: http://www.linkedin.com/groups?about=&gid=2745168&trk=anet_ug_grppro

You'll see that "This Demographics User Group Network site is open to both members and non-members (commercial companies, public services, academic, etc.) to exchange news and ideas"



February 20, 2010

As well as ONS's Consultations on 2011 Census Outputs, we now have Scotland's too

Following ONS's announcement of its Census Consultations for England and Wales (see earlier message below), we now have an announcement from GRO Scotland:

"This consultation is intended to provide the basis for developing, over the course of 2010, an agreed set of final specifications of the main outputs from the 2011 Census that meet the needs of a majority of users and make the best use of data collected. It also seeks views from users on a number of issues relating to census outputs including:

•pre-defined tabular outputs;

•comparisons of 2001 and 2011 census results;

•data delivery formats;

•outputs geography – including which intermediate geographies users need and what demand exists for workplace zone statistics; and

•timetable for release of 2011 Census results"

Read all about it at:

The closing date is 14 May 


February 7, 2010

2008 stats on Dwelling Stock by Council Tax Band - at Output Area level

Data for Neighbourhood Renewal has alerted me to the following new statistics now available on ONS's Neighbourhood Statistics website:

* Dwelling stock by council tax band (Updated for 2008) - down to Output Area (OA) level for England & Wales: http://www.data4nr.net/resources/53/

We now have datasets for each year since 2001, which in principle enables the tracking of changes in numbers of dwellings and their status (Bands A-H) for very small areas.  


February 1, 2010

Evening Standard reports DUG evidence at House of Commons Committee

Our evidence to the new House of Commons London Regional Committee for its inquiry into “London’s Population and the 2011 Census”, led to an invitation to give evidence at its first hearing today.

I'm pleased to report that we've got some excellent press coverage in tonight's Evening Standard:



January 22, 2010

Tim Berners-Lee unveils government data project - data.gov.uk

Pushing forward the Making Public Data Public initiative, the BBC reports the big news that:

"Web founder Sir Tim Berners-Lee has unveiled his latest venture for the UK government, which offers the public better access to official data.

A new website, data.gov.uk, will offer reams of public sector data, ranging from traffic statistics to crime figures, for private or commercial use......"

You can read the full report at: http://news.bbc.co.uk/1/hi/technology/8470797.stm

& see the new website at: http://www.data.gov.uk/


January 18, 2010

DUG Training Course “Developing as an analyst” run for a public service organisation for the first time

This course, developed and presented for DUG by Stewart Robbins, Head of Customer Knowledge at E.ON, has been run for DUG member companies several times.

Friends in public sector organisations have also expressed interest, and on 18 January it was run for the Department for Transport. Here’s their feedback:

The delegates to this course, all of whom had a background in statistics, expected to end the day as more effective analysts. They wanted to learn how to identify customer requirements, and how to fulfil these in the most relevant way. The day lived up to their expectations:

“Lots of helpful advice in determining what stakeholders require”
“Highlighted the need to take time to think about a problem and then to think about a solution”
The delegates appreciated the trainer’s knowledge and the lucid and engaging way in which he delivered the course:
“Had good relevant experience, delivered material at good pace. Engaged well with participants”
“Good range of examples………..opportunities to ask questions”
“Kept course relevant and interesting”

Summing up: “Excellent course with lots of practical suggestions. Covered good range of topics in about the right level of detail”


January 14, 2010

House of Commons London Regional Committee – London's Population inquiry – DUG's Evidence

We’ve been invited to provide a submission to another House of Commons Committee.
I’ve sent a note with the following key themes:

• The importance of good population statistics to commercial companies
• The need for several different measures of “the population”
• The currently crucial importance of the Census as a source
• The 2001 Census serving as an alarm bell to focus on coverage in 2011, leading to 3 priorities for improving coverage and quality:
- Create a definitive address register – for 2011, and beyond
- Target difficult areas for hand delivery, and collection, of Census forms
- Use administrative files for quality checking 2011
• Looking further ahead, more use should be made of administrative files for statistical purposes, aiming towards the creation of a population register, validated by a final traditional Census in 2016 (rather than waiting until 2021). 


January 12, 2010

OBiT's not-for-profit sharing of deceased names - 25 organisations now participating - might you join too?

Many of you will know Emma Reid, who was a very active member of DUG when she was at Saga, and who now has her own consultancy Greysells, which focuses on the over 50's.

Emma also initiated OBiT - the Stop Dead Consortium - to encourage organisations to pool their records of deceased people on a not-for-profit basis.

Its major benefits are the reducing the existing costs of cleaning lists, reducing the costs of wasted mailings, and preventing distress to the bereaved.

I'm very pleased to report that 25 organisations are now pooling their information - contributing to a file of 4.7 million deceased names. These organisations include DUG members Boots and M&S, but obviously Emma is keen to extend the list further. If you, or colleagues in another part of your company, are interested, do get in touch with her via http://greysells.co.uk/ 


January 7, 2010

Boris Johnson launches London 'Datastore' with hundreds of sets of data

Guardian Technology has trailed this story:

"The Mayor of London, Boris Johnson, will on Thursday launch a website hosting hundreds of sets of data - including previously unreleased information - about the capital, as part of a new scheme intended to encourage people to create "mashups" of data to boost the city's transparency and accountability. Channel 4 will also be offering up to £200,000 through its 4ip fund to help develop the most innovative uses of the data."

Read more at: http://www.guardian.co.uk/technology/2010/jan/06/london-datastore-launch-johnson-mashups  

The formal launch takes place at the end of this month, but the GLA is already seeking users' views: http://data.london.gov.uk/

It's good to see that Boris Johnson is as enthusiastic about "Making Public Data Public" as Gordon Brown (even if Lord Mandelson is reluctant). 

PS. A postscript to my earlier message - you can now see Boris promoting the case for free data (but also struggling with Skype - it's a hoot):


January 1, 2010

ONS's Census News - Latest Update, including Consultations on 2011 Outputs - Statistics and Geography

ONS has announced two consultations which are really important to us:

* Output Consultation: Main statistical outputs

This goes into considerable detail. It takes the 2001 tables as a starting point (which makes life simpler), and then highlights the impacts of changed questions (e.g. Type of Central Heating, and Ethnicity), and the opportunities created by the new questions - Second Residence & Language (& also National Identity, Citizenship, Month & Year of Entry, Intention to Stay, and Number of Bedrooms).
I shall start to review the particular priorities that we've argued for in earlier consultations (e.g. Workplace stats, OAC, Social Grade, Multivariate counts, etc.) but it will be really helpful to get your views on particular 2001 tables (or, failing that, topics) that have been valuable to you, including mainstream stuff on age, sex, tenure, NS-SEC, etc.

* Census Output Geography Consultation

This consultation is less demanding. ONS are already committed to keeping existing Output Areas wherever possible, and I'm assuming that we shall continue to support that vigorously. 3 issues are raised:
- A new geography for improved reporting of business statistics and statistics relating to the workplace [Sounds beneficial, as long as we continue to get simple workplace counts for all OAs, plus the few OAs with big working populations are split further?]
- An upper layer of Super Output Areas (USOAs) [Of no use to us?]
- A request for exceptional instances of current OAs and SOAs, as designed for 2001, that do not fit present criteria for statistical zones [No objection to a bit of tidying up?]

We have a reasonable amount of time (the closing date is 26 March), but it would be good to get your initial thoughts by the end of January, and then I'll produce a first draft.

These ONS consultations do, of course, apply to England & Wales. Scotland and N Ireland will be announcing their own consultations shortly.  


December 31, 2009

Letter in the Daily Telegraph - "Liberating Royal Mail data will generate tax revenues"

Following the FT's report "Mandy and Gordon - the unravelling" (over PAF), the Daily Telegraph also ran the story.
I wrote on our behalf, and the letter (somewhat altered) was published yesterday 29 December:
http://www.telegraph.co.uk/comment/letters/6901342/A-different-American-election-result-in-2000-would-have-changed-everything.html   & scroll down to the letter.

I'll also send the original letter to Lord Mandelson's Department, to make sure that they realise the opposition to his view. 


December 29, 2009

Andrew Tasker - some very bad news

I'm very sorry to have to break the distressing news to you that Andrew Tasker died at the weekend.

Some of you will be aware that Andrew was diagnosed with cancer early in November, and, despite immediately starting a course of chemotherapy, chaired our strategy day on 18 November, and joined our awayday in Bristol on 3 December. I'm sure that we are all shocked to learn how swiftly he has died, and, for those of us lucky enough to know him well, to remember what a fine person he was.


Annette Dellevoet at Sainsbury's subsequently wrote the following appreciation of Andrew:

In June 1998 Andrew helped to found the Demographics User Group (DUG) along with Keith Dugmore and representatives from Whitbread and M&S. DUG lobbies the government on behalf of commercial users of government statistics as well as sharing methods and techniques and offering analytical training to its member companies. Andrew was an active and enthusiastic member of the group speaking at DUG conferences and running several training courses. It is a measure of his contribution that DUG retained him as an honorary member when he became a consultant. Despite his worsening illness Andrew was an animated contributor to the last DUG meeting on 3rd December. He will be greatly missed by all his DUG colleagues, who valued his wise advice, friendship and support, and have sent many messages of appreciation.


December 27, 2009

New free datasets - House prices, Household Income estimates, Supermarket accessibility, & LA comparisons

Here's news of several new free datasets from government:

Dwelling prices by dwelling type
"This dataset provides the total number of changes of ownership of dwellings provided by the Land Registry including the type of dwelling and the dwelling price"

Income: Model-Based Estimates
"New model-based estimates of average household income have been produced for England and Wales at Middle Layer Super Output Area (MSOA) level"

Accessibility - Supermarkets
"The Department for Transport has published statistics on the Core Accessibility Indicators for 2008. The Indicators provide a number of measures of accessibility by public transport, walking, cycling and car to seven service types" - one of these is Supermarkets:

The Places Database
"Is now available offering downloads of spatial data from national to local level covering crime, retail floorspace, health, transport etc." The stats are at Local Authority level & above.


December 24, 2009

"Mandy and Gordon - the unravelling" (over PAF), + Consultation on Ordnance Survey

Yesterday's Financial Times contained an article http://www.ft.com/cms/s/0/4afe99e0-ef44-11de-86c4-00144feab49a.html about Lord Mandelson resisting Gordon Brown's announcement about Making Public Data Public.
The article drags in PAF (rather than just postcodes & grid references), which hasn't been mentioned before.

The article also mentions the consultation on Ordnance Survey being delayed, but this morning I've just been alerted to: http://www.communities.gov.uk/publications/corporate/ordnancesurveyconsultation  
Being faced with 91 pages from CLG makes your heart sink, but at least it's out before Xmas, rather than delayed, which would have increased the risk of the standard 3 month consultation period running into the election buffers.


December 11, 2009

ONS's new Wealth and Assets Survey published

This is a new survey from ONS which I think may be of interest to many DUG members. Here are some headlines:

"Private household net wealth in Great Britain totalled £9 trillion in 2006/08, according to the Wealth in Great Britain report published today by the Office for National Statistics. The report presents results from the new Wealth and Assets survey, which found that wealth held in property and private pensions each account for 39 per cent of the total amount of private net wealth. Net financial wealth and physical goods, such as cars and antiques, each accounted for 11 per cent.

The report also shows how wealth was distributed across households. Median household net wealth was £204,500 in 2006/08. The least wealthy half of households accounted for only 9 per cent of wealth, while the wealthiest 20 per cent of households had 62 per cent of total wealth. The least wealthy 10 per cent of households had negative total net wealth.

Estimates of wealth varied across Great Britain. The wealthiest area was the South East of England with a median household wealth of £287,900. The area with the lowest median was Scotland with £150,600."

You'll find more details including the 174-page report at: http://www.statistics.gov.uk/StatBase/Product.asp?vlnk=15074


December 9, 2009

Financial Times - "Data giveaway aims to bolster growth"

The Financial Times (8 December 2009) reports the latest exciting news of Making Public Data Public:

"Masses of mapping, weather and public transport data that the government currently sells are to be given away in a move that ministers hope will create scores of new businesses whose earnings will boost both the economy and taxes.
In a revolutionary initiative, a range of Ordnance Survey, Met Office, public transport and postcode data will be made available for commercial reuse. The step will cost the organisations involved many tens of millions of pounds in current income but John Denham, the communities secretary, said it could create billions of pounds worth of business over the years as entrepreneurs found new ways to deploy it........"

Read the full article at: http://www.ft.com/cms/s/0/cdcc60a2-e399-11de-9f4f-00144feab49a.html


November 20, 2009

Public Administration Select Committee - Census Questions - & DUG mentioned on Radio 4

The House of Commons Public Administration Select Committee held its session on 2011 Census questions yesterday.
The MPs focused on Sexual Orientation, Migration & Religion - & of course we witnesses raised Income.
The hearing can be viewed on the web at: http://www.parliament.uk/parliamentary_committees/public_administration_select_committee.cfm 

& the transcript should be published in a couple of weeks too.

Radio 4's Today programme picked up the story this morning (c.6.45) & included a clip DUG's views on a question on sexual orientation ("well down our list of priorities").#

#. This exchange started:
Q: What do you think about sex?
A: It's OK (laughter)


November 20, 2009

Making Public Data Public - more news

This news, which is potentially of great significance, is moving fast.

Here's the latest instalment from the Guardian's Free Our Data Campaign, which starts:

"The Free Our Data campaign has scored a major victory, with the announcement by the government that it intends to make Ordnance Survey maps free for use online by any organisation – including commercial ones – at resolutions more detailed than commercial 1:25,000 Landranger maps from April next year.
The announcement of the opening of a consultation on the plan by Gordon Brown at Downing Street on Tuesday, as part of a seminar on making public data public – set in the wider context of public service reform, under the "Smarter Government" umbrella – indicates that the ideas underpinning the campaign have now been taken on board at the highest levels of government........"

Read more at: http://www.guardian.co.uk/technology/2009/nov/19/ordnance-survey-maps-free-online


November 17, 2009

Breaking news: Better access to Ordnance Survey data

Here's what looks like significant news from the Advisory Panel for Public Sector Information (APPSI). It's picked up a CLG press release which starts:

“The Prime Minister and Communities Secretary John Denham will today announce that the public will have more access to Ordnance Survey maps from next year, as part of a Government drive to open up data to improve transparency. Speaking at a seminar on Smarter Government in Downing Street later today, attended by Sir Tim Berners-Lee and Professor Nigel Shadbolt, the Prime Minister will set out how the Government and Ordnance Survey, Great Britain’s national mapping agency, will open up its data relating to electoral and local authority boundaries, postcode areas and mid scale mapping information”………

We’ll no doubt learn more detail - and whether the OS dam has finally cracked - in tomorrow’s press. (The Guardian tried to contact me this afternoon).


November 13, 2009

Department for Transport - National Public Transport Gazetteer

Following my message in September about public transport access points (see below), the Department for Transport has also announced its National Public Transport Gazetteer: " A topographic database of towns and settlements in the UK providing a common frame of reference for UK Public Transport Information schemas. The appropriate naming of towns and places is vital for providing effective place and stop finding in all modern on-line journey planners and other Passenger Information systems".

Detail can be found at: http://www.nptg.org.uk/


November 12, 2009

Conservatives seek to cut 2011 Census questions; + House of Commons Committee hearing

Here's some potentially serious news about the 2011 Census.

Nick Hurd MP, the Shadow Cabinet Office minister has written to the head of the UK Statistics Authority (Sir Michael Scholar) saying "We do not believe that the 2011 Census can have the size and scope that has been proposed, particularly since we believe that many of the proposed questions are unsuitable........and urge you to scale back its cost and scope in order to save money and reduce its intrusive impact on citizens".

The Public Administration Select Committee is now to hold a session next Thursday 19 November on the 2011 census questions. Fortunately, I've been asked to give evidence at the hearing, along with Prof. Phil Rees (Leeds University) & a representative of the Equality and Human Rights Commission.


November 6, 2009

Appointment of new head of ONS - Stephen Penneck

I'm delighted to report that Stephen Penneck - winner of the DUG Award this year - has now been appointed as the new boss of ONS.


October 31, 2009

2011 Census plans across European countries

A conference on Population and Housing Censuses across Europe took place in Geneva last week (28-30 October).
All the papers are available at: http://www.unece.org/stats/documents/2009.10.census.htm

The great majority of the 100+ participants were from the Census Offices as suppliers, but I had the chance to talk about "Meeting the needs of Census users in the United Kingdom’s private sector".

For me, the headlines of the conference were:

Data collection:
* The increasing use of administrative registers (good news), although traditional Censuses are still in the majority; there's also increased use of sampling (bad news for small area stats)
* The importance of comprehensive address registers, to get maximum coverage

* Progress on harmonising topics / questions
* BUT, geographical detail is very variable
* Eurostat are interested in Regions & LAs (rather than small areas), and are getting bogged down in trying to deliver multi-dimensional tables (their planned hypercubes run into problems of disclosing info about individuals)
* I made the case (& got it recorded) that the major interest of UK users is for the smallest possible areas, even if for limited variables. This got picked up, with the focus for a meeting next year on dissemination and geo-visualisation.

If you are interested in pursuing the availability of data for other European countries, do get back to me.

Keith Dugmore


October 16, 2009

DUG's 2009 Conference - News 

Here's news of our Conference "Insight and the recession – How we can achieve more and better for less” which was held on 8 October:

* Feedback - this was probably best summarised by Rob Harris, Head of Branch Knowledge at Barclays: "The value this conference has added, and the benefits to my team, are alone worth the annual DUG subscription"

* The presentations are now on our website (see the tab)

* The DUG Award for “Better information from government” was scooped by Stephen Penneck of the Office for National Statistics for the decision to code the ONS’s sample surveys with its OAC geodemographic classification. To quote the citation: "DUG's Conference Board sees this as an excellent example of government using existing sources to create valuable new information at modest cost, which is so important in the current economic climate." 


October 15, 2009

Statistics User Forum conference - 19 November 2009 

Here below are details of this year’s Stats User Forum conference on “The measurement of progress”.

The world is changing. The financial crisis has shown that the gains from an excessive focus on debt-fuelled economic growth were short-lived and failed to deliver a sustainable increase in well-being. The pursuit of ever larger economies led to environmental degradation and global imbalances rather than a reduction in poverty, an increase in social justice or a happier society. Post-recession, leaders and their electorates will strive for a greater focus on sustainability to ensure that future gains will be longer-lasting. But what will be the new goals and how will progress towards them be monitored?

See more details at: www.rss.org.uk/sufconference 


October 14, 2009

BBC - Bringing the fight for data home

Following in the trail of the Guardian's long-running Free Our Data campaign, we now have coverage from the BBC:

" Bringing the fight for data home.
The data held by local councils has become the latest target of digital activists. They are keen to get at the information so citizens can put it to their own uses.
Before now the push to get at official data has concentrated on central government. Moves to open up that data took a big step forward in early October when a few web developers were invited to a trial of the data.gov.uk site."

Read more at: http://news.bbc.co.uk/1/hi/technology/8297572.stm


October 6, 2009

Postcode news: UK (bad) & Republic of Ireland (good)

We have both bad & good postcode news today:

1. Legal threat closes postcode feed (in the UK) - news from the BBC

"Websites that help people find jobs or hospitals have been hit by legal action threatened by the Royal Mail. The threat was issued against the company supplying them, and many other sites, with postcode data. Royal Mail said the legal action was threatened to stop "unauthorised access" to the postcode data." Read on at:

2. Online retailers welcome postcode system for Republic of Ireland - news from Retail Week

"Retailers have welcomed the revival of plans to introduce a national postcode system in the Republic of Ireland and believe the move should help catalogue and online businesses operate efficiently there. Other than Greece, Ireland is the only EU member state that does not have an established postcode system, which means that offering a home delivery service is a complex and often expensive challenge. But last week the Irish government said it intends to introduce codes comprising digits and letters in 2011.". There's more at:

You'll find further detail & comment at:




October 2, 2009

MRS Census & Geodemographics Group Seminar - Targeting for multi-channel marketing - Monday 23rd November (afternoon)

The next CGG seminar has just been launched, on the important topic of Targeting for multi-channel marketing.

When: afternoon of Monday 23rd November
Where: MRS, 15 Northburgh Street, London EC1

Here's a synopsis of the topics to be discussed, followed by a link to the MRS website - for further details and online booking.

Today's consumers live in a multi-channel maze, targeted by advertising, direct marketing and sales calls. And in the online world, they also receive email marketing, search engine ads and web site offers. With such a range of offline and online channels at their disposal, how do companies select their marketing resources most effectively and measure the ROI? And is there still a role for geodemographics in targeting across offline and online channels?

This half -day seminar will discuss these topics and look at data for targeting, whether online or offline, and some of the latest analytic techniques being applied to multi-channel media. Further details and online registration are available at:


September 30, 2009

DUG Conference, The Royal Society, 8 October 2009

"Insight and the recession – How we can achieve more and better for less”


9.30 Registration and coffee

10.00 Chair’s introduction. Richard Alldritt (UK Statistics Authority)

10.15 Demonstrating the value of insight at O2. Andrew Day (O2)

10.45 ONS’s business case for the 2011 Census. Glen Watson (ONS)

(11.15 Coffee)

11.40 Using OAC to create new insight from the Expenditure & Food Survey. Professor Martin Callingham (Birkbeck College)

12.10 Google’s vision for more and better information. Kelly Allison (Google)

(12.40 Lunch)

14.00 Parallel workshop sessions:
A) Triangulation: what’s the scope for comparing more information derived from company customer records and government administrative files? Facilitators: Martin Bellingham (The Children’s Mutual) and Roma Chappell (ONS)
B) How can we demonstrate the real value of customer and citizen segmentation? Facilitators: Ian Symis (Boots) and Sara Jones (DCSF)
C) What makes for a successful insight / research & information team? Facilitators: Mike Whitelegge (M&S) and Jenni Venn (Coventry City Council)

(15.30 Tea)

15.50 Workshops – summaries of views

16.10 Panel Session: More and better for less – how we can improve insight and its impact during the recession
Featuring views from Professor Mike Batty (UCL), Steve Penneck (ONS), Andrew Tasker (Optimum Locations & DUG), and Keith Dugmore (DUG)

16.45 Chair’s closing remarks, and The DUG Award 2009

17.00 Reception

The presentations will be posted on www.demographicsusergroup.co.uk


September 12, 2009

Department of Transport adds 350,000 public transport access points to Open Street Map

The Department for Transport has recently released data from the National Public Transport Access Node (NaPTAN) database to be put on Open Street Map (OSM). The data contains details of some 350,000 public transport access points in Great Britain including bus stops, railway stations, tram stops and ferry terminals, and meaningful text descriptions of the stop point and its location.

See lots more detail at:

It is also stated that Ordnance Survey has no rights to the data, and adds a comment: "This is excellent news - and big kudos to the DfT for donating the data! We hope that other departments consider following suit and adding their geodata to OSM!"


September 12, 2009

Guardian's Free Our Data - Ordnance Survey under fire again

Charles Arthur has written two more articles about Ordnance Survey - the most recent one is a cracker:

3 September, 2009: "Ordnance Survey business model costs appear in leaked presentation"


10 September, 2009: Don't write it down: Ordnance Survey kept no notes of talks with 'expert'

"You have an important study that will be part of your case to a government minister considering the future of your organisation. You have found someone to review it. Would you commission them without swapping letters or email, and without taking any notes of their observations afterwards?
If you're Ordnance Survey, then it seems yes, you would. In a response to a Freedom of Information request – part of a series of requests Guardian Technology has made since May – OS revealed that after it prepared an internal study comparing international models of funding mapping agencies, it asked an "internationally ­ recognised expert in geographical information and national mapping" to review it.
But in an admission that makes the preparation of this potentially key report look rather slipshod, OS said in response to a further FoI request that "a copy of the report was provided to the person concerned and engagement on this matter was conducted orally with no permanent record made of these conversations".........

Read on, at http://www.guardian.co.uk/technology/2009/sep/11/free-our-data-freedomofinformation


August 28, 2009

ONS's latest Population & Migration estimates - UK population grows to 61.4m; birth rate rising

Here below is the latest population news from ONS.
You'll probably have seen lots of press coverage today, e.g. "New baby boom for Britain" (Daily Telegraph):

[what a surprising web address / index!]


August 19, 2009

"Data Strategy" - MP attacks "scandal" of Census address data

Read all about it at:



August 7, 2009

Jil Matheson appointed as the new National Statistician

I'm delighted to report that Jil Matheson has been appointed as the new National Statistician.

The official announcement is at http://www.statisticsauthority.gov.uk/news/appointment-of-the-national-statistician.pdf

The scope of the job has been increased, with a wider remit across government, and involvement in the recruitment of a new Director General for the ONS.

Jil has been very supportive of DUG for many years, and at last year's Conference she was on the panel session.


July 30, 2009

Guardian's Free Our Data - "Government advisors criticise OS's strategy"

Charles Arthur writes:

"Ordnance Survey's proposed business strategy has been strongly criticised by the government's Advisory Panel on Public Sector Information (APPSI), the body that advises ministers on how to encourage wider reuse of public sector data. Separately, OS has finally released an almost unredacted version of its internal report comparing international funding models for mapping agencies – but has declined to name the "internationally recognised expert" who checked it, saying the person wants to remain private......"

and concludes with:

"..............Separately, OS responded, at the end of the last possible day under Freedom of Information Act rules, to Guardian Technology's request for a copy of its internal report on funding models. Almost all the data redacted in the first version has now been released – but nothing seems to back up the statement in May by Sir Rob Margetts, OS's chair, that it showed that moving to a free data model would in the first five years cost "between £500m and £1bn". Guardian Technology will seek to ascertain the basis of Margetts's claims, and any more information about the identity of the expert who checked the study – such as, for example, whether they are an employee of Ordnance Survey."

You can read the full article at http://www.guardian.co.uk/technology/2009/jul/29/free-our-data-ordnance-survey

July 15, 2009

Seven out of Ten Woolworths stores still empty

Here’s a news item from The Local Data Company:

"We recently undertook some research into what happened to over 800 Woolworths stores following Woolworths demise in January - below is some of the commentary from the press:







July 11, 2009

National Address Register - support from MPs & the UK Stats Authority

Getting a definitive national address register is one of DUG's priorities, and we are making progress.

At the House of Commons Public Administration Select Committee session on the Census on 25 June Gordon Prentice MP questioned the head of ONS & quoted our submission:

"But these organisations [Royal Mail, Ordnance Survey and Local Government] hold intellectual property rights on the information that they are making available and you have to buy this information from those three organisations. I have been reading the submission that we have from the Demographic User Group and they say that it is scandalous - I thought I had copyright on that word - that you are spending all this money pulling together this information to create an address register and it is only going to be used once; it is only going to be used for the next Census and it is not going to be made available to commercial organisations and other people who could use that information. That really is scandalous, is it not?"

On 8 July the Chair of the UK Statistics Authority, Sir Michael Scholar, wrote to the Minister of State for Housing, John Healey, making the case for establishing a national address register.


July 2, 2009

"Ordnance Survey redacts public data in controversial report"

The Guardian's Charles Arthur's latest instalment: 

"A controversial OS study, released in a censored version, leads to a single conclusion: the original was incomplete and fundamentally flawed.......
Are the land mass, population and GDP per capita of the US, Australia, Canada, France, Sweden, the Netherlands, Norway and New Zealand – as well as the legal form of their national mapping agencies – commercially confidential information? If you believe the UK's mapping agency, Ordnance Survey, certainly. Those details were redacted – blacked out in the manner of an MP's expenses – in the copy of a key study it carried out and released to the Guardian under the Freedom of Information (FOI) Act last week.....
Ed Parsons, the former OS chief ­technology officer who is now Google UK's geospatial technologist, dismissed the report: "OS's 'Research' Report reads like a poor MSc thesis. Why no user ­perspective?" "

Read all about it at: http://www.guardian.co.uk/technology/2009/jul/01/ordnance-survey-censored-report-released


June 25, 2009

"Ordnance Survey responds, at last, to Freedom of Information request"

The Guardian's Charles Arthur reports:

"The Ordnance Survey consulted mapping agencies in Australia, Canada, France, the Netherlands, Norway, New Zealand, Sweden and the US in compiling its study into possible financial models, it has revealed. However, OS considers that "there are no wholly equivalent ­organisations ... given [our] range of scales of mapping and other activities", according to data that it released under a Freedom Of Information (FOI) request from the Guardian.

Attention was immediately drawn to the identity of an unnamed "internationally recognised expert in geographical information and national mapping" who OS said had reviewed its study and "agreed with the analysis and conclusions......."

Read all about it at: http://www.guardian.co.uk/technology/2009/jun/24/free-our-data-ordnance-survey 


June 18, 2009

OAC User Group free seminar: Developments in Geodemographic Visualisation

The OAC (Output Area Classification) User Group is going from strength to strength, and has arranged the following meeting in September:

* Seminar: Developments in Geodemographic Visualisation

* Objective: The aim is to present a number of innovative developments in visualisation methods and technology which can enhance interpretation of geodemographic classification. This seminar is aimed at both a professional and academic audiences and will be interest to all users of geodemographics.

* Programme:
– Understanding Through Visualisation (Dan Vickers, University of Sheffield) – Effective Communication (Laura Edwards, Local Futures) – Geographic Visualisation (Alex Singleton, University College London) – Data Visualisation (Aidan Slingsby, City University) – 3-D Visualisation (Duncan Smith, University College London)

* Time & Date: 1.30pm, Tuesday 15 September 2009

* Venue: Royal Statistical Society headquarters, 12 Errol Street, London, EC1Y 8LX

* Further details & Registration: http://areaclassification.org.uk/2009/06/16/visualisation/#more-208

I expect that places will be snapped up quickly, so do register (not me - see above!) now if you'd like to attend.


June 18, 2009

Guardian's Free Our Data campaign - 2 more good articles today

Today's Guardian has two articles by Charles Arthur:

1) Commercial rival to take on Ordnance Survey with its own map data

“Ordnance Survey has a new challenger: a privately owned company – UKMap, part of the GeoInformation Group – that aims to provide local authorities and private companies with large-scale topographic mapping and an address database for British cities, created entirely separately of the government-owned agency…………..
……its arrival means there are now three organisations vying to produce maps of the UK: Ordnance Survey, which derives roughly half its income from licences by taxpayer-funded organisations; OpenStreetMap, a volunteer effort using advances in GPS technology to create a "wikimap" used by some local authorities (and by the Guardian for its Open Platform offering); and now The GeoInformation Group.”

(See the full article at http://www.guardian.co.uk/technology/2009/jun/17/free-data-ordnance-survey-maps/print  )

2) Let Sir Tim lead the fight for free data

“….Gordon Brown has tasked Sir Tim Berners-Lee with getting government-collected data into more accessible forms, and hence available to more of us…….. Sir Tim nailed his colours to the mast in a riveting talk earlier this year, where he built on Hans Rosling's already thrilling work at Gapminder to emphasise the value of getting data out there into the hands of people who can think of useful, innovative, imaginative things to do with it; he got the audience to chant "Raw data now! Raw data now!"…… [Even we haven't done that]

(See the full article at http://www.guardian.co.uk/technology/2009/jun/17/charles-arthur-free-data/print  )


June 5, 2009

ONS advice on measuring change in small areas

ONS is providing some good advice on analysing change in small areas at:

You can also get hold of a free copy of their Change over Time Analysis (CoTA) Viewer on CD at:


May 16, 2009

ONS admits to getting its sums wrong with overstated retail sales - FT 16 May 2009   

Following the discussion of members' doubts about ONS retail stats at our Awayday in December, the Financial Times (Saturday 16 May 2009) reports:

"One of Britain's most closely watched economic indicators has heavily overstated the quantity of high street sales over the past two years, the Office for National Statistics admitted yesterday. Britain's supplier of official statistics conceded that since the financial crisis began in August 2007, it has overstated the volume of retail sales growth by 56 per cent."

Read all about it at: http://www.ft.com/cms/s/0/97faf9b2-41b3-11de-bdb7-00144feabdc0.html?nclick_check=1


May 16, 2009

Shop vacancy rate in towns rises to 12% - FT 16 May 2009   

The Financial Times (Saturday 16 May 2009) reports:

"The number of empty stores in the UK's high streets leapt three-fold in just six months as a growing number of retailers were forced out of business by the worsening economic climate.
High-profile casualties such as Woolworths contributed to a rise in vacancy rates from 4 per cent to 12 per cent in the six months to the end of March. Vacancy rates in terms of floorspace are lower, at 10.8 per cent.
Barnaby Oswald, managing director of the Local Data Company, which surveys 675 town centres across the UK, said that the number of businesses closing has increased significantly since last summer"

Read all about it at: http://www.ft.com/cms/s/0/b7a7955e-41b3-11de-bdb7-00144feabdc0.html


May 8, 2009

Ordnance Survey - "new" business strategy - further criticism in The Guardian   

Following the news of Ordnance Survey's "new" business strategy (see below), the Guardian has two more articles critical of the government's plans:

* Budget misses free chances (Michael Cross, 30 April 2009):

"The government has thrown away an opportunity to boost the digital economy through a "free data" regime. Announcements made with the budget dashed hopes of radical reforms to the "trading fund" status of state-owned information businesses.
Even worse, the plans do not deal with "derived data" - one of the most contentious fields in intellectual property creation, whereby the creator of a product claims ownership of any IP built with it. Ordnance Survey's aggressive annexation of "derived data" - any detail, such as a cycling route, created using an OS product - has riled a number of people in the commercial sector. "Derived data is the elephant in the room," says Ed Parsons, Google UK's geospatial technologist..........."
Read all about it at: http://www.guardian.co.uk/technology/2009/apr/30/government-digital-economy

* Break down the boundaries in Ordnance Survey (Charles Arthur, 7 May 2009):

"The government's decision to consult the public on its reorganisation of Ordnance Survey - announced with the budget last month - is proving popular with would-be users. The proposed reorganisation, however, is not.
Comments on the web page, where the consultation is open for feedback are running overwhelmingly in favour of more freedom for OS data............"
Read all about it at: http://www.guardian.co.uk/technology/2009/may/07/ordnance-survey 


April 26, 2009

Ordnance Survey - "new" business strategy announced    

Triggered by the Budget, a "new" business strategy has been announced for Ordnance Survey.
(I put the "new" in quotes. The strategy has at least avoided the madness of a rumoured privatisation, but has not gone for the radical option of freeing the data to increase both use and consequent economic activity. Rather, having hit the internet and Google Maps icebergs, it is another attempt to rearrange the deckchairs on the Titanic).

Here are the headlines, followed by Michael Cross's Free Our Data view in the Guardian:

1) The OS Announcement

"The strategy aims to improve ease of access to geographic data and services for both commercial and non-commercial use, whilst also striving to maintain a balance with the need to stimulate innovation in the geographic information market and make data more widely available. This means that Ordnance Survey will continue to be self-funded and earn revenue by licensing its data, but it will make sure it is easier for customers and other businesses to access its data and services.

The strategy focuses on five key areas:
• Promoting innovation – with an enhanced free OS OpenSpace service to allow experimentation with digital information and a clear path from this service to greater commercialisation;
• Reforming Ordnance Survey’s licensing framework – so that it is much simpler to use Ordnance Survey data and services in other applications;
• Reducing costs over time – to ensure that Ordnance Survey continues to offer value-for-money;
• Supporting the sharing of information across the public sector – to enable better public policy and services;
• Creating an innovative trading entity – to explore commercial opportunities around providing a better platform for consumers to access Ordnance Survey products.

The new strategy will be developed further and implemented over the coming 12 months but the five key areas above are being opened up for comment from Ordnance Survey’s customers and other stakeholders."

Full details of the strategy and how to comment can be found at: http://strategy.ordnancesurvey.co.uk/

2) Guardian Free Our Data article: Government ducks free data decision (23 April 2009)

Michael Cross refers to the headline finding that "a model where a user pays a licence fee for OS data continues to be the most effective way of balancing the need to increase the availability of geographic information to the wider UK economy and society while maintaining the quality of OS data". He considers the extension of OS’s OpenSpace service as a win for the Free Our Data campaign. However, he believes that the new subsidiary company set up to “ensure that OS is sustainable for the medium term and value is generated for the taxpayer" implies that this "innovative trading entity" has been established with an eye to privatisation – although, he points out that Iain Wright, shareholder minister for OS says this is not the case.

The full article can be accessed at: http://www.guardian.co.uk/technology/2009/apr/23/state-owned-digital-information 

I'll be interested to hear your views. In particular, have you got opinions about OS OpenSpace?

April 19, 2009

"Social Trends 2009" published    

You probably saw some lurid headlines on Wednesday's newspapers - the latest Social Trends report has been published.
Amidst all the comment about social change in Britain, the report is often a good first start for finding out what are the key data sources for particular topics.
You can download it (or bits of it) free from: http://www.statistics.gov.uk/socialtrends39/

April 9, 2009

UK Statistics Authority report on migration statistics    

This report has just been published. It is policy stuff, rather than providing new stats now, but its recommendations (Pages 8-11) reflect our views on the need for an address register, & more use being made of government administrative data:

"We recommend that, in the light of the complex dependencies involved, discussions about matters such as the future of the Census beyond 2011, the longer term maintenance of a national address register, related developments (such as identity cards and a population register) and the best way to improve migration statistics, should now be taken forward together at the highest level. Discussions should involve the Statistics Authority and ONS and the relevant government departments – notably HM Treasury, the Cabinet Office, Department for Communities and Local Government, Home Office, Department for Work and Pensions, and the devolved administrations."

You can download the full report at:


April 8, 2009

European Geodemographics Conference - 1 April - Presentations now available    

This MRS conference broke new ground, looking at the availability of demographic data across Europe.

The presentations are now available on the MRS website at: http://www.mrs.org.uk/networking/cgg/euro_conference_prog.htm

A report on the conference – including the action points for the future – will be posted on the website in the next couple of weeks and there will be an article in the June issue of GeoConnexion.


March 18, 2009

Google Street View    

Google Street View  - providing recent photos - has just been released: http://maps.google.co.uk/ There's coverage of Greater London and c.20 other cities. Drag the person icon onto the road that you want to see.

Early reactions from DUG members:

"Already tested it out in Manchester - very impressive - will make mapping exact city centre locations much more accurate."

"We were looking at this this morning, looks fab ..we have been virtually wandering around Nottingham and Edinburgh."

"We had an enjoyable 5 minutes finding Tony Blair's house and discovering that they haven't pixellated the policeman's face outside. Google seems to hit the nail everytime at the moment !!!"

"We spent a good 20 minutes on this earlier - absolutely fantastic stuff - love the zoom feature, the 360 toggle, and there is a good coverage across a lot of the major towns and cities (and Scunthorpe). Picture quality and load rate seems to be very impressive."


March 10, 2009

Availability of services at Output Area level    

Here’s news of a new dataset from the Commission for Rural Communities (in England):

“Service availability and accessibility has been a key rural policy issue for many years. In order to monitor changing levels of service availability, this indicator combines datasets that show where outlets of particular services can be found, with postcode level information on the distribution of households to calculate numbers of households within distance of key service indicators.”

Looking at the link: http://www.ruralcommunities.gov.uk//projects/ruralservicesseriesdata/overview

• The input data includes the locations of banks, building societies, ATMs, supermarkets, pubs, etc.
• The output data appears to be for all Output Areas, not just rural ones (the data file is very large)
• The website includes some maps


March 5, 2009

Guardian - Free Our Data - The price of un-free data: one-off census address list will cost £12m    

Michael Cross, writing in The Guardian, is adding to the pressure for the ONS's 2011 Census address file to be made freely available:

"The cost of creating a national list of addresses for the next census has risen to £12m, the government has admitted. The figure – £2m up on previous estimates – is the latest revelation of the true cost of "unfree" data across the economy....The Office for National Statistics has to build its own address database for the 2011 census in England and Wales (Scotland has its own parallel exercise) because none of the three existing, state-owned address databases is comprehensive enough.......Attempts to create a unified list have failed because of arguments about intellectual property. The three list-owners – local authorities, Ordnance Survey and Royal Mail – all treat address lists as their commercial assets."

See Michael's article at: http://www.guardian.co.uk/technology/2009/mar/05/free-data-addresses


March 4, 2009

Power of Information Taskforce Report published   

This report (written by some influential people in government) has just been published.

It calls for action in several areas where the Taskforce believes significant improvements can be made to government's use of digital technologies. These include:
* freeing up the UK's mapping and address data for use in new services.
* ensuring that public sector information is made as simple as possible for people to find and use.

Looking at Recommendation 7 in particular:
"It is the Taskforce's view that 'freeing up' geospatial data should be a priority. The Ordnance Survey requires urgent reform. Recent announcements of cost reductions at the Ordnance Survey point the way to wider reforms. This reform should include at a minimum:
* Basic geographic data such as electoral and administrative boundaries, the location of public buildings, etc should be available for (re)use free of charge to all.
* There should be simple, free access to general mapping and address data for modest levels of use by any user.
* Voluntary and community organisations pursuing public policy objects should benefit from straightforward standard provisions for ensuring access to geospatial data at all levels of use.
* Licensing conditions should be simplified and standardised across the board and, for all but the heaviest levels of use, should be on standard terms and conditions and should not depend on the intended use or the intended business model of the user.
* The OpenSpace API, similar to but currently a constrained version of Google Maps, should become the primary delivery point for the Ordnance Survey's services.
* Creation of a freely available single definitive address and postcode [file] available for the UK for (re)use."

All this is heading in our direction (albeit in difficult economic & political times).

Read more at http://poit.cabinetoffice.gov.uk/poit/


January 28, 2009

Experian in the news   

Here's a story in yesterday's Times - "Lord Taylor tried to aid credit-rating agencies access to business" - which includes mention of Experian.


January 24, 2009

How geodemographics is being applied across an ever expanding Europe

MRS CGG Conference - 1 April 2009, London   

This year's Market Research Society Census & Geodemographics Group conference focuses on European data, and on how analytical techniques such as geodemographics are being used across Europe to help organisations to optimise their definitions of target markets.

This will be considered from a number of perspectives, ranging from data availability through to business applications, and will drive towards setting an agenda of priority actions required to fill existing gaps in the European data landscape. The programme includes speakers from CACI, Eurodirect, & Experian.

For full details and to book your place, see: http://www.mrs.org.uk/networking/cgg/euro_conference.htm


January 21, 2009

Dwelling stock by council tax band, 2007 - statistics for small areas  

(From a message to  DUG Contacts)

Data for Neighbourhood Renewal has alerted me to the following new statistics now available on ONS's Neighbourhood Statistics website:
* Dwelling stock by council tax band (Updated for 2007) - down to Output Area (OA) level for England & Wales: http://www.data4nr.net/resources/53/

We now have datasets for each year since 2001, which in principle enables the tracking of changes in numbers of dwellings and their status (Bands A-H) for very small areas.

If you've done any analysis, I'll be pleased to hear.


January 8, 2009

Guardian - Free Our Data - Royal Mail is wiping many rural homes off its postcode database 

(From a message to  DUG Contacts)

Here's Michael Cross's latest article in today's Guardian:

It includes:
"Royal Mail says it has a policy of removing addresses from the database when houses are unoccupied. "If the postie can no longer reach the delivery point, or if a house is obviously completely unoccupied, the postie informs us and the address is removed from the PAF. If it later becomes occupied, it would be put back on""

He then goes on to point to the feuding over address files between Royal Mail, Ordnance Survey and local government.


December 21, 2008

Members' Locations - latest news

(From a message to  DUG Contacts)

I'm pleased to let you know that our new member Boots is planning to join in our initiative to share information about locations, and that we are also planning quarterly updates.
For the record, the current list of sharers is:

* Abbey
* Argos
* Boots
* Co-op
* John Lewis
* M&S
* Sainsbury's

In addition, Tesco have pointed me to a SatNav link which may be helpful:



December 18, 2008

The Times 18 Dec 2008 - Analysis: Why retail figures don't add up

(From a message to DUG Contacts)

Following our discussion at our Awayday, I've been alerted more doubts about ONS's retail stats. 
Here's the link:


[This was the first message on our new website]