Welcome to the Demographics User Group
Representing the needs of commercial users of information
Get alerted to future DUG news stories by entering your email address in this box....
DUG: More News
July 29, 2010
"Ordnance Survey Leading The Way On Government Transparency"
The Department for Communities and Local Government has issued a Press Release which starts:
“The national mapping agency is the pioneer in a new era of information transparency by Government, Communities Minister Baroness Hanham said last week.
Three months after launching the OpenData initiative, Baroness Hanham said Ordnance Survey is leading the race to make public data freely available and accessible........”
You can read it all at: http://www.egovmonitor.com/node/37518
Given OS’s bitter resistance over the last decade to opening up free data, this spin is truly boggling. George Orwell would have been proud.
But, better late than never, and it’s good to see the new Coalition strongly committed to making public data freely available.
_____________________________________
July 29, 2010
Job vacancies – Tesco & Boots
For details, see the tab to the left.
_____________________________________
July 27, 2010
2011 Census to proceed as planned – Ministerial Statement
The UK Statistics Authority has just issued a brief Press Release:
“The UK Statistics Authority welcomes the Government’s decision to proceed with the existing plans for the 2011 Census in England and Wales, set out in an answer by the Minister for the Cabinet Office, Rt. Hon. Francis Maude MP, to a Written Parliamentary Question.
The Office for National Statistics’ preparations for the Census on 27 March 2011 will now press ahead with all speed.
The Statistics Authority is determined that, with the full support of the Government and all the other parties concerned, the 2011 Census will be the success that the country needs it to be, and will provide the information about our population which can only be derived from the full-scale Census, which has now been confirmed.”
http://www.statisticsauthority.gov.uk/news/2011-england-and-wales-census.html
So it appears that Francis Maude – despite previous hostility – has now been convinced. Phew.
_____________________________________
July 22, 2010
OAC conference – PM, Monday September 6th, 2010 – free
Here’s news of this year’s OAC (Output Area Classification) conference. The programme is pasted below. The emphasis is on use in the public sector, but it includes sessions on the British Population Survey (an alternative to the TGI), and Open Data, Free Tools, and there will no doubt bee wider discussion & ideas. There is more information on the OAC User Group website: http://areaclassification.org.uk/
You can book (no charge) at: http://oac.eventbrite.com
_____________________________________
July 12, 2010
"National Census to be axed after 200 years" – Daily Telegraph – DUG's letter published
Our letter has been published in today's Daily Telegraph:
"Counting the population
SIR – The Government has decided that next year’s Census will be the last (report, July 10). Hitherto, censuses have provided detailed counts of the population that are vital for both public and private organisations when deciding where to target investment, and for the public, to inform democracy. Census returns released after 100 years are also of interest to family researchers.
Francis Maude, the Cabinet Office minister, is right to examine different and potentially cheaper ways to count the population. In particular, the Government’s own extensive databases would be much the best starting point, learning from the experiences of several Nordic countries.
However, to announce a decision to scrap future censuses before proving that alternative sources are adequate is reckless.
Keith Dugmore
London SW1"
& can be found on their website at:
http://www.telegraph.co.uk/comment/letters/7884088/Reducing-our-Armed-Forces-will-leave-Britain-dangerously-reliant-on-our-allies.html
_____________________________________
July 3, 2010
UN / Eurostat Census meeting – papers on Censuses overseas
For those of you interested in Census information for countries outside the UK, here's news of the "Joint UNECE/Eurostat Group of Experts on Population and Housing Censuses", which takes place in Geneva, 7-9 July:
http://www.unece.org/stats/documents/2010.07.census.htm
You'll see papers from many countries, including one from me on "Disseminating Census information to maximise use and value", which highlights the need for a better balance between statistical disclosure risk, and utility, guided by Jeremy Bentham’s view: “It is the greatest good to the greatest number of people which is the measure of right and wrong.”
_____________________________________
June 21, 2010
British Population Survey – coded with geodemographics, including OAC
Here’s news of the British Population Survey (a commercial survey, a bit like the Target Group Index), which is now coded with OAC and other geodem classifications.
http://www.thebps.co.uk/#/segmentation/4540464055
You’ll see the word FREE (I understand that only the most recent year is charged).
_____________________________________
June 19, 2010
News of an open license for a NSPD postcode lookup file
Here's news from the OAC User Group:
"An open license for the non PAF version of the National Statistics Postcode Directory postcode lookup file is now available. The terms of which enable a user to:
• Copy, distribute and transmit the Data
• Adapt the Data
• Exploit the Data commercially whether by sub-licensing it, combining it with other data or by including it in your own product or application
This is great news as it includes OAC as one of the variables and makes address lookups a much simpler process. We shall update the tutorial guides on the website soon to reflect this change"
The ONS website goes into more detail, starting:
"The NSPD is released quarterly in February, May, August and November and NSPD products are supplied with various elements of Postcode Address File (PAF) data depending on the level required. These extracts are available as Full PAF or Part PAF options. Additionally, an ‘Open’ version, containing no PAF elements (and not attracting Royal Mail royalties) is also available......."
See more at: http://www.ons.gov.uk/about-statistics/geography/products/geog-products-postcode/nspd/index.html
_____________________________________
June 19, 2010
More free data: a) Transport for London; b) Postcode grid references & lat / long
APPSI have just alerted me to more free data:
1) Transport for London - as reported in The Guardian:
"Transport for London has opened up huge swathes of its data for free re-use by developers keen to see what they can do - and build - with it. After years of lobbying by developers who have wanted to be able to exploit the data - which, being publicly funded, logically belongs to the public too - TfL today acknowledged its willingness to listen to what people want to do as the London Data Store, headed by Emer Coleman, announced the datasets that will be provided.........."
Read all about it at: http://www.guardian.co.uk/technology/blog/2010/jun/15/london-datastore-tfl-data-free
2) Geopostcode:
"An API for retrieving geographic information (grid reference, latitude and longitude) relating to postcodes. Data is available as XML or JSON" http://data.gov.uk/apps/geopostcode
_____________________________________
June 10, 2010
UKSA interim report "Strengthening User Engagement" – DUG's response
Here is DUG’s response to the UKSA consultation. It makes the case for going further than just better user dialogue, with UKSA's monitoring role being more proactive, the government statistical service becoming more customer focussed - and adding a proposal to rename the Government Statistical Service the National Statistical Service – to reinforce the need to serve all customers, not primarily those in central government.
_____________________________________
June 8, 2010
Independent newspaper – The real cost of a compromised census will be inaccurate data – DUG's views quoted
ONS has just alerted me to an article in last Saturday's Independent:
"With less than a year to go to the 2011 Census, is Francis Maude about to change the rules? In opposition he was strongly critical of the Census, describing it as burdensome, intrusive and poor value for money. Now he's in charge, as Minister for the Cabinet Office. He's also central to the programme of Government cuts that the coalition has promised. So demographers and statisticians are worried that he may make good on his promise to "scale back" the Census even at this late stage........
.....if he doesn't make any changes when he's now in a position to, he's going to have to eat a lot of words. He's already had a warning from the Demographics User Group, a private-sector body that represents heavy-hitters such as John Lewis, Marks & Spencer, Sainsbury's, Nationwide and Alliance Boots, saying that they regard the Census as of "fundamental importance" when making decisions about opening new stores, the products to be stocked, and the customers to be targeted. A late decision to cut the number of questions would disrupt existing plans and contracts, save little money – or even add to the cost – and result in reduced and inconsistent information between England and Wales, Scotland and Northern Ireland, says Keith Dugmore, director of the group........."
You can see the full article at:
http://www.independent.co.uk/opinion/commentators/nigel-hawkes-the-real-cost-of-a-compromised-census-will-be-inaccurate-data-1992013.html
_____________________________________
June 8, 2010
Big role for supermarkets in regenerating poor communities – report by Demos
Yesterday's Guardian reported: "Demos sees big role for supermarkets in regenerating poor communities" & argues for tax breaks for supermarkets if they can turn sink areas around: http://www.guardian.co.uk/uk/2010/jun/07/demos-supermarkets-poverty-regeneration-pride
& here's the link to the Demos website, where you can download the full report Civic Streets: http://www.demos.co.uk/publications/civicstreets
_____________________________________
June 5, 2010
Victory for campaigners as coalition releases data deluge
Today's papers have further news of the new government releasing more data into the public domain.
The Guardian's article http://www.guardian.co.uk/politics/2010/jun/04/government-spending-database-special-report focuses on COINS (Combined Online Information System) and information on government spending, whilst Charles Arthur reports "A flood of data is on its way ... but we will need to make sense of it"
http://www.guardian.co.uk/politics/2010/jun/04/coins-treasury-public-sector-data
Much of this won't be immediately relevant to us, but it does confirm the new government's general policy that government information should be in the public domain.
_____________________________________
May 26, 2010
Experian partners with The Local Data Company
Here’s some news from Experian about The Local Data Company (who presented to our meeting last September, and are used by several DUG member companies):
http://www.localdatacompany.com/press-releases/2010/5/26/experian-partners-with-the-local-data-company.html
You’ll see that it’s referred to as partnering, rather than continuing Experian’s recent history of buying small companies.
_____________________________________
May 20, 2010
Database valuation
Several members have been interested for some time in how a value might be put on their customer databases. At DUG’s most recent meeting Mark Perrett (E.ON) mentioned his recent discussions with Call Credit, who are researching the topic.
Mark has also unearthed two earlier documents from the Institute of Direct Marketing:
http://www.theidm.com/marketing-events/calendar/idm-data-council-valuing-your-database/
* White paper - Database Valuation: Putting a Price on Your Prime Asset.
* A case study in the valuation of a database prepared by Valuation Consulting.
_____________________________________
May 18, 2010
Ordnance Survey free "opendata" – more vector mapping released – 1:25,000 scale
Following up the news of 10 April, OS has now released another map dataset: "OS VectorMap™ District". It's nominally at 1:25,000 scale, and in both vector & raster formats.
Details are given at: http://www.ordnancesurvey.co.uk/oswebsite/products/vectormap/district/index.html
_____________________________________
May 18, 2010
Letter to the Rt. Hon. Francis Maude MP: The 2011 Census – its importance to commercial companies
The new Cabinet Office Minister Francis Maude has in recent months actively made the case to reduce the scope & cost of the Census (& one of the options now being mentioned is to scrap it completely). We have therefore sent him this letter to get the commercial users’ case for the Census in front of the new government without delay.
_____________________________________
May 6, 2010
2011 Census Output Consultation – Scotland – DUG’s Response
See our response to GRO Scotland here
_____________________________________
April 10, 2010
Ordnance Survey free "opendata": what does it actually include?
Following up my message of 3 April, here's the Guardian's Data Blog answer to the above question:
http://www.guardian.co.uk/news/datablog/2010/apr/02/ordnance-survey-open-data
If you have any obs about this treasure trove - maps, boundaries, postcode grid references, gazetteers - or, indeed, a wish-list of more government data you'd like to push to be made freely available, do get back to me.
_____________________________________
April 4, 2010
Job Vacancies – advertisements by DUG members
Several members are seeking staff – see the “Jobs” tab on the left.
_____________________________________
April 3, 2010
Major Government announcement – Ordnance Survey – significant datasets now FREE
Following the consultation on Ordnance Survey the Government has published its response:
http://www.communities.gov.uk/documents/corporate/pdf/1528263.pdf
The Guardian’s Free Our Data report http://www.guardian.co.uk/technology/2010/apr/01/ordnance-survey-maps-download-free starts off:
“It is the biggest shakeup in the Ordnance Survey's 260-year history: you can download maps as detailed as 1:10,000 scale, or collect a list of locations appearing on maps at the 1:50,000 scale, or a conversion system for postcodes to grid references – free for personal or commercial use.”
_____________________________________
April 1, 2010
London Regional Committee's first report, London's population and the 2011 Census
Here's the report http://www.publications.parliament.uk/pa/cm200910/cmselect/cmlonreg/349/34902.htm from the House of Commons committee to which we gave evidence. I'm pleased to see that DUG gets quoted.
You'll see strong support for the Census, but also the predictable concern about the dangers of a low 2011 Census response in London - and the need to target effort.
Also: "The Committee welcomes the creation of a national address register for the Census, which will greatly improve accurate delivery of census forms, but say it is 'barely credible' that this valuable resource is not to be maintained after 2011. The report recommends that the Government acts urgently to resolve the intellectual property issues which are preventing maintenance of the register past 2011."
_____________________________________
March 31, 2010
Orange joins DUG
Orange is the latest company to join the Demographics User Group (DUG), whose membership already includes major companies across a range of sectors.
Orange is one of the largest mobile operators in the UK, with a current consumer base of 17million. It has a wealth of internal data in its analytical environment, from point of sale through to individual call record - who, when, where, and how often a customer calls.
Orange’s Strategic Insight's team's main responsibilities are to provide: Customer segmentation; a behavioural view to increase value, loyalty & enhance the customer experience; bespoke deep dive analytics from a customer led approach for the whole of Orange.
Anthony Mawby, Head of Strategic Insight, stated "We are very pleased to be joining DUG, and are looking forward to sharing experiences of data, analysis, and insight with other major companies from a range of different sectors including retail, financial services, and energy."
_____________________________________
March 27, 2010
Department of Transport's 350,000 public transport access points "NaPTAN" – available free
Following my message back in September (see below), and the DfT then saying that commercial users have to pay, I've now checked about the data being part of the (free) www.data.gov.uk initiative. Here's the answer that I've received from the DfT:
"Thanks for your email. I had a quick look at the data.gov.uk website and snapshot bus stop location data is available to download at: http://data.gov.uk/node/8473
I've clarified the use of this data with colleagues at Transport Direct, and the snapshot data is freely available to all for use. However, this is snapshot data and so will already be out of date, as changes are made to the bus stop data almost on a daily basis. Therefore, if commercial users wanted to access more up-to-date information, then they would need to request a license and follow the procedures as set out on the NaPTAN website: www.dft.gov.uk/naptan .
I understand that updates will be made to the data held on the data.gov.uk website, but am not sure of the frequency of these."
_____________________________________
March 24, 2010
Prime Minister's speech: "Building Britain's Digital Future" (more news of free data, inc. Ordnance Survey)
See: http://www.number10.gov.uk/Page22897
_____________________________________
March 13, 2010
Google Street View - now extended to UK coverage
Google's Street View has now increased coverage for the whole UK: http://maps.google.co.uk/
To quote a geographer at UCL: "It is pretty amazing – just drag the little yellow man onto the map of the UK.....".
_____________________________________
March 13, 2010
Consultation on Ordnance Survey – DUG's response, + extra comment
Here's DUG’s response to the Department for Communities’ consultation on OS.
APPSI has also made a heavy duty response (led by David Rhind): http://www.appsi.gov.uk/content/binary/APPSI-response-to-GI-Consultation.pdf
I'm very pleased that our own original description of the address register situation as a "national scandal" [Copyright DUG] is now ubiquitous.
_____________________________________ March 4, 2010
Sharing customer data & insight with government – report now published by ONS
ONS commissioned Demographic Decision to carry out a project on “Information collected by commercial companies: what might be of value to ONS?”. This is to help them with both their current population updates, and also planning 2011 Census coverage.
I’m very grateful to several DUG members and other companies who have made time to discuss this. The report is now complete, and ONS have just made an abridged version public: http://www.ons.gov.uk/about-statistics/methodology-and-quality/imps/updates-reports/current-updates-reports/index.html
ONS are now considering the recommendations as part of their Improving Migration and Population Statistics programme, and their planning for the Census next March. I hope that this leads to further steps forward in sharing data and insight across the public / private divide.
_____________________________________
February 28, 2010
DUG's new LinkedIn networking site – please make the most of it to exchange news & views!
We've now set up a community communication / social network site, and I'm really grateful to Lee Madden of Barclays for getting us started.
Here's the link: http://www.linkedin.com/groups?about=&gid=2745168&trk=anet_ug_grppro
You'll see that "This Demographics User Group Network site is open to both members and non-members (commercial companies, public services, academic, etc.) to exchange news and ideas"
_______________________________________
February 20, 2010
As well as ONS's Consultations on 2011 Census Outputs, we now have Scotland's too
Following ONS's announcement of its Census Consultations for England and Wales (see earlier message below), we now have an announcement from GRO Scotland:
"This consultation is intended to provide the basis for developing, over the course of 2010, an agreed set of final specifications of the main outputs from the 2011 Census that meet the needs of a majority of users and make the best use of data collected. It also seeks views from users on a number of issues relating to census outputs including:
•pre-defined tabular outputs;
•comparisons of 2001 and 2011 census results;
•data delivery formats;
•outputs geography – including which intermediate geographies users need and what demand exists for workplace zone statistics; and
•timetable for release of 2011 Census results"
Read all about it at:
http://www.gro-scotland.gov.uk/census/censushm2011/preparations/consultation-and-research/formal-consultations/spring-10-consultation.html
The closing date is 14 May
______________________________________
February 7, 2010
2008 stats on Dwelling Stock by Council Tax Band - at Output Area level
Data for Neighbourhood Renewal has alerted me to the following new statistics now available on ONS's Neighbourhood Statistics website:
* Dwelling stock by council tax band (Updated for 2008) - down to Output Area (OA) level for England & Wales: http://www.data4nr.net/resources/53/
We now have datasets for each year since 2001, which in principle enables the tracking of changes in numbers of dwellings and their status (Bands A-H) for very small areas.
_______________________________________
February 1, 2010
Evening Standard reports DUG evidence at House of Commons Committee
Our evidence to the new House of Commons London Regional Committee for its inquiry into “London’s Population and the 2011 Census”, led to an invitation to give evidence at its first hearing today.
I'm pleased to report that we've got some excellent press coverage in tonight's Evening Standard:
______________________________________
January 22, 2010
Tim Berners-Lee unveils government data project - data.gov.uk
Pushing forward the Making Public Data Public initiative, the BBC reports the big news that:
"Web founder Sir Tim Berners-Lee has unveiled his latest venture for the UK government, which offers the public better access to official data.
A new website, data.gov.uk, will offer reams of public sector data, ranging from traffic statistics to crime figures, for private or commercial use......"
You can read the full report at: http://news.bbc.co.uk/1/hi/technology/8470797.stm
& see the new website at: http://www.data.gov.uk/
_______________________________________
January 18, 2010
DUG Training Course “Developing as an analyst” run for a public service organisation for the first time
This course, developed and presented for DUG by Stewart Robbins, Head of Customer Knowledge at E.ON, has been run for DUG member companies several times.
Friends in public sector organisations have also expressed interest, and on 18 January it was run for the Department for Transport. Here’s their feedback:
The delegates to this course, all of whom had a background in statistics, expected to end the day as more effective analysts. They wanted to learn how to identify customer requirements, and how to fulfil these in the most relevant way. The day lived up to their expectations:
“Lots of helpful advice in determining what stakeholders require”
“Highlighted the need to take time to think about a problem and then to think about a solution”
The delegates appreciated the trainer’s knowledge and the lucid and engaging way in which he delivered the course:
“Had good relevant experience, delivered material at good pace. Engaged well with participants”
“Good range of examples………..opportunities to ask questions”
“Kept course relevant and interesting”
Summing up: “Excellent course with lots of practical suggestions. Covered good range of topics in about the right level of detail”
_______________________________________
January 14, 2010
House of Commons London Regional Committee – London's Population inquiry – DUG's Evidence
We’ve been invited to provide a submission to another House of Commons Committee.
I’ve sent a note with the following key themes:
• The importance of good population statistics to commercial companies
• The need for several different measures of “the population”
• The currently crucial importance of the Census as a source
• The 2001 Census serving as an alarm bell to focus on coverage in 2011, leading to 3 priorities for improving coverage and quality:
- Create a definitive address register – for 2011, and beyond
- Target difficult areas for hand delivery, and collection, of Census forms
- Use administrative files for quality checking 2011
• Looking further ahead, more use should be made of administrative files for statistical purposes, aiming towards the creation of a population register, validated by a final traditional Census in 2016 (rather than waiting until 2021).
_______________________________________
January 12, 2010
OBiT's not-for-profit sharing of deceased names - 25 organisations now participating - might you join too?
Many of you will know Emma Reid, who was a very active member of DUG when she was at Saga, and who now has her own consultancy Greysells, which focuses on the over 50's.
Emma also initiated OBiT - the Stop Dead Consortium - to encourage organisations to pool their records of deceased people on a not-for-profit basis.
Its major benefits are the reducing the existing costs of cleaning lists, reducing the costs of wasted mailings, and preventing distress to the bereaved.
I'm very pleased to report that 25 organisations are now pooling their information - contributing to a file of 4.7 million deceased names. These organisations include DUG members Boots and M&S, but obviously Emma is keen to extend the list further. If you, or colleagues in another part of your company, are interested, do get in touch with her via http://greysells.co.uk/
_______________________________________
January 7, 2010
Boris Johnson launches London 'Datastore' with hundreds of sets of data
Guardian Technology has trailed this story:
"The Mayor of London, Boris Johnson, will on Thursday launch a website hosting hundreds of sets of data - including previously unreleased information - about the capital, as part of a new scheme intended to encourage people to create "mashups" of data to boost the city's transparency and accountability. Channel 4 will also be offering up to £200,000 through its 4ip fund to help develop the most innovative uses of the data."
Read more at: http://www.guardian.co.uk/technology/2010/jan/06/london-datastore-launch-johnson-mashups
The formal launch takes place at the end of this month, but the GLA is already seeking users' views: http://data.london.gov.uk/
It's good to see that Boris Johnson is as enthusiastic about "Making Public Data Public" as Gordon Brown (even if Lord Mandelson is reluctant).
PS. A postscript to my earlier message - you can now see Boris promoting the case for free data (but also struggling with Skype - it's a hoot):
http://www.youtube.com/watch?v=NjcZOefdmXE&feature=player_embedded#
_______________________________________
January 1, 2010
ONS's Census News - Latest Update, including Consultations on 2011 Outputs - Statistics and Geography
ONS has announced two consultations which are really important to us:
http://www.ons.gov.uk/census/2011-census/consultations/index.html
* Output Consultation: Main statistical outputs
This goes into considerable detail. It takes the 2001 tables as a starting point (which makes life simpler), and then highlights the impacts of changed questions (e.g. Type of Central Heating, and Ethnicity), and the opportunities created by the new questions - Second Residence & Language (& also National Identity, Citizenship, Month & Year of Entry, Intention to Stay, and Number of Bedrooms).
I shall start to review the particular priorities that we've argued for in earlier consultations (e.g. Workplace stats, OAC, Social Grade, Multivariate counts, etc.) but it will be really helpful to get your views on particular 2001 tables (or, failing that, topics) that have been valuable to you, including mainstream stuff on age, sex, tenure, NS-SEC, etc.
* Census Output Geography Consultation
This consultation is less demanding. ONS are already committed to keeping existing Output Areas wherever possible, and I'm assuming that we shall continue to support that vigorously. 3 issues are raised:
- A new geography for improved reporting of business statistics and statistics relating to the workplace [Sounds beneficial, as long as we continue to get simple workplace counts for all OAs, plus the few OAs with big working populations are split further?]
- An upper layer of Super Output Areas (USOAs) [Of no use to us?]
- A request for exceptional instances of current OAs and SOAs, as designed for 2001, that do not fit present criteria for statistical zones [No objection to a bit of tidying up?]
We have a reasonable amount of time (the closing date is 26 March), but it would be good to get your initial thoughts by the end of January, and then I'll produce a first draft.
These ONS consultations do, of course, apply to England & Wales. Scotland and N Ireland will be announcing their own consultations shortly.
_______________________________________
December 31, 2009
Letter in the Daily Telegraph - "Liberating Royal Mail data will generate tax revenues"
Following the FT's report "Mandy and Gordon - the unravelling" (over PAF), the Daily Telegraph also ran the story.
I wrote on our behalf, and the letter (somewhat altered) was published yesterday 29 December:
http://www.telegraph.co.uk/comment/letters/6901342/A-different-American-election-result-in-2000-would-have-changed-everything.html & scroll down to the letter.
I'll also send the original letter to Lord Mandelson's Department, to make sure that they realise the opposition to his view.
_______________________________________
December 29, 2009
Andrew Tasker - some very bad news
I'm very sorry to have to break the distressing news to you that Andrew Tasker died at the weekend.
Some of you will be aware that Andrew was diagnosed with cancer early in November, and, despite immediately starting a course of chemotherapy, chaired our strategy day on 18 November, and joined our awayday in Bristol on 3 December. I'm sure that we are all shocked to learn how swiftly he has died, and, for those of us lucky enough to know him well, to remember what a fine person he was.
-------------
Annette Dellevoet at Sainsbury's subsequently wrote the following appreciation of Andrew:
In June 1998 Andrew helped to found the Demographics User Group (DUG) along with Keith Dugmore and representatives from Whitbread and M&S. DUG lobbies the government on behalf of commercial users of government statistics as well as sharing methods and techniques and offering analytical training to its member companies. Andrew was an active and enthusiastic member of the group speaking at DUG conferences and running several training courses. It is a measure of his contribution that DUG retained him as an honorary member when he became a consultant. Despite his worsening illness Andrew was an animated contributor to the last DUG meeting on 3rd December. He will be greatly missed by all his DUG colleagues, who valued his wise advice, friendship and support, and have sent many messages of appreciation.
_______________________________________
December 27, 2009
New free datasets - House prices, Household Income estimates, Supermarket accessibility, & LA comparisons
Here's news of several new free datasets from government:
Dwelling prices by dwelling type
"This dataset provides the total number of changes of ownership of dwellings provided by the Land Registry including the type of dwelling and the dwelling price"
http://www.data4nr.net/resources/50/
Income: Model-Based Estimates
"New model-based estimates of average household income have been produced for England and Wales at Middle Layer Super Output Area (MSOA) level"
http://www.neighbourhood.statistics.gov.uk/dissemination/Info.do?page=news/newsitems/income-model-based-estimates.htm
Accessibility - Supermarkets
"The Department for Transport has published statistics on the Core Accessibility Indicators for 2008. The Indicators provide a number of measures of accessibility by public transport, walking, cycling and car to seven service types" - one of these is Supermarkets:
http://www.dft.gov.uk/pgr/statistics/datatablespublications/ltp/coreaccessindicators2008
and
http://www.data4nr.net/resources/836/
The Places Database
"Is now available offering downloads of spatial data from national to local level covering crime, retail floorspace, health, transport etc." The stats are at Local Authority level & above.
http://www.places.communities.gov.uk/
_______________________________________
December 24, 2009
"Mandy and Gordon - the unravelling" (over PAF), + Consultation on Ordnance Survey
Yesterday's Financial Times contained an article http://www.ft.com/cms/s/0/4afe99e0-ef44-11de-86c4-00144feab49a.html about Lord Mandelson resisting Gordon Brown's announcement about Making Public Data Public.
The article drags in PAF (rather than just postcodes & grid references), which hasn't been mentioned before.
The article also mentions the consultation on Ordnance Survey being delayed, but this morning I've just been alerted to: http://www.communities.gov.uk/publications/corporate/ordnancesurveyconsultation
Being faced with 91 pages from CLG makes your heart sink, but at least it's out before Xmas, rather than delayed, which would have increased the risk of the standard 3 month consultation period running into the election buffers.
_______________________________________
December 11, 2009
ONS's new Wealth and Assets Survey published
This is a new survey from ONS which I think may be of interest to many DUG members. Here are some headlines:
"Private household net wealth in Great Britain totalled £9 trillion in 2006/08, according to the Wealth in Great Britain report published today by the Office for National Statistics. The report presents results from the new Wealth and Assets survey, which found that wealth held in property and private pensions each account for 39 per cent of the total amount of private net wealth. Net financial wealth and physical goods, such as cars and antiques, each accounted for 11 per cent.
The report also shows how wealth was distributed across households. Median household net wealth was £204,500 in 2006/08. The least wealthy half of households accounted for only 9 per cent of wealth, while the wealthiest 20 per cent of households had 62 per cent of total wealth. The least wealthy 10 per cent of households had negative total net wealth.
Estimates of wealth varied across Great Britain. The wealthiest area was the South East of England with a median household wealth of £287,900. The area with the lowest median was Scotland with £150,600."
You'll find more details including the 174-page report at: http://www.statistics.gov.uk/StatBase/Product.asp?vlnk=15074
_______________________________________
December 9, 2009
Financial Times - "Data giveaway aims to bolster growth"
The Financial Times (8 December 2009) reports the latest exciting news of Making Public Data Public:
"Masses of mapping, weather and public transport data that the government currently sells are to be given away in a move that ministers hope will create scores of new businesses whose earnings will boost both the economy and taxes.
In a revolutionary initiative, a range of Ordnance Survey, Met Office, public transport and postcode data will be made available for commercial reuse. The step will cost the organisations involved many tens of millions of pounds in current income but John Denham, the communities secretary, said it could create billions of pounds worth of business over the years as entrepreneurs found new ways to deploy it........"
Read the full article at: http://www.ft.com/cms/s/0/cdcc60a2-e399-11de-9f4f-00144feab49a.html
_______________________________________
November 20, 2009
Public Administration Select Committee - Census Questions - & DUG mentioned on Radio 4
The House of Commons Public Administration Select Committee held its session on 2011 Census questions yesterday.
The MPs focused on Sexual Orientation, Migration & Religion - & of course we witnesses raised Income.
The hearing can be viewed on the web at: http://www.parliament.uk/parliamentary_committees/public_administration_select_committee.cfm
& the transcript should be published in a couple of weeks too.
Radio 4's Today programme picked up the story this morning (c.6.45) & included a clip DUG's views on a question on sexual orientation ("well down our list of priorities").#
#. This exchange started:
Q: What do you think about sex?
A: It's OK (laughter)
_______________________________________
November 20, 2009
Making Public Data Public - more news
This news, which is potentially of great significance, is moving fast.
Here's the latest instalment from the Guardian's Free Our Data Campaign, which starts:
"The Free Our Data campaign has scored a major victory, with the announcement by the government that it intends to make Ordnance Survey maps free for use online by any organisation – including commercial ones – at resolutions more detailed than commercial 1:25,000 Landranger maps from April next year.
The announcement of the opening of a consultation on the plan by Gordon Brown at Downing Street on Tuesday, as part of a seminar on making public data public – set in the wider context of public service reform, under the "Smarter Government" umbrella – indicates that the ideas underpinning the campaign have now been taken on board at the highest levels of government........"
Read more at: http://www.guardian.co.uk/technology/2009/nov/19/ordnance-survey-maps-free-online
_______________________________________
November 17, 2009
Breaking news: Better access to Ordnance Survey data
Here's what looks like significant news from the Advisory Panel for Public Sector Information (APPSI). It's picked up a CLG press release which starts:
“The Prime Minister and Communities Secretary John Denham will today announce that the public will have more access to Ordnance Survey maps from next year, as part of a Government drive to open up data to improve transparency. Speaking at a seminar on Smarter Government in Downing Street later today, attended by Sir Tim Berners-Lee and Professor Nigel Shadbolt, the Prime Minister will set out how the Government and Ordnance Survey, Great Britain’s national mapping agency, will open up its data relating to electoral and local authority boundaries, postcode areas and mid scale mapping information”………
We’ll no doubt learn more detail - and whether the OS dam has finally cracked - in tomorrow’s press. (The Guardian tried to contact me this afternoon).
_______________________________________
November 13, 2009
Department for Transport - National Public Transport Gazetteer
Following my message in September about public transport access points (see below), the Department for Transport has also announced its National Public Transport Gazetteer: " A topographic database of towns and settlements in the UK providing a common frame of reference for UK Public Transport Information schemas. The appropriate naming of towns and places is vital for providing effective place and stop finding in all modern on-line journey planners and other Passenger Information systems".
Detail can be found at: http://www.nptg.org.uk/
_____________________________________
November 12, 2009
Conservatives seek to cut 2011 Census questions; + House of Commons Committee hearing
Here's some potentially serious news about the 2011 Census.
Nick Hurd MP, the Shadow Cabinet Office minister has written to the head of the UK Statistics Authority (Sir Michael Scholar) saying "We do not believe that the 2011 Census can have the size and scope that has been proposed, particularly since we believe that many of the proposed questions are unsuitable........and urge you to scale back its cost and scope in order to save money and reduce its intrusive impact on citizens".
The Public Administration Select Committee is now to hold a session next Thursday 19 November on the 2011 census questions. Fortunately, I've been asked to give evidence at the hearing, along with Prof. Phil Rees (Leeds University) & a representative of the Equality and Human Rights Commission.
_______________________________________
November 6, 2009
Appointment of new head of ONS - Stephen Penneck
I'm delighted to report that Stephen Penneck - winner of the DUG Award this year - has now been appointed as the new boss of ONS.
_______________________________________
July 29, 2010
"Ordnance Survey Leading The Way On Government Transparency"
The Department for Communities and Local Government has issued a Press Release which starts:
“The national mapping agency is the pioneer in a new era of information transparency by Government, Communities Minister Baroness Hanham said last week.
Three months after launching the OpenData initiative, Baroness Hanham said Ordnance Survey is leading the race to make public data freely available and accessible........”
You can read it all at: http://www.egovmonitor.com/node/37518
Given OS’s bitter resistance over the last decade to opening up free data, this spin is truly boggling. George Orwell would have been proud.
But, better late than never, and it’s good to see the new Coalition strongly committed to making public data freely available.
_____________________________________
July 29, 2010
Job vacancies – Tesco & Boots
For details, see the tab to the left.
_____________________________________
July 27, 2010
2011 Census to proceed as planned – Ministerial Statement
The UK Statistics Authority has just issued a brief Press Release:
“The UK Statistics Authority welcomes the Government’s decision to proceed with the existing plans for the 2011 Census in England and Wales, set out in an answer by the Minister for the Cabinet Office, Rt. Hon. Francis Maude MP, to a Written Parliamentary Question.
The Office for National Statistics’ preparations for the Census on 27 March 2011 will now press ahead with all speed.
The Statistics Authority is determined that, with the full support of the Government and all the other parties concerned, the 2011 Census will be the success that the country needs it to be, and will provide the information about our population which can only be derived from the full-scale Census, which has now been confirmed.”
http://www.statisticsauthority.gov.uk/news/2011-england-and-wales-census.html
So it appears that Francis Maude – despite previous hostility – has now been convinced. Phew.
_____________________________________
July 22, 2010
OAC conference – PM, Monday September 6th, 2010 – free
Here’s news of this year’s OAC (Output Area Classification) conference. The programme is pasted below. The emphasis is on use in the public sector, but it includes sessions on the British Population Survey (an alternative to the TGI), and Open Data, Free Tools, and there will no doubt bee wider discussion & ideas. There is more information on the OAC User Group website: http://areaclassification.org.uk/
You can book (no charge) at: http://oac.eventbrite.com
_____________________________________
July 12, 2010
"National Census to be axed after 200 years" – Daily Telegraph – DUG's letter published
Our letter has been published in today's Daily Telegraph:
"Counting the population
SIR – The Government has decided that next year’s Census will be the last (report, July 10). Hitherto, censuses have provided detailed counts of the population that are vital for both public and private organisations when deciding where to target investment, and for the public, to inform democracy. Census returns released after 100 years are also of interest to family researchers.
Francis Maude, the Cabinet Office minister, is right to examine different and potentially cheaper ways to count the population. In particular, the Government’s own extensive databases would be much the best starting point, learning from the experiences of several Nordic countries.
However, to announce a decision to scrap future censuses before proving that alternative sources are adequate is reckless.
Keith Dugmore
London SW1"
& can be found on their website at:
http://www.telegraph.co.uk/comment/letters/7884088/Reducing-our-Armed-Forces-will-leave-Britain-dangerously-reliant-on-our-allies.html
_____________________________________
July 3, 2010
UN / Eurostat Census meeting – papers on Censuses overseas
For those of you interested in Census information for countries outside the UK, here's news of the "Joint UNECE/Eurostat Group of Experts on Population and Housing Censuses", which takes place in Geneva, 7-9 July:
http://www.unece.org/stats/documents/2010.07.census.htm
You'll see papers from many countries, including one from me on "Disseminating Census information to maximise use and value", which highlights the need for a better balance between statistical disclosure risk, and utility, guided by Jeremy Bentham’s view: “It is the greatest good to the greatest number of people which is the measure of right and wrong.”
_____________________________________
June 21, 2010
British Population Survey – coded with geodemographics, including OAC
Here’s news of the British Population Survey (a commercial survey, a bit like the Target Group Index), which is now coded with OAC and other geodem classifications.
http://www.thebps.co.uk/#/segmentation/4540464055
You’ll see the word FREE (I understand that only the most recent year is charged).
Here’s news of the British Population Survey (a commercial survey, a bit like the Target Group Index), which is now coded with OAC and other geodem classifications.
_____________________________________
June 19, 2010
News of an open license for a NSPD postcode lookup file
Here's news from the OAC User Group:
"An open license for the non PAF version of the National Statistics Postcode Directory postcode lookup file is now available. The terms of which enable a user to:
• Copy, distribute and transmit the Data
• Adapt the Data
• Exploit the Data commercially whether by sub-licensing it, combining it with other data or by including it in your own product or application
This is great news as it includes OAC as one of the variables and makes address lookups a much simpler process. We shall update the tutorial guides on the website soon to reflect this change"
The ONS website goes into more detail, starting:
"The NSPD is released quarterly in February, May, August and November and NSPD products are supplied with various elements of Postcode Address File (PAF) data depending on the level required. These extracts are available as Full PAF or Part PAF options. Additionally, an ‘Open’ version, containing no PAF elements (and not attracting Royal Mail royalties) is also available......."
See more at: http://www.ons.gov.uk/about-statistics/geography/products/geog-products-postcode/nspd/index.html
_____________________________________
June 19, 2010
More free data: a) Transport for London; b) Postcode grid references & lat / long
APPSI have just alerted me to more free data:
1) Transport for London - as reported in The Guardian:
"Transport for London has opened up huge swathes of its data for free re-use by developers keen to see what they can do - and build - with it. After years of lobbying by developers who have wanted to be able to exploit the data - which, being publicly funded, logically belongs to the public too - TfL today acknowledged its willingness to listen to what people want to do as the London Data Store, headed by Emer Coleman, announced the datasets that will be provided.........."
Read all about it at: http://www.guardian.co.uk/technology/blog/2010/jun/15/london-datastore-tfl-data-free
2) Geopostcode:
"An API for retrieving geographic information (grid reference, latitude and longitude) relating to postcodes. Data is available as XML or JSON" http://data.gov.uk/apps/geopostcode
_____________________________________
June 10, 2010
UKSA interim report "Strengthening User Engagement" – DUG's response
Here is DUG’s response to the UKSA consultation. It makes the case for going further than just better user dialogue, with UKSA's monitoring role being more proactive, the government statistical service becoming more customer focussed - and adding a proposal to rename the Government Statistical Service the National Statistical Service – to reinforce the need to serve all customers, not primarily those in central government.
_____________________________________
June 8, 2010
Independent newspaper – The real cost of a compromised census will be inaccurate data – DUG's views quoted
ONS has just alerted me to an article in last Saturday's Independent:
"With less than a year to go to the 2011 Census, is Francis Maude about to change the rules? In opposition he was strongly critical of the Census, describing it as burdensome, intrusive and poor value for money. Now he's in charge, as Minister for the Cabinet Office. He's also central to the programme of Government cuts that the coalition has promised. So demographers and statisticians are worried that he may make good on his promise to "scale back" the Census even at this late stage........
.....if he doesn't make any changes when he's now in a position to, he's going to have to eat a lot of words. He's already had a warning from the Demographics User Group, a private-sector body that represents heavy-hitters such as John Lewis, Marks & Spencer, Sainsbury's, Nationwide and Alliance Boots, saying that they regard the Census as of "fundamental importance" when making decisions about opening new stores, the products to be stocked, and the customers to be targeted. A late decision to cut the number of questions would disrupt existing plans and contracts, save little money – or even add to the cost – and result in reduced and inconsistent information between England and Wales, Scotland and Northern Ireland, says Keith Dugmore, director of the group........."
You can see the full article at:
http://www.independent.co.uk/opinion/commentators/nigel-hawkes-the-real-cost-of-a-compromised-census-will-be-inaccurate-data-1992013.html
_____________________________________
June 8, 2010
Big role for supermarkets in regenerating poor communities – report by Demos
Yesterday's Guardian reported: "Demos sees big role for supermarkets in regenerating poor communities" & argues for tax breaks for supermarkets if they can turn sink areas around: http://www.guardian.co.uk/uk/2010/jun/07/demos-supermarkets-poverty-regeneration-pride
& here's the link to the Demos website, where you can download the full report Civic Streets: http://www.demos.co.uk/publications/civicstreets
_____________________________________
June 5, 2010
Victory for campaigners as coalition releases data deluge
Today's papers have further news of the new government releasing more data into the public domain.
The Guardian's article http://www.guardian.co.uk/politics/2010/jun/04/government-spending-database-special-report focuses on COINS (Combined Online Information System) and information on government spending, whilst Charles Arthur reports "A flood of data is on its way ... but we will need to make sense of it"
http://www.guardian.co.uk/politics/2010/jun/04/coins-treasury-public-sector-data
Much of this won't be immediately relevant to us, but it does confirm the new government's general policy that government information should be in the public domain.
_____________________________________
May 26, 2010
Experian partners with The Local Data Company
Here’s some news from Experian about The Local Data Company (who presented to our meeting last September, and are used by several DUG member companies):
http://www.localdatacompany.com/press-releases/2010/5/26/experian-partners-with-the-local-data-company.html
You’ll see that it’s referred to as partnering, rather than continuing Experian’s recent history of buying small companies.
_____________________________________
May 20, 2010
Database valuation
Several members have been interested for some time in how a value might be put on their customer databases. At DUG’s most recent meeting Mark Perrett (E.ON) mentioned his recent discussions with Call Credit, who are researching the topic.
Mark has also unearthed two earlier documents from the Institute of Direct Marketing:
http://www.theidm.com/marketing-events/calendar/idm-data-council-valuing-your-database/
* White paper - Database Valuation: Putting a Price on Your Prime Asset.
* A case study in the valuation of a database prepared by Valuation Consulting.
_____________________________________
May 18, 2010
Ordnance Survey free "opendata" – more vector mapping released – 1:25,000 scale
Following up the news of 10 April, OS has now released another map dataset: "OS VectorMap™ District". It's nominally at 1:25,000 scale, and in both vector & raster formats.
Details are given at: http://www.ordnancesurvey.co.uk/oswebsite/products/vectormap/district/index.html
_____________________________________
May 18, 2010
Letter to the Rt. Hon. Francis Maude MP: The 2011 Census – its importance to commercial companies
The new Cabinet Office Minister Francis Maude has in recent months actively made the case to reduce the scope & cost of the Census (& one of the options now being mentioned is to scrap it completely). We have therefore sent him this letter to get the commercial users’ case for the Census in front of the new government without delay.
_____________________________________
May 6, 2010
2011 Census Output Consultation – Scotland – DUG’s Response
See our response to GRO Scotland here
_____________________________________
April 10, 2010
Ordnance Survey free "opendata": what does it actually include?
Following up my message of 3 April, here's the Guardian's Data Blog answer to the above question:
http://www.guardian.co.uk/news/datablog/2010/apr/02/ordnance-survey-open-data
If you have any obs about this treasure trove - maps, boundaries, postcode grid references, gazetteers - or, indeed, a wish-list of more government data you'd like to push to be made freely available, do get back to me.
_____________________________________
April 4, 2010
Job Vacancies – advertisements by DUG members
Several members are seeking staff – see the “Jobs” tab on the left.
_____________________________________
April 3, 2010
Major Government announcement – Ordnance Survey – significant datasets now FREE
Following the consultation on Ordnance Survey the Government has published its response:
http://www.communities.gov.uk/documents/corporate/pdf/1528263.pdf
The Guardian’s Free Our Data report http://www.guardian.co.uk/technology/2010/apr/01/ordnance-survey-maps-download-free starts off:
“It is the biggest shakeup in the Ordnance Survey's 260-year history: you can download maps as detailed as 1:10,000 scale, or collect a list of locations appearing on maps at the 1:50,000 scale, or a conversion system for postcodes to grid references – free for personal or commercial use.”
_____________________________________
April 1, 2010
London Regional Committee's first report, London's population and the 2011 Census
Here's the report http://www.publications.parliament.uk/pa/cm200910/cmselect/cmlonreg/349/34902.htm from the House of Commons committee to which we gave evidence. I'm pleased to see that DUG gets quoted.
You'll see strong support for the Census, but also the predictable concern about the dangers of a low 2011 Census response in London - and the need to target effort.
Also: "The Committee welcomes the creation of a national address register for the Census, which will greatly improve accurate delivery of census forms, but say it is 'barely credible' that this valuable resource is not to be maintained after 2011. The report recommends that the Government acts urgently to resolve the intellectual property issues which are preventing maintenance of the register past 2011."
_____________________________________
March 31, 2010
Orange joins DUG
Orange is the latest company to join the Demographics User Group (DUG), whose membership already includes major companies across a range of sectors.
Orange is one of the largest mobile operators in the UK, with a current consumer base of 17million. It has a wealth of internal data in its analytical environment, from point of sale through to individual call record - who, when, where, and how often a customer calls.
Orange’s Strategic Insight's team's main responsibilities are to provide: Customer segmentation; a behavioural view to increase value, loyalty & enhance the customer experience; bespoke deep dive analytics from a customer led approach for the whole of Orange.
Anthony Mawby, Head of Strategic Insight, stated "We are very pleased to be joining DUG, and are looking forward to sharing experiences of data, analysis, and insight with other major companies from a range of different sectors including retail, financial services, and energy."
_____________________________________
March 27, 2010
Department of Transport's 350,000 public transport access points "NaPTAN" – available free
Following my message back in September (see below), and the DfT then saying that commercial users have to pay, I've now checked about the data being part of the (free) www.data.gov.uk initiative. Here's the answer that I've received from the DfT:
"Thanks for your email. I had a quick look at the data.gov.uk website and snapshot bus stop location data is available to download at: http://data.gov.uk/node/8473
I've clarified the use of this data with colleagues at Transport Direct, and the snapshot data is freely available to all for use. However, this is snapshot data and so will already be out of date, as changes are made to the bus stop data almost on a daily basis. Therefore, if commercial users wanted to access more up-to-date information, then they would need to request a license and follow the procedures as set out on the NaPTAN website: www.dft.gov.uk/naptan .
I understand that updates will be made to the data held on the data.gov.uk website, but am not sure of the frequency of these."
_____________________________________
March 24, 2010
Prime Minister's speech: "Building Britain's Digital Future" (more news of free data, inc. Ordnance Survey)
See: http://www.number10.gov.uk/Page22897
_____________________________________
March 13, 2010
Google Street View - now extended to UK coverage
Google's Street View has now increased coverage for the whole UK: http://maps.google.co.uk/
To quote a geographer at UCL: "It is pretty amazing – just drag the little yellow man onto the map of the UK.....".
_____________________________________
March 13, 2010
Consultation on Ordnance Survey – DUG's response, + extra comment
Here's DUG’s response to the Department for Communities’ consultation on OS.
APPSI has also made a heavy duty response (led by David Rhind): http://www.appsi.gov.uk/content/binary/APPSI-response-to-GI-Consultation.pdf
I'm very pleased that our own original description of the address register situation as a "national scandal" [Copyright DUG] is now ubiquitous.
March 4, 2010
Sharing customer data & insight with government – report now published by ONS
ONS commissioned Demographic Decision to carry out a project on “Information collected by commercial companies: what might be of value to ONS?”. This is to help them with both their current population updates, and also planning 2011 Census coverage.
I’m very grateful to several DUG members and other companies who have made time to discuss this. The report is now complete, and ONS have just made an abridged version public: http://www.ons.gov.uk/about-statistics/methodology-and-quality/imps/updates-reports/current-updates-reports/index.html
ONS are now considering the recommendations as part of their Improving Migration and Population Statistics programme, and their planning for the Census next March. I hope that this leads to further steps forward in sharing data and insight across the public / private divide.
_____________________________________
February 28, 2010
DUG's new LinkedIn networking site – please make the most of it to exchange news & views!
We've now set up a community communication / social network site, and I'm really grateful to Lee Madden of Barclays for getting us started.
Here's the link: http://www.linkedin.com/groups?about=&gid=2745168&trk=anet_ug_grppro
You'll see that "This Demographics User Group Network site is open to both members and non-members (commercial companies, public services, academic, etc.) to exchange news and ideas"
_______________________________________
February 20, 2010
As well as ONS's Consultations on 2011 Census Outputs, we now have Scotland's too
Following ONS's announcement of its Census Consultations for England and Wales (see earlier message below), we now have an announcement from GRO Scotland:
"This consultation is intended to provide the basis for developing, over the course of 2010, an agreed set of final specifications of the main outputs from the 2011 Census that meet the needs of a majority of users and make the best use of data collected. It also seeks views from users on a number of issues relating to census outputs including:
•pre-defined tabular outputs;
•comparisons of 2001 and 2011 census results;
•data delivery formats;
•outputs geography – including which intermediate geographies users need and what demand exists for workplace zone statistics; and
•timetable for release of 2011 Census results"
Read all about it at:
http://www.gro-scotland.gov.uk/census/censushm2011/preparations/consultation-and-research/formal-consultations/spring-10-consultation.html
The closing date is 14 May
______________________________________
February 7, 2010
2008 stats on Dwelling Stock by Council Tax Band - at Output Area level
Data for Neighbourhood Renewal has alerted me to the following new statistics now available on ONS's Neighbourhood Statistics website:
* Dwelling stock by council tax band (Updated for 2008) - down to Output Area (OA) level for England & Wales: http://www.data4nr.net/resources/53/
We now have datasets for each year since 2001, which in principle enables the tracking of changes in numbers of dwellings and their status (Bands A-H) for very small areas.
_______________________________________
February 1, 2010
Evening Standard reports DUG evidence at House of Commons Committee
Our evidence to the new House of Commons London Regional Committee for its inquiry into “London’s Population and the 2011 Census”, led to an invitation to give evidence at its first hearing today.
I'm pleased to report that we've got some excellent press coverage in tonight's Evening Standard:
______________________________________
January 22, 2010
Tim Berners-Lee unveils government data project - data.gov.uk
Pushing forward the Making Public Data Public initiative, the BBC reports the big news that:
"Web founder Sir Tim Berners-Lee has unveiled his latest venture for the UK government, which offers the public better access to official data.
A new website, data.gov.uk, will offer reams of public sector data, ranging from traffic statistics to crime figures, for private or commercial use......"
You can read the full report at: http://news.bbc.co.uk/1/hi/technology/8470797.stm
& see the new website at: http://www.data.gov.uk/
_______________________________________
January 18, 2010
DUG Training Course “Developing as an analyst” run for a public service organisation for the first time
This course, developed and presented for DUG by Stewart Robbins, Head of Customer Knowledge at E.ON, has been run for DUG member companies several times.
Friends in public sector organisations have also expressed interest, and on 18 January it was run for the Department for Transport. Here’s their feedback:
The delegates to this course, all of whom had a background in statistics, expected to end the day as more effective analysts. They wanted to learn how to identify customer requirements, and how to fulfil these in the most relevant way. The day lived up to their expectations:
“Lots of helpful advice in determining what stakeholders require”
“Highlighted the need to take time to think about a problem and then to think about a solution”
The delegates appreciated the trainer’s knowledge and the lucid and engaging way in which he delivered the course:
“Had good relevant experience, delivered material at good pace. Engaged well with participants”
“Good range of examples………..opportunities to ask questions”
“Kept course relevant and interesting”
Summing up: “Excellent course with lots of practical suggestions. Covered good range of topics in about the right level of detail”
This course, developed and presented for DUG by Stewart Robbins, Head of Customer Knowledge at E.ON, has been run for DUG member companies several times.
Friends in public sector organisations have also expressed interest, and on 18 January it was run for the Department for Transport. Here’s their feedback:
The delegates to this course, all of whom had a background in statistics, expected to end the day as more effective analysts. They wanted to learn how to identify customer requirements, and how to fulfil these in the most relevant way. The day lived up to their expectations:
“Lots of helpful advice in determining what stakeholders require”
“Highlighted the need to take time to think about a problem and then to think about a solution”
The delegates appreciated the trainer’s knowledge and the lucid and engaging way in which he delivered the course:
“Had good relevant experience, delivered material at good pace. Engaged well with participants”
“Good range of examples………..opportunities to ask questions”
“Kept course relevant and interesting”
Summing up: “Excellent course with lots of practical suggestions. Covered good range of topics in about the right level of detail”
_______________________________________
January 14, 2010
House of Commons London Regional Committee – London's Population inquiry – DUG's Evidence
We’ve been invited to provide a submission to another House of Commons Committee.
I’ve sent a note with the following key themes:
• The importance of good population statistics to commercial companies
• The need for several different measures of “the population”
• The currently crucial importance of the Census as a source
• The 2001 Census serving as an alarm bell to focus on coverage in 2011, leading to 3 priorities for improving coverage and quality:
- Create a definitive address register – for 2011, and beyond
- Target difficult areas for hand delivery, and collection, of Census forms
- Use administrative files for quality checking 2011
• Looking further ahead, more use should be made of administrative files for statistical purposes, aiming towards the creation of a population register, validated by a final traditional Census in 2016 (rather than waiting until 2021).
_______________________________________
January 12, 2010
OBiT's not-for-profit sharing of deceased names - 25 organisations now participating - might you join too?
Many of you will know Emma Reid, who was a very active member of DUG when she was at Saga, and who now has her own consultancy Greysells, which focuses on the over 50's.
Emma also initiated OBiT - the Stop Dead Consortium - to encourage organisations to pool their records of deceased people on a not-for-profit basis.
Its major benefits are the reducing the existing costs of cleaning lists, reducing the costs of wasted mailings, and preventing distress to the bereaved.
I'm very pleased to report that 25 organisations are now pooling their information - contributing to a file of 4.7 million deceased names. These organisations include DUG members Boots and M&S, but obviously Emma is keen to extend the list further. If you, or colleagues in another part of your company, are interested, do get in touch with her via http://greysells.co.uk/
_______________________________________
January 7, 2010
Boris Johnson launches London 'Datastore' with hundreds of sets of data
Guardian Technology has trailed this story:
"The Mayor of London, Boris Johnson, will on Thursday launch a website hosting hundreds of sets of data - including previously unreleased information - about the capital, as part of a new scheme intended to encourage people to create "mashups" of data to boost the city's transparency and accountability. Channel 4 will also be offering up to £200,000 through its 4ip fund to help develop the most innovative uses of the data."
Read more at: http://www.guardian.co.uk/technology/2010/jan/06/london-datastore-launch-johnson-mashups
The formal launch takes place at the end of this month, but the GLA is already seeking users' views: http://data.london.gov.uk/
It's good to see that Boris Johnson is as enthusiastic about "Making Public Data Public" as Gordon Brown (even if Lord Mandelson is reluctant).
PS. A postscript to my earlier message - you can now see Boris promoting the case for free data (but also struggling with Skype - it's a hoot):
http://www.youtube.com/watch?v=NjcZOefdmXE&feature=player_embedded#
_______________________________________
January 1, 2010
ONS's Census News - Latest Update, including Consultations on 2011 Outputs - Statistics and Geography
ONS has announced two consultations which are really important to us:
http://www.ons.gov.uk/census/2011-census/consultations/index.html
* Output Consultation: Main statistical outputs
This goes into considerable detail. It takes the 2001 tables as a starting point (which makes life simpler), and then highlights the impacts of changed questions (e.g. Type of Central Heating, and Ethnicity), and the opportunities created by the new questions - Second Residence & Language (& also National Identity, Citizenship, Month & Year of Entry, Intention to Stay, and Number of Bedrooms).
I shall start to review the particular priorities that we've argued for in earlier consultations (e.g. Workplace stats, OAC, Social Grade, Multivariate counts, etc.) but it will be really helpful to get your views on particular 2001 tables (or, failing that, topics) that have been valuable to you, including mainstream stuff on age, sex, tenure, NS-SEC, etc.
* Census Output Geography Consultation
This consultation is less demanding. ONS are already committed to keeping existing Output Areas wherever possible, and I'm assuming that we shall continue to support that vigorously. 3 issues are raised:
- A new geography for improved reporting of business statistics and statistics relating to the workplace [Sounds beneficial, as long as we continue to get simple workplace counts for all OAs, plus the few OAs with big working populations are split further?]
- An upper layer of Super Output Areas (USOAs) [Of no use to us?]
- A request for exceptional instances of current OAs and SOAs, as designed for 2001, that do not fit present criteria for statistical zones [No objection to a bit of tidying up?]
We have a reasonable amount of time (the closing date is 26 March), but it would be good to get your initial thoughts by the end of January, and then I'll produce a first draft.
These ONS consultations do, of course, apply to England & Wales. Scotland and N Ireland will be announcing their own consultations shortly.
_______________________________________
December 31, 2009
Letter in the Daily Telegraph - "Liberating Royal Mail data will generate tax revenues"
Following the FT's report "Mandy and Gordon - the unravelling" (over PAF), the Daily Telegraph also ran the story.
I wrote on our behalf, and the letter (somewhat altered) was published yesterday 29 December:
http://www.telegraph.co.uk/comment/letters/6901342/A-different-American-election-result-in-2000-would-have-changed-everything.html & scroll down to the letter.
I'll also send the original letter to Lord Mandelson's Department, to make sure that they realise the opposition to his view.
_______________________________________
December 29, 2009
Andrew Tasker - some very bad news
I'm very sorry to have to break the distressing news to you that Andrew Tasker died at the weekend.
Some of you will be aware that Andrew was diagnosed with cancer early in November, and, despite immediately starting a course of chemotherapy, chaired our strategy day on 18 November, and joined our awayday in Bristol on 3 December. I'm sure that we are all shocked to learn how swiftly he has died, and, for those of us lucky enough to know him well, to remember what a fine person he was.
-------------
Annette Dellevoet at Sainsbury's subsequently wrote the following appreciation of Andrew:
In June 1998 Andrew helped to found the Demographics User Group (DUG) along with Keith Dugmore and representatives from Whitbread and M&S. DUG lobbies the government on behalf of commercial users of government statistics as well as sharing methods and techniques and offering analytical training to its member companies. Andrew was an active and enthusiastic member of the group speaking at DUG conferences and running several training courses. It is a measure of his contribution that DUG retained him as an honorary member when he became a consultant. Despite his worsening illness Andrew was an animated contributor to the last DUG meeting on 3rd December. He will be greatly missed by all his DUG colleagues, who valued his wise advice, friendship and support, and have sent many messages of appreciation.
_______________________________________
December 27, 2009
New free datasets - House prices, Household Income estimates, Supermarket accessibility, & LA comparisons
Here's news of several new free datasets from government:
Dwelling prices by dwelling type
"This dataset provides the total number of changes of ownership of dwellings provided by the Land Registry including the type of dwelling and the dwelling price"
http://www.data4nr.net/resources/50/
Income: Model-Based Estimates
"New model-based estimates of average household income have been produced for England and Wales at Middle Layer Super Output Area (MSOA) level"
http://www.neighbourhood.statistics.gov.uk/dissemination/Info.do?page=news/newsitems/income-model-based-estimates.htm
Accessibility - Supermarkets
"The Department for Transport has published statistics on the Core Accessibility Indicators for 2008. The Indicators provide a number of measures of accessibility by public transport, walking, cycling and car to seven service types" - one of these is Supermarkets:
http://www.dft.gov.uk/pgr/statistics/datatablespublications/ltp/coreaccessindicators2008
and
http://www.data4nr.net/resources/836/
The Places Database
"Is now available offering downloads of spatial data from national to local level covering crime, retail floorspace, health, transport etc." The stats are at Local Authority level & above.
http://www.places.communities.gov.uk/
_______________________________________
December 24, 2009
"Mandy and Gordon - the unravelling" (over PAF), + Consultation on Ordnance Survey
Yesterday's Financial Times contained an article http://www.ft.com/cms/s/0/4afe99e0-ef44-11de-86c4-00144feab49a.html about Lord Mandelson resisting Gordon Brown's announcement about Making Public Data Public.
The article drags in PAF (rather than just postcodes & grid references), which hasn't been mentioned before.
The article also mentions the consultation on Ordnance Survey being delayed, but this morning I've just been alerted to: http://www.communities.gov.uk/publications/corporate/ordnancesurveyconsultation
Being faced with 91 pages from CLG makes your heart sink, but at least it's out before Xmas, rather than delayed, which would have increased the risk of the standard 3 month consultation period running into the election buffers.
_______________________________________
December 11, 2009
ONS's new Wealth and Assets Survey published
This is a new survey from ONS which I think may be of interest to many DUG members. Here are some headlines:
"Private household net wealth in Great Britain totalled £9 trillion in 2006/08, according to the Wealth in Great Britain report published today by the Office for National Statistics. The report presents results from the new Wealth and Assets survey, which found that wealth held in property and private pensions each account for 39 per cent of the total amount of private net wealth. Net financial wealth and physical goods, such as cars and antiques, each accounted for 11 per cent.
The report also shows how wealth was distributed across households. Median household net wealth was £204,500 in 2006/08. The least wealthy half of households accounted for only 9 per cent of wealth, while the wealthiest 20 per cent of households had 62 per cent of total wealth. The least wealthy 10 per cent of households had negative total net wealth.
Estimates of wealth varied across Great Britain. The wealthiest area was the South East of England with a median household wealth of £287,900. The area with the lowest median was Scotland with £150,600."
You'll find more details including the 174-page report at: http://www.statistics.gov.uk/StatBase/Product.asp?vlnk=15074
_______________________________________
December 9, 2009
Financial Times - "Data giveaway aims to bolster growth"
The Financial Times (8 December 2009) reports the latest exciting news of Making Public Data Public:
"Masses of mapping, weather and public transport data that the government currently sells are to be given away in a move that ministers hope will create scores of new businesses whose earnings will boost both the economy and taxes.
In a revolutionary initiative, a range of Ordnance Survey, Met Office, public transport and postcode data will be made available for commercial reuse. The step will cost the organisations involved many tens of millions of pounds in current income but John Denham, the communities secretary, said it could create billions of pounds worth of business over the years as entrepreneurs found new ways to deploy it........"
Read the full article at: http://www.ft.com/cms/s/0/cdcc60a2-e399-11de-9f4f-00144feab49a.html
_______________________________________
November 20, 2009
Public Administration Select Committee - Census Questions - & DUG mentioned on Radio 4
The House of Commons Public Administration Select Committee held its session on 2011 Census questions yesterday.
The MPs focused on Sexual Orientation, Migration & Religion - & of course we witnesses raised Income.
The hearing can be viewed on the web at: http://www.parliament.uk/parliamentary_committees/public_administration_select_committee.cfm
& the transcript should be published in a couple of weeks too.
Radio 4's Today programme picked up the story this morning (c.6.45) & included a clip DUG's views on a question on sexual orientation ("well down our list of priorities").#
#. This exchange started:
Q: What do you think about sex?
A: It's OK (laughter)
_______________________________________
November 20, 2009
Making Public Data Public - more news
This news, which is potentially of great significance, is moving fast.
Here's the latest instalment from the Guardian's Free Our Data Campaign, which starts:
"The Free Our Data campaign has scored a major victory, with the announcement by the government that it intends to make Ordnance Survey maps free for use online by any organisation – including commercial ones – at resolutions more detailed than commercial 1:25,000 Landranger maps from April next year.
The announcement of the opening of a consultation on the plan by Gordon Brown at Downing Street on Tuesday, as part of a seminar on making public data public – set in the wider context of public service reform, under the "Smarter Government" umbrella – indicates that the ideas underpinning the campaign have now been taken on board at the highest levels of government........"
Read more at: http://www.guardian.co.uk/technology/2009/nov/19/ordnance-survey-maps-free-online
_______________________________________
November 17, 2009
Breaking news: Better access to Ordnance Survey data
Here's what looks like significant news from the Advisory Panel for Public Sector Information (APPSI). It's picked up a CLG press release which starts:
“The Prime Minister and Communities Secretary John Denham will today announce that the public will have more access to Ordnance Survey maps from next year, as part of a Government drive to open up data to improve transparency. Speaking at a seminar on Smarter Government in Downing Street later today, attended by Sir Tim Berners-Lee and Professor Nigel Shadbolt, the Prime Minister will set out how the Government and Ordnance Survey, Great Britain’s national mapping agency, will open up its data relating to electoral and local authority boundaries, postcode areas and mid scale mapping information”………
We’ll no doubt learn more detail - and whether the OS dam has finally cracked - in tomorrow’s press. (The Guardian tried to contact me this afternoon).
_______________________________________
November 13, 2009
Department for Transport - National Public Transport Gazetteer
Following my message in September about public transport access points (see below), the Department for Transport has also announced its National Public Transport Gazetteer: " A topographic database of towns and settlements in the UK providing a common frame of reference for UK Public Transport Information schemas. The appropriate naming of towns and places is vital for providing effective place and stop finding in all modern on-line journey planners and other Passenger Information systems".
Detail can be found at: http://www.nptg.org.uk/
November 12, 2009
Conservatives seek to cut 2011 Census questions; + House of Commons Committee hearing
Here's some potentially serious news about the 2011 Census.
Nick Hurd MP, the Shadow Cabinet Office minister has written to the head of the UK Statistics Authority (Sir Michael Scholar) saying "We do not believe that the 2011 Census can have the size and scope that has been proposed, particularly since we believe that many of the proposed questions are unsuitable........and urge you to scale back its cost and scope in order to save money and reduce its intrusive impact on citizens".
The Public Administration Select Committee is now to hold a session next Thursday 19 November on the 2011 census questions. Fortunately, I've been asked to give evidence at the hearing, along with Prof. Phil Rees (Leeds University) & a representative of the Equality and Human Rights Commission.
_______________________________________
I'm delighted to report that Stephen Penneck - winner of the DUG Award this year - has now been appointed as the new boss of ONS.
October 31, 2009
2011 Census plans across European countries
A conference on Population and Housing Censuses across Europe took place in Geneva last week (28-30 October).
All the papers are available at: http://www.unece.org/stats/documents/2009.10.census.htm
The great majority of the 100+ participants were from the Census Offices as suppliers, but I had the chance to talk about "Meeting the needs of Census users in the United Kingdom’s private sector".
For me, the headlines of the conference were:
Data collection:
* The increasing use of administrative registers (good news), although traditional Censuses are still in the majority; there's also increased use of sampling (bad news for small area stats)
* The importance of comprehensive address registers, to get maximum coverage
Outputs:
* Progress on harmonising topics / questions
* BUT, geographical detail is very variable
* Eurostat are interested in Regions & LAs (rather than small areas), and are getting bogged down in trying to deliver multi-dimensional tables (their planned hypercubes run into problems of disclosing info about individuals)
* I made the case (& got it recorded) that the major interest of UK users is for the smallest possible areas, even if for limited variables. This got picked up, with the focus for a meeting next year on dissemination and geo-visualisation.
If you are interested in pursuing the availability of data for other European countries, do get back to me.
Keith Dugmore
_______________________________________
October 16, 2009
DUG's 2009 Conference - News
Here's news of our Conference "Insight and the recession – How we can achieve more and better for less” which was held on 8 October:
* Feedback - this was probably best summarised by Rob Harris, Head of Branch Knowledge at Barclays: "The value this conference has added, and the benefits to my team, are alone worth the annual DUG subscription"
* The presentations are now on our website (see the tab)
* The DUG Award for “Better information from government” was scooped by Stephen Penneck of the Office for National Statistics for the decision to code the ONS’s sample surveys with its OAC geodemographic classification. To quote the citation: "DUG's Conference Board sees this as an excellent example of government using existing sources to create valuable new information at modest cost, which is so important in the current economic climate."
_______________________________________
October 15, 2009
Statistics User Forum conference - 19 November 2009
Here below are details of this year’s Stats User Forum conference on “The measurement of progress”.
The world is changing. The financial crisis has shown that the gains from an excessive focus on debt-fuelled economic growth were short-lived and failed to deliver a sustainable increase in well-being. The pursuit of ever larger economies led to environmental degradation and global imbalances rather than a reduction in poverty, an increase in social justice or a happier society. Post-recession, leaders and their electorates will strive for a greater focus on sustainability to ensure that future gains will be longer-lasting. But what will be the new goals and how will progress towards them be monitored?
See more details at: www.rss.org.uk/sufconference
_______________________________________
October 14, 2009
BBC - Bringing the fight for data home
Following in the trail of the Guardian's long-running Free Our Data campaign, we now have coverage from the BBC:
" Bringing the fight for data home.
The data held by local councils has become the latest target of digital activists. They are keen to get at the information so citizens can put it to their own uses.
Before now the push to get at official data has concentrated on central government. Moves to open up that data took a big step forward in early October when a few web developers were invited to a trial of the data.gov.uk site."
Read more at: http://news.bbc.co.uk/1/hi/technology/8297572.stm
_______________________________________
October 6, 2009
Postcode news: UK (bad) & Republic of Ireland (good)
We have both bad & good postcode news today:
1. Legal threat closes postcode feed (in the UK) - news from the BBC
"Websites that help people find jobs or hospitals have been hit by legal action threatened by the Royal Mail. The threat was issued against the company supplying them, and many other sites, with postcode data. Royal Mail said the legal action was threatened to stop "unauthorised access" to the postcode data." Read on at:
http://news.bbc.co.uk/1/hi/technology/7700621.stm
2. Online retailers welcome postcode system for Republic of Ireland - news from Retail Week
"Retailers have welcomed the revival of plans to introduce a national postcode system in the Republic of Ireland and believe the move should help catalogue and online businesses operate efficiently there. Other than Greece, Ireland is the only EU member state that does not have an established postcode system, which means that offering a home delivery service is a complex and often expensive challenge. But last week the Irish government said it intends to introduce codes comprising digits and letters in 2011.". There's more at:
http://www.retail-week.com/multichannel/online-retail/online-retailers-welcome-postcode-system-for-republic-of-ireland/5006857.article
You'll find further detail & comment at:
http://www.insideireland.ie/index.cfm/section/news/ext/postalcode001/category/1087
http://www.greenparty.ie/news/latest_news/ryan_announces_new_post_code_system_for_ireland
________________________________________October 2, 2009
MRS Census & Geodemographics Group Seminar - Targeting for multi-channel marketing - Monday 23rd November (afternoon)
The next CGG seminar has just been launched, on the important topic of Targeting for multi-channel marketing.
When: afternoon of Monday 23rd November
Where: MRS, 15 Northburgh Street, London EC1
Here's a synopsis of the topics to be discussed, followed by a link to the MRS website - for further details and online booking.
Today's consumers live in a multi-channel maze, targeted by advertising, direct marketing and sales calls. And in the online world, they also receive email marketing, search engine ads and web site offers. With such a range of offline and online channels at their disposal, how do companies select their marketing resources most effectively and measure the ROI? And is there still a role for geodemographics in targeting across offline and online channels?
This half -day seminar will discuss these topics and look at data for targeting, whether online or offline, and some of the latest analytic techniques being applied to multi-channel media. Further details and online registration are available at:
http://www.mrs.org.uk/networking/cgg/multi_channel.htm
________________________________________
September 30, 2009
DUG Conference, The Royal Society, 8 October 2009
"Insight and the recession – How we can achieve more and better for less”
9.30 Registration and coffee
10.00 Chair’s introduction. Richard Alldritt (UK Statistics Authority)
10.15 Demonstrating the value of insight at O2. Andrew Day (O2)
10.45 ONS’s business case for the 2011 Census. Glen Watson (ONS)
(11.15 Coffee)
11.40 Using OAC to create new insight from the Expenditure & Food Survey. Professor Martin Callingham (Birkbeck College)
12.10 Google’s vision for more and better information. Kelly Allison (Google)
(12.40 Lunch)
14.00 Parallel workshop sessions:
A) Triangulation: what’s the scope for comparing more information derived from company customer records and government administrative files? Facilitators: Martin Bellingham (The Children’s Mutual) and Roma Chappell (ONS)
B) How can we demonstrate the real value of customer and citizen segmentation? Facilitators: Ian Symis (Boots) and Sara Jones (DCSF)
C) What makes for a successful insight / research & information team? Facilitators: Mike Whitelegge (M&S) and Jenni Venn (Coventry City Council)
(15.30 Tea)
15.50 Workshops – summaries of views
16.10 Panel Session: More and better for less – how we can improve insight and its impact during the recession
Featuring views from Professor Mike Batty (UCL), Steve Penneck (ONS), Andrew Tasker (Optimum Locations & DUG), and Keith Dugmore (DUG)
16.45 Chair’s closing remarks, and The DUG Award 2009
17.00 Reception
The presentations will be posted on www.demographicsusergroup.co.uk
________________________________________
10.00 Chair’s introduction. Richard Alldritt (UK Statistics Authority)
10.15 Demonstrating the value of insight at O2. Andrew Day (O2)
10.45 ONS’s business case for the 2011 Census. Glen Watson (ONS)
(11.15 Coffee)
11.40 Using OAC to create new insight from the Expenditure & Food Survey. Professor Martin Callingham (Birkbeck College)
12.10 Google’s vision for more and better information. Kelly Allison (Google)
(12.40 Lunch)
14.00 Parallel workshop sessions:
A) Triangulation: what’s the scope for comparing more information derived from company customer records and government administrative files? Facilitators: Martin Bellingham (The Children’s Mutual) and Roma Chappell (ONS)
B) How can we demonstrate the real value of customer and citizen segmentation? Facilitators: Ian Symis (Boots) and Sara Jones (DCSF)
C) What makes for a successful insight / research & information team? Facilitators: Mike Whitelegge (M&S) and Jenni Venn (Coventry City Council)
(15.30 Tea)
15.50 Workshops – summaries of views
16.10 Panel Session: More and better for less – how we can improve insight and its impact during the recession
Featuring views from Professor Mike Batty (UCL), Steve Penneck (ONS), Andrew Tasker (Optimum Locations & DUG), and Keith Dugmore (DUG)
16.45 Chair’s closing remarks, and The DUG Award 2009
17.00 Reception
The presentations will be posted on www.demographicsusergroup.co.uk
September 12, 2009
Department of Transport adds 350,000 public transport access points to Open Street Map
The Department for Transport has recently released data from the National Public Transport Access Node (NaPTAN) database to be put on Open Street Map (OSM). The data contains details of some 350,000 public transport access points in Great Britain including bus stops, railway stations, tram stops and ferry terminals, and meaningful text descriptions of the stop point and its location.
See lots more detail at:
http://blog.okfn.org/2009/08/20/where-is-the-nearest-bus-stop-uk-department-for-transport-adds-naptan-data-to-open-street-map/
It is also stated that Ordnance Survey has no rights to the data, and adds a comment: "This is excellent news - and big kudos to the DfT for donating the data! We hope that other departments consider following suit and adding their geodata to OSM!"
________________________________________
September 12, 2009
Guardian's Free Our Data - Ordnance Survey under fire again
Charles Arthur has written two more articles about Ordnance Survey - the most recent one is a cracker:
3 September, 2009: "Ordnance Survey business model costs appear in leaked presentation"
http://www.guardian.co.uk/technology/2009/sep/02/ordnance-survey-business-model-costs-leak
10 September, 2009: Don't write it down: Ordnance Survey kept no notes of talks with 'expert'
"You have an important study that will be part of your case to a government minister considering the future of your organisation. You have found someone to review it. Would you commission them without swapping letters or email, and without taking any notes of their observations afterwards?
If you're Ordnance Survey, then it seems yes, you would. In a response to a Freedom of Information request – part of a series of requests Guardian Technology has made since May – OS revealed that after it prepared an internal study comparing international models of funding mapping agencies, it asked an "internationally recognised expert in geographical information and national mapping" to review it.
But in an admission that makes the preparation of this potentially key report look rather slipshod, OS said in response to a further FoI request that "a copy of the report was provided to the person concerned and engagement on this matter was conducted orally with no permanent record made of these conversations".........
Read on, at http://www.guardian.co.uk/technology/2009/sep/11/free-our-data-freedomofinformation
________________________________________
August 28, 2009
ONS's latest Population & Migration estimates - UK population grows to 61.4m; birth rate rising
Here below is the latest population news from ONS.
You'll probably have seen lots of press coverage today, e.g. "New baby boom for Britain" (Daily Telegraph):
http://www.telegraph.co.uk/news/newstopics/politics/lawandorder/6100469/New-baby-boom-for-Britain.html
[what a surprising web address / index!]
________________________________________
August 19, 2009
"Data Strategy" - MP attacks "scandal" of Census address data
Read all about it at:
________________________________________
August 7, 2009
Jil Matheson appointed as the new National Statistician
I'm delighted to report that Jil Matheson has been appointed as the new National Statistician.
The official announcement is at http://www.statisticsauthority.gov.uk/news/appointment-of-the-national-statistician.pdf
The scope of the job has been increased, with a wider remit across government, and involvement in the recruitment of a new Director General for the ONS.
Jil has been very supportive of DUG for many years, and at last year's Conference she was on the panel session.
________________________________________
July 30, 2009
Guardian's Free Our Data - "Government advisors criticise OS's strategy"
Charles Arthur writes:
"Ordnance Survey's proposed business strategy has been strongly criticised by the government's Advisory Panel on Public Sector Information (APPSI), the body that advises ministers on how to encourage wider reuse of public sector data. Separately, OS has finally released an almost unredacted version of its internal report comparing international funding models for mapping agencies – but has declined to name the "internationally recognised expert" who checked it, saying the person wants to remain private......"
and concludes with:
"..............Separately, OS responded, at the end of the last possible day under Freedom of Information Act rules, to Guardian Technology's request for a copy of its internal report on funding models. Almost all the data redacted in the first version has now been released – but nothing seems to back up the statement in May by Sir Rob Margetts, OS's chair, that it showed that moving to a free data model would in the first five years cost "between £500m and £1bn". Guardian Technology will seek to ascertain the basis of Margetts's claims, and any more information about the identity of the expert who checked the study – such as, for example, whether they are an employee of Ordnance Survey."
You can read the full article at http://www.guardian.co.uk/technology/2009/jul/29/free-our-data-ordnance-survey
________________________________________
July 15, 2009
Seven out of Ten Woolworths stores still empty
Here’s a news item from The Local Data Company:
"We recently undertook some research into what happened to over 800 Woolworths stores following Woolworths demise in January - below is some of the commentary from the press:
http://business.timesonline.co.uk/tol/business/industry_sectors/retailing/article6695205.ece
http://www.retail-week.com/property/in-town/70-of-old-woolworths-stores-still-lie-vacant/5004439.article
http://itn.co.uk/63820fae409ac3b6119964fb906539ea.html
http://www.teletext.co.uk/regionalnews/anglia/c7e9a62136faeb94acf8d3b0307b5371/Woolies+stores+still+empty.aspx
http://www.google.com/hostednews/ukpress/article/ALeqM5i0B2uY7HUZLFMXa7uT_oRisfBdgw
______________________________________
July 11, 2009
National Address Register - support from MPs & the UK Stats Authority
Getting a definitive national address register is one of DUG's priorities, and we are making progress.
At the House of Commons Public Administration Select Committee session on the Census on 25 June Gordon Prentice MP questioned the head of ONS & quoted our submission:
"But these organisations [Royal Mail, Ordnance Survey and Local Government] hold intellectual property rights on the information that they are making available and you have to buy this information from those three organisations. I have been reading the submission that we have from the Demographic User Group and they say that it is scandalous - I thought I had copyright on that word - that you are spending all this money pulling together this information to create an address register and it is only going to be used once; it is only going to be used for the next Census and it is not going to be made available to commercial organisations and other people who could use that information. That really is scandalous, is it not?"
http://www.publications.parliament.uk/pa/cm200809/cmselect/cmpubadm/uc742-i/uc74202.htm
On 8 July the Chair of the UK Statistics Authority, Sir Michael Scholar, wrote to the Minister of State for Housing, John Healey, making the case for establishing a national address register.
http://www.statisticsauthority.gov.uk/reports---correspondence/correspondence/index.html
_______________________________________
July 2, 2009
"Ordnance Survey redacts public data in controversial report"
The Guardian's Charles Arthur's latest instalment:
"A controversial OS study, released in a censored version, leads to a single conclusion: the original was incomplete and fundamentally flawed.......
Are the land mass, population and GDP per capita of the US, Australia, Canada, France, Sweden, the Netherlands, Norway and New Zealand – as well as the legal form of their national mapping agencies – commercially confidential information? If you believe the UK's mapping agency, Ordnance Survey, certainly. Those details were redacted – blacked out in the manner of an MP's expenses – in the copy of a key study it carried out and released to the Guardian under the Freedom of Information (FOI) Act last week.....
Ed Parsons, the former OS chief technology officer who is now Google UK's geospatial technologist, dismissed the report: "OS's 'Research' Report reads like a poor MSc thesis. Why no user perspective?" "
Read all about it at: http://www.guardian.co.uk/technology/2009/jul/01/ordnance-survey-censored-report-released
________________________________________
June 25, 2009
"Ordnance Survey responds, at last, to Freedom of Information request"
The Guardian's Charles Arthur reports:
"The Ordnance Survey consulted mapping agencies in Australia, Canada, France, the Netherlands, Norway, New Zealand, Sweden and the US in compiling its study into possible financial models, it has revealed. However, OS considers that "there are no wholly equivalent organisations ... given [our] range of scales of mapping and other activities", according to data that it released under a Freedom Of Information (FOI) request from the Guardian.
Attention was immediately drawn to the identity of an unnamed "internationally recognised expert in geographical information and national mapping" who OS said had reviewed its study and "agreed with the analysis and conclusions......."
Read all about it at: http://www.guardian.co.uk/technology/2009/jun/24/free-our-data-ordnance-survey
________________________________________
June 18, 2009
OAC User Group free seminar: Developments in Geodemographic Visualisation
The OAC (Output Area Classification) User Group is going from strength to strength, and has arranged the following meeting in September:
* Seminar: Developments in Geodemographic Visualisation
* Objective: The aim is to present a number of innovative developments in visualisation methods and technology which can enhance interpretation of geodemographic classification. This seminar is aimed at both a professional and academic audiences and will be interest to all users of geodemographics.
* Programme:
– Understanding Through Visualisation (Dan Vickers, University of Sheffield) – Effective Communication (Laura Edwards, Local Futures) – Geographic Visualisation (Alex Singleton, University College London) – Data Visualisation (Aidan Slingsby, City University) – 3-D Visualisation (Duncan Smith, University College London)
* Time & Date: 1.30pm, Tuesday 15 September 2009
* Venue: Royal Statistical Society headquarters, 12 Errol Street, London, EC1Y 8LX
* Further details & Registration: http://areaclassification.org.uk/2009/06/16/visualisation/#more-208
I expect that places will be snapped up quickly, so do register (not me - see above!) now if you'd like to attend.
________________________________________
June 18, 2009
Guardian's Free Our Data campaign - 2 more good articles today
Today's Guardian has two articles by Charles Arthur:
1) Commercial rival to take on Ordnance Survey with its own map data
“Ordnance Survey has a new challenger: a privately owned company – UKMap, part of the GeoInformation Group – that aims to provide local authorities and private companies with large-scale topographic mapping and an address database for British cities, created entirely separately of the government-owned agency…………..
……its arrival means there are now three organisations vying to produce maps of the UK: Ordnance Survey, which derives roughly half its income from licences by taxpayer-funded organisations; OpenStreetMap, a volunteer effort using advances in GPS technology to create a "wikimap" used by some local authorities (and by the Guardian for its Open Platform offering); and now The GeoInformation Group.”
(See the full article at http://www.guardian.co.uk/technology/2009/jun/17/free-data-ordnance-survey-maps/print )
2) Let Sir Tim lead the fight for free data
“….Gordon Brown has tasked Sir Tim Berners-Lee with getting government-collected data into more accessible forms, and hence available to more of us…….. Sir Tim nailed his colours to the mast in a riveting talk earlier this year, where he built on Hans Rosling's already thrilling work at Gapminder to emphasise the value of getting data out there into the hands of people who can think of useful, innovative, imaginative things to do with it; he got the audience to chant "Raw data now! Raw data now!"…… [Even we haven't done that]
(See the full article at http://www.guardian.co.uk/technology/2009/jun/17/charles-arthur-free-data/print )
________________________________________
June 5, 2009
ONS advice on measuring change in small areas
ONS is providing some good advice on analysing change in small areas at:
http://neighbourhood.statistics.gov.uk/dissemination/Info.do?page=analysisandguidance/analysistoolkit/analysis-toolkit.htm
You can also get hold of a free copy of their Change over Time Analysis (CoTA) Viewer on CD at:
http://neighbourhood.statistics.gov.uk/dissemination/Info.do?page=analysisandguidance/analysis.htm
________________________________________
May 16, 2009
ONS admits to getting its sums wrong with overstated retail sales - FT 16 May 2009
Following the discussion of members' doubts about ONS retail stats at our Awayday in December, the Financial Times (Saturday 16 May 2009) reports:
"One of Britain's most closely watched economic indicators has heavily overstated the quantity of high street sales over the past two years, the Office for National Statistics admitted yesterday. Britain's supplier of official statistics conceded that since the financial crisis began in August 2007, it has overstated the volume of retail sales growth by 56 per cent."
Read all about it at: http://www.ft.com/cms/s/0/97faf9b2-41b3-11de-bdb7-00144feabdc0.html?nclick_check=1
________________________________________
May 16, 2009
Shop vacancy rate in towns rises to 12% - FT 16 May 2009
The Financial Times (Saturday 16 May 2009) reports:
"The number of empty stores in the UK's high streets leapt three-fold in just six months as a growing number of retailers were forced out of business by the worsening economic climate.
High-profile casualties such as Woolworths contributed to a rise in vacancy rates from 4 per cent to 12 per cent in the six months to the end of March. Vacancy rates in terms of floorspace are lower, at 10.8 per cent.
Barnaby Oswald, managing director of the Local Data Company, which surveys 675 town centres across the UK, said that the number of businesses closing has increased significantly since last summer"
Read all about it at: http://www.ft.com/cms/s/0/b7a7955e-41b3-11de-bdb7-00144feabdc0.html
_______________________________________
May 8, 2009
Ordnance Survey - "new" business strategy - further criticism in The Guardian
Following the news of Ordnance Survey's "new" business strategy (see below), the Guardian has two more articles critical of the government's plans:
* Budget misses free chances (Michael Cross, 30 April 2009):
"The government has thrown away an opportunity to boost the digital economy through a "free data" regime. Announcements made with the budget dashed hopes of radical reforms to the "trading fund" status of state-owned information businesses.
Even worse, the plans do not deal with "derived data" - one of the most contentious fields in intellectual property creation, whereby the creator of a product claims ownership of any IP built with it. Ordnance Survey's aggressive annexation of "derived data" - any detail, such as a cycling route, created using an OS product - has riled a number of people in the commercial sector. "Derived data is the elephant in the room," says Ed Parsons, Google UK's geospatial technologist..........."
Read all about it at: http://www.guardian.co.uk/technology/2009/apr/30/government-digital-economy
* Break down the boundaries in Ordnance Survey (Charles Arthur, 7 May 2009):
"The government's decision to consult the public on its reorganisation of Ordnance Survey - announced with the budget last month - is proving popular with would-be users. The proposed reorganisation, however, is not.
Comments on the web page, where the consultation is open for feedback are running overwhelmingly in favour of more freedom for OS data............"
Read all about it at: http://www.guardian.co.uk/technology/2009/may/07/ordnance-survey
________________________________________
April 26, 2009
Ordnance Survey - "new" business strategy announced
Triggered by the Budget, a "new" business strategy has been announced for Ordnance Survey.
(I put the "new" in quotes. The strategy has at least avoided the madness of a rumoured privatisation, but has not gone for the radical option of freeing the data to increase both use and consequent economic activity. Rather, having hit the internet and Google Maps icebergs, it is another attempt to rearrange the deckchairs on the Titanic).
Here are the headlines, followed by Michael Cross's Free Our Data view in the Guardian:
1) The OS Announcement
"The strategy aims to improve ease of access to geographic data and services for both commercial and non-commercial use, whilst also striving to maintain a balance with the need to stimulate innovation in the geographic information market and make data more widely available. This means that Ordnance Survey will continue to be self-funded and earn revenue by licensing its data, but it will make sure it is easier for customers and other businesses to access its data and services.
The strategy focuses on five key areas:
• Promoting innovation – with an enhanced free OS OpenSpace service to allow experimentation with digital information and a clear path from this service to greater commercialisation;
• Reforming Ordnance Survey’s licensing framework – so that it is much simpler to use Ordnance Survey data and services in other applications;
• Reducing costs over time – to ensure that Ordnance Survey continues to offer value-for-money;
• Supporting the sharing of information across the public sector – to enable better public policy and services;
• Creating an innovative trading entity – to explore commercial opportunities around providing a better platform for consumers to access Ordnance Survey products.
The new strategy will be developed further and implemented over the coming 12 months but the five key areas above are being opened up for comment from Ordnance Survey’s customers and other stakeholders."
Full details of the strategy and how to comment can be found at: http://strategy.ordnancesurvey.co.uk/
2) Guardian Free Our Data article: Government ducks free data decision (23 April 2009)
Michael Cross refers to the headline finding that "a model where a user pays a licence fee for OS data continues to be the most effective way of balancing the need to increase the availability of geographic information to the wider UK economy and society while maintaining the quality of OS data". He considers the extension of OS’s OpenSpace service as a win for the Free Our Data campaign. However, he believes that the new subsidiary company set up to “ensure that OS is sustainable for the medium term and value is generated for the taxpayer" implies that this "innovative trading entity" has been established with an eye to privatisation – although, he points out that Iain Wright, shareholder minister for OS says this is not the case.
The full article can be accessed at: http://www.guardian.co.uk/technology/2009/apr/23/state-owned-digital-information
I'll be interested to hear your views. In particular, have you got opinions about OS OpenSpace?
________________________________________
April 19, 2009
"Social Trends 2009" published
You probably saw some lurid headlines on Wednesday's newspapers - the latest Social Trends report has been published.
Amidst all the comment about social change in Britain, the report is often a good first start for finding out what are the key data sources for particular topics.
You can download it (or bits of it) free from: http://www.statistics.gov.uk/socialtrends39/
________________________________________
April 9, 2009
UK Statistics Authority report on migration statistics
This report has just been published. It is policy stuff, rather than providing new stats now, but its recommendations (Pages 8-11) reflect our views on the need for an address register, & more use being made of government administrative data:
"We recommend that, in the light of the complex dependencies involved, discussions about matters such as the future of the Census beyond 2011, the longer term maintenance of a national address register, related developments (such as identity cards and a population register) and the best way to improve migration statistics, should now be taken forward together at the highest level. Discussions should involve the Statistics Authority and ONS and the relevant government departments – notably HM Treasury, the Cabinet Office, Department for Communities and Local Government, Home Office, Department for Work and Pensions, and the devolved administrations."
You can download the full report at:
http://www.statisticsauthority.gov.uk/reports---correspondence/reports/migration-statistics--interim-report.pdf
________________________________________
April 8, 2009
European Geodemographics Conference - 1 April - Presentations now available
This MRS conference broke new ground, looking at the availability of demographic data across Europe.
The presentations are now available on the MRS website at: http://www.mrs.org.uk/networking/cgg/euro_conference_prog.htm
A report on the conference – including the action points for the future – will be posted on the website in the next couple of weeks and there will be an article in the June issue of GeoConnexion.
________________________________________
March 18, 2009
Google Street View
Google Street View - providing recent photos - has just been released: http://maps.google.co.uk/ There's coverage of Greater London and c.20 other cities. Drag the person icon onto the road that you want to see.
Early reactions from DUG members:
"Already tested it out in Manchester - very impressive - will make mapping exact city centre locations much more accurate."
"We were looking at this this morning, looks fab ..we have been virtually wandering around Nottingham and Edinburgh."
"We had an enjoyable 5 minutes finding Tony Blair's house and discovering that they haven't pixellated the policeman's face outside. Google seems to hit the nail everytime at the moment !!!"
"We spent a good 20 minutes on this earlier - absolutely fantastic stuff - love the zoom feature, the 360 toggle, and there is a good coverage across a lot of the major towns and cities (and Scunthorpe). Picture quality and load rate seems to be very impressive."
________________________________________
March 10, 2009
Availability of services at Output Area level
Here’s news of a new dataset from the Commission for Rural Communities (in England):
“Service availability and accessibility has been a key rural policy issue for many years. In order to monitor changing levels of service availability, this indicator combines datasets that show where outlets of particular services can be found, with postcode level information on the distribution of households to calculate numbers of households within distance of key service indicators.”
Looking at the link: http://www.ruralcommunities.gov.uk//projects/ruralservicesseriesdata/overview
• The input data includes the locations of banks, building societies, ATMs, supermarkets, pubs, etc.
• The output data appears to be for all Output Areas, not just rural ones (the data file is very large)
• The website includes some maps
________________________________________
March 5, 2009
Guardian - Free Our Data - The price of un-free data: one-off census address list will cost £12m
Michael Cross, writing in The Guardian, is adding to the pressure for the ONS's 2011 Census address file to be made freely available:
"The cost of creating a national list of addresses for the next census has risen to £12m, the government has admitted. The figure – £2m up on previous estimates – is the latest revelation of the true cost of "unfree" data across the economy....The Office for National Statistics has to build its own address database for the 2011 census in England and Wales (Scotland has its own parallel exercise) because none of the three existing, state-owned address databases is comprehensive enough.......Attempts to create a unified list have failed because of arguments about intellectual property. The three list-owners – local authorities, Ordnance Survey and Royal Mail – all treat address lists as their commercial assets."
See Michael's article at: http://www.guardian.co.uk/technology/2009/mar/05/free-data-addresses
________________________________________
March 4, 2009
Power of Information Taskforce Report published
This report (written by some influential people in government) has just been published.
It calls for action in several areas where the Taskforce believes significant improvements can be made to government's use of digital technologies. These include:
* freeing up the UK's mapping and address data for use in new services.
* ensuring that public sector information is made as simple as possible for people to find and use.
Looking at Recommendation 7 in particular:
"It is the Taskforce's view that 'freeing up' geospatial data should be a priority. The Ordnance Survey requires urgent reform. Recent announcements of cost reductions at the Ordnance Survey point the way to wider reforms. This reform should include at a minimum:
* Basic geographic data such as electoral and administrative boundaries, the location of public buildings, etc should be available for (re)use free of charge to all.
* There should be simple, free access to general mapping and address data for modest levels of use by any user.
* Voluntary and community organisations pursuing public policy objects should benefit from straightforward standard provisions for ensuring access to geospatial data at all levels of use.
* Licensing conditions should be simplified and standardised across the board and, for all but the heaviest levels of use, should be on standard terms and conditions and should not depend on the intended use or the intended business model of the user.
* The OpenSpace API, similar to but currently a constrained version of Google Maps, should become the primary delivery point for the Ordnance Survey's services.
* Creation of a freely available single definitive address and postcode [file] available for the UK for (re)use."
All this is heading in our direction (albeit in difficult economic & political times).
Read more at http://poit.cabinetoffice.gov.uk/poit/
________________________________________
January 28, 2009
Experian in the news
Here's a story in yesterday's Times - "Lord Taylor tried to aid credit-rating agencies access to business" - which includes mention of Experian.
http://www.timesonline.co.uk/tol/news/politics/article5601428.ece
________________________________________
January 24, 2009
How geodemographics is being applied across an ever expanding Europe
MRS CGG Conference - 1 April 2009, London
This year's Market Research Society Census & Geodemographics Group conference focuses on European data, and on how analytical techniques such as geodemographics are being used across Europe to help organisations to optimise their definitions of target markets.
This will be considered from a number of perspectives, ranging from data availability through to business applications, and will drive towards setting an agenda of priority actions required to fill existing gaps in the European data landscape. The programme includes speakers from CACI, Eurodirect, & Experian.
For full details and to book your place, see: http://www.mrs.org.uk/networking/cgg/euro_conference.htm
_______________________________________
January 21, 2009
Dwelling stock by council tax band, 2007 - statistics for small areas
(From a message to DUG Contacts)
Data for Neighbourhood Renewal has alerted me to the following new statistics now available on ONS's Neighbourhood Statistics website:
* Dwelling stock by council tax band (Updated for 2007) - down to Output Area (OA) level for England & Wales: http://www.data4nr.net/resources/53/
We now have datasets for each year since 2001, which in principle enables the tracking of changes in numbers of dwellings and their status (Bands A-H) for very small areas.
If you've done any analysis, I'll be pleased to hear.
__________________________________________
January 8, 2009
Guardian - Free Our Data - Royal Mail is wiping many rural homes off its postcode database
(From a message to DUG Contacts)
Here's Michael Cross's latest article in today's Guardian:
http://www.guardian.co.uk/technology/2009/jan/08/scotland-applecross-royal-mail-highlands-paf-postcode-address-file
It includes:
"Royal Mail says it has a policy of removing addresses from the database when houses are unoccupied. "If the postie can no longer reach the delivery point, or if a house is obviously completely unoccupied, the postie informs us and the address is removed from the PAF. If it later becomes occupied, it would be put back on""
He then goes on to point to the feuding over address files between Royal Mail, Ordnance Survey and local government.
______________________________________________
December 21, 2008
Members' Locations - latest news
(From a message to DUG Contacts)
I'm pleased to let you know that our new member Boots is planning to join in our initiative to share information about locations, and that we are also planning quarterly updates.
For the record, the current list of sharers is:
* Abbey
* Argos
* Boots
* Co-op
* John Lewis
* M&S
* Sainsbury's
In addition, Tesco have pointed me to a SatNav link which may be helpful:
http://www.tesco.com/storeLocator/satnav_download/
______________________________________________
December 18, 2008
The Times 18 Dec 2008 - Analysis: Why retail figures don't add up
(From a message to DUG Contacts)
Following our discussion at our Awayday, I've been alerted more doubts about ONS's retail stats.
Here's the link:
http://business.timesonline.co.uk/tol/business/industry_sectors/retailing/article5364630.ece
[This was the first message on our new website]