Reform geospatial data (draft)

 

Trading Funds

The Taskforce has been impressed by the extent to which access to geospatial data has been a recurrent theme during its activities.  For example, the Show Us A Better Way competition had around 500 entries and of these over one third were for ideas around maps and location. It is possible to speculate why this is: perhaps people want ‘government’ to present a simple, smart face based on location and service; perhaps they want to plan how to get to the hospital or the quickest route to school.

The Taskforce spent some time looking at the issue of crime mapping which has excited much interest over recent months. We were struck that police forces such as the Met Police chose to implement a service based upon GoogleMaps rather than any directly-sourced Ordnance Survey product.  The Taskforce followed the interesting debate that was generated around the use of data which has been ‘derived’ from Ordnance Survey maps.

Maps are an easy to understand way of presenting complex information. However, until recently creating tools for presenting information on maps was very difficult and expensive to do. This is no longer the case, e.g. Flowing data. Since around the time of the launch of online mapping services such as Multimap and Google maps with subsequent opening of APIs for easy reuse there has been a steady decline in the complexity and cost of development. It is now possible for people to create innovative mapping services in their spare time on a cheap laptop.  This should be a tremendous spur for innovation in the UK.

In the Ordnance Survey the British Public have very high quality maps with universal coverage and rapid incorporation of changes but there seemed to be an unusual number of barriers to society and the economy making the best use of this service.  There is a high demand for map-based public sector information services. But the complex and legalistic licensing and charging regime offered by the Ordnance Survey is acting as a barrier, both real and perceived, to innovation in this area.

The importance of the information held by the trading funds has been highlighted repeatedly over the years. This has been reinforced in recent times by e.g. in the original Power of Information report, the Government’s Location Strategy, the work of APPSI, the OFT CUPI study and a report by  Cambridge economists commissioned by the Treasury. Research by Oxera for the Ordnance Survey suggests that their information alone underpins 12-20% of economic activity. While the points in this section can generalise to all government information businesses e.g other trading funds or the Environment Agency, both the scale of the prize and the behaviour change needed create a focus on Ordnance Survey.

However, the current access regime is aimed at maximising the average return on capital for a single data business, not the overall welfare of the UK.  Economic theory generally holds that maximum welfare is generated from charging at marginal cost, but the Ordnance Survey charges out at average cost as part of its Trading Fund approach. Analysis by Cambridge University suggests that current pricing directly reduces the size of the UK economy by £190m a year, in a sector that has been growing at an average rate of 9% a year.  If the impact found by Oxera is true then this figure will be much higher.

The taskforce judges that the charging and licencing regime stifles innovation in public service delivery and economic activity.

  • Who runs local services – The ability to discover easily administrative boundaries is essential for democracy.  At present these are held by the Ordnance Survey and cannot be presented free at the point of use to the public in a form they can reuse. For example, despite the fact that electoral areas are set down in statute, the Ordnance Survey hold the copyright to displaying the regions on a map.
  • Finding public services – Bulk information about schools has recently been made available. It does not contain precise locations provided to the Ordnance Survey by the schools because of perceived problems with licensing.
  • Crime Mapping – Crime Maps were announced by the Home Secretary in the July 2007 Crime Strategy. Inspired by the Taskforce’s crime map mock ups, some forces were looking at a Google Maps interface. Ordnance Survey claimed that this would breach their licence but don’t (at date of writing) allow public sector use of Openspace.
  • Census information – The census provides high quality local information. Despite the census areas being original work, licensing concerns have stopped the ONS providing an online geospatial interface to their data
  • Local Authority information – one large local authority expressed bewilderment to the Task Force that the location data for its own street furniture seemed to be owned by the Ordnance Survey.  The Ordnance Survey often claims derived copyright in public service locations, often despite the original information coming from other public bodies.

This is not new analysis. The importance of geospatial data was identified by Steinberg and Mayo in 2007 but for users, the situation remains unchanged.  There is now a pressing need for reform at the Ordnance Survey. Shareholder Executive and HM Treasury are currently undertaking a review of the trading fund business model. This should seize the opportunity to recast the Ordnance Survey as a mapping agency for the 21st Century. Technological advances have shifted the fundamentals of the traditional Ordnance Survey business model and there is a real risk that it will find itself an anachronism by open rivals supported by technological change.

It is the Taskforce’s view that the Ordnance Survey requires urgent reform.  Recent announcements of cost reductions at the Ordnance Survey point the way to wider reforms. This reform should include at a minimum:

  • Basic geographic data such as electoral and administrative boundaries, the location of public buildings, etc should be available free of charge to all.
  • There should be simple, free access to general mapping and address data for modest levels of use by any user
  • Voluntary and community organisations pursuing public policy objects should benefit from straightforward standard provisions for ensuring access to geospatial data at all levels of use
  • Licensing conditions should be simplified and standardised across the board and, for all but the heaviest levels of use, should be on standard terms and conditions and should not depend on the intended use or the intended business model of the user.
  • The OpenSpace API, similar to but currently a constrained version of Google Maps, should become the primary delivery point for the Ordnance Survey’s services



RSS feed of comments 39 Responses to “Trading Funds”

  1. Neil says:

    Yes,
    I’m glad that the taskforce understand that allowing the public access to data that has been paid for by their money is the right way to go.

    Of course we need license conditions in some case, but the current climate makes it hard to get data.

    In the Show Us A Better Way competition, I was after Royal Mail PAF data etc, the T&C’s that came with it were huge. Oh, and I didn’t get access to it either!

  2. One change that would make a HUGE difference would be to make “derived data” rules both clearer and much less restrictive.

    At the moment it would appear that if geographic data about anywhere in the UK has been generated with any assistance from OS mapping then it becomes the property of OS and hence covered by Crown Copyright. Since almost all maps in the UK are derived from OS data, this makes it very difficult to share geographic information.

    While copyright should (possibly) need to remain on bulk OS data, to protect OS revenue from bulk data licences, there should be little need to apply Crown Copyright to individual points, lines, and areas that are insufficient to re-generate the OS dataset. There seems to be EU legislation already in existence covering a similar problem with the copyrighting of databases: perhaps this sort of law could be applied to OS map data too?

    An alterative model would be to follow the USA, and distribute basic geodata freely (including boundaries, roads, etc.) for anyone to use. The OS could still charge licenses for more detailed or more regularly updated data where businesses need this.

  3. While the OpenSpace API’s current technological limitations (compared with the Google Maps API) might be acceptable as a trade-off for being able to use OS mapping freely, it would be much nicer to be allowed to display “derived data” such as points, lines and areas in any map API, including Google’s. But this, again, would require a relaxation of the derived data rules, or a more relaxed interpretation of the Google Maps API terms of use by the OS.

  4. Feargal Hogan says:

    It is astonishing that it is so difficult to analyse any public data around simple boundary data. This (http://tinyurl.com/bb9jb7) pdf data should be easily displayed on a map and yet it isn’t.

  5. Jeni Tennison says:

    It’s not clear what the link to Flowing data in the third paragraph is supposed to demonstrate, especially since it appears that the page it links to is constantly being updated with new projects. Is there a particular project there that demonstrates the ease/lack of expense of creating map-based visualisations? If so, link to it directly.

  6. Jeni Tennison says:

    ED: 5th paragraph, remove the final “of activity” in “Research by Oxera for the Ordnance Survey suggests that their information alone underpins 12-20% of economic activity of activity.”

    MODERATOR NOTE – thanks, now corrected.

  7. Jeni Tennison says:

    The focus is very much on OS in this section, but there are other sources of geospatial data that need to be made more open and reusable in order to support applications based on maps and location.

    The primary one is postcode data. Most people identify their location through a postcode (rather than a latitude and longitude), so many geospatial applications require a mapping from postcode to location. There should be at least some mention of Royal Mail’s ownership of that data.

    A second and third are the national street gazetteer (http://www.thensg.org.uk/) and the national land and property gazetteer (http://www.nlpg.org.uk/).

    A fourth is the set of geographic/neighbourhood information held by the Office for National Statistics.

    I think the report would benefit from a bit more detail on each of these to give a wider impression of the barriers to using geospatial data rather than focusing solely on the (admittedly huge) barrier posed by the OS.

  8. Andy Nicholson says:

    The report refers to OS updating regime as one that offers the “rapid incorporation of changes”. Whilst in my opinion there is no argument that OS requires sufficient funding to maintain its base mapping update programme through its field survey teams I must question the use of the phrase “rapid” in relation to the current OS Update target. To the best of my knowledge the OS update SLA has not been improved for at least 10 years. With our world changing around us 6 months for properties to appear on a map is no longer acceptable to Service companies who have to plan routes and communication plans for the customer who is going to occupy these premises, let alone provide a service once they move in. Sadly there are just some things that have to be paid for to maintain position and detailed mapping I would place solidly in that category.

    It is also right to draw reference to the over complicated and tangled web that OS weave with their product licensing and copyright documentation. I realise that OS accredit much of this to the diverse range of customers they have to service. All of these have differing commercial and private operating models and hence OS say that sadly they have to preserve this level of documentation so as not to dis-enfranchise any one customer. However in line with far more worthy commentators in the UK world of geo-spatial commentary I feel that there is a strong need for a sell it cheap approach (if not free). Funding should perhaps come from a more central support in recognition of the over-arching importance of detailed mapping data to the digital future the UK deserves. If we could all put our mistrusts of OS aside and obtain detailed address and mapping data a low cost there then we would all have a common and uniform Spatial Data Infra-structure which we could work with. The challenge of background detailed mapping could not be held up as a reason to not share data for the common cause and neither would the GIS community suffer from the numerous challenges a multitude of data sources and backgrounds causes.

    I also support other commentators in their criticism of this report in that it largely refers only to geo-spatial data within the OS portfolio and this is a distinct weakness. Of equal importance would be many other data streams including but not limited to spatially maintained address data, aerial and satellite imagery and the maintaining of public records in a spatially enabled method.

    Lastly as a frustrated GIS user I would appeal for action – not every step has to be in the right direction but we cannot afford to stand still.

  9. Simon Whitehouse says:

    I think that you are right to highlight the need to reform the way Ordnance Survey licences data. As a public sector worker I find that this is stifling innovation and creativity in presenting information to the public. As a member of the public I am frustrated that my data is not being presented to me in a format that I am increasingly coming to expect to be the norm.

    There should be work done to standardise the Application Programming Interfaces into the data as soon as this becomes available, or before. Mapping websites will want to work across local government boundaries and having a common API for all will enable such sites to develop a lot quicker and more easily.

  10. Dave Crossland says:

    I think the current proprietary, tight-fisted nature of OS maps is something of a public outrage.

    These days with all the economic stimulus packages flying about, making OS maps much more useful – eg, Creative Commons BY-SA – would be something useful for building useful things instead of wasting money.

  11. Tony Hirst says:

    Looking at the national Statistics website the other day, I noticed that a large number of statistics are reported according to Government Office Regions. making available a simple flash map with GOR boundaries marked and a simple API (something like http://www.ammap.com/examples/countries/united_kingdom_regions maybe?) would allow for easy production of choropleth maps based on the GOR data?

  12. Stuart Lester says:

    The OS derived data is key alongside postcode search. These are the major barriers to presenting the simplest data that would cover 80% of the public need. Having said that they are a fine excuse for a Local Authority not being able to share data covering up a myriad of internal problems – that would be the next layer to expose.

  13. Christopher Roper says:

    I am delighted the Task Force wants to reform OS, and I agree with many of the things you want, BUT you also have to will the means, and that requires a much more fundamental rethink, beginning with the question: “What does the government require from its National Mapping Agency in the 21st Century”. The existing business model is broken. Agreed. What do you want to put in its place, and how should it be funded?

  14. Christopher’s absolutely right.

    I think what’s needed isn’t _mere_ reform, but _radical_ reform.

    The data currently owned by OS isn’t similar to data owned by public authorities. It is far more important than that. Accurate geospacial data is a part of the infrastructure of the country. It is important, one way or another, to almost every area of commerce, let alone public service.

    If the Taskforce could do only one thing, and I got to pick it (!) I would change the OS from a trading fund to an executive agency or NDPB, and make its data sets available to all for free, including in bulk.

  15. Denis Payne says:

    There is no question that OS (& Royal Mail) need to protect their IPR.

    However, restrictive practices in this arena are not in the best interest of UK plc.

    Whilst it may be important to get this right, it is also vitally important to sort the key issues quickly.

    Generation of new licence terms (eg Creative Commons), agreeing the basics of how to make this data free could be, and should be, done in 3-6 months.

    Responding to the points raised by Christopher Roper and Henry Metcalfe is important, but should proceed in parallel, rather than series, with the basics.

  16. Rob Dunfey says:

    I think you have hit the nail on the head here.

    One possible typo:

    “This should seize the opportunity to recast the Ordnance Survey as a mapping agency for the 21st Century.”

    Should “This” be “They”?

  17. Rob Dunfey says:

    Just a couple of thoughts on two of the recommendations:

    Number One:

    “Basic geographic data such as electoral and administrative boundaries, the location of public buildings, etc should be available free of charge to all.”

    I agree whole heartedly, but just wonder if the point could be made more explicitly, perhaps such details are beyond the scope of the report – but I can see the following argument being made by the Ordnance Survey in response to such a request…”this information is already available via the Ordnance Survey Openspaces API”. The problem is that the information is provided as images, which restricts its usefulness. In essence you can do little more than ‘view’ the data; access to geographic data in a vector format would allow for computational ‘analysis’ of spatial data, which, provides further scope for innovative use of the datasets.

    Number Two:

    “The OpenSpace API, similar to but currently a constrained version of Google Maps should become the primary delivery point for the Ordnance Survey’s services”

    I question the benefit of making reference to the OpenSpace and Google Maps API specifically. There are technical limitations to such API’s, which could restrict their suitability, as a portal to what is essentially a vast dataset.

  18. Rob Dunfey says:

    A commentator makes reference to making data available with a Creative Commons BY-SA license.

    I’m not a lawyer, nor very familiar with data licenses, so please excuse me if I’m wide of the mark, but the Share Alike clause of such a license strikes me as very restrictive and would essential prohibit innovation by private enterprise?

  19. Steve Maller says:

    Yell would be able to innovate, collaborate and compete much more effectively with a different licensing arrangement for OS data. We feel the OS approach is not good for the competitiveness of the UK economy, in particular we lose further ground to the US.
    I write as the Chief Technology Officer of Yell.com

  20. Sebastian Crump says:

    I’m not sure it’s wise to ’speculate’ – perhaps better to provide examples, even if ad hoc evidence (1st para)

    Overall this section seems rather like a ’soap box’ address and seems to repeat itself too.

    As others have commented it seems to concentrate solely on Ordnance Survey, possible unfairly – how about other location indicators, such as PostCodes, telephone number area codes, etc.

    It seems to be missing the Recommendation subtitle.

  21. Great to see this issue being tackled so comprehensively. I’d like to add a bit of special pleading – it would be great to harmonise the availability and cost of mapping data usage across the UK i.e. include OS Northern Ireland. Because we have a separate OS, many decent web applications (like FixMyStreet) do not work here. It’s about time that was fixed IMHO.
    Darryl

  22. Gary Fenton says:

    Making basic data available (such as electoral boundaries) would be a god-send for many services that aim to benefit the public.

    I would not like to access data that OS may make freely available via their OpenSpace API. It could never be as good as using Google Maps or MS Virtual Earth and most web apps already support these more advanced mapping services and would cost unnecessary money to redevelop them for OpenSpace. To make future “openly available” OS data truly open it needs to provide access to the *RAW* data so it can be used with apps that already use Google/MS maps. We need the data itself, not images and not an API. Using real boundary data web apps can become smarter and actually become aware of boundaries computed at the application/database level.

    OS boundary and public building data can go straight to the client from the application/web server, it does not have to go via Google/MS servers and OS need to better understand how it works and produce some sensible T&Cs.

    Making Royal Mail PAF & Postzon data freely available to benefit public services would be a great achievement too. e.g. a web service for the public with the backing of a council requires the use of PAF data but Royal Mail charge a separate license for it (very expensive) despite the fact that the council collect & supply the originating address data for the Royal Mail. Where is the logic and fairness in that? The result is the tax payers have to pay twice, again.

    British innovation and to some extent the economy are being held back by restrictive licensing and double charging tax payers and councils. Let’s move forward as quickly as possible.

  23. Lee Bryant says:

    Good stuff – much needed

  24. For me, this section and those that follow it – where the focus is on what information to free up, where and how – are tghe strongest in the report.

    They clearly fit the remit of the Taskforce and it shows in the quality of the research and the recommendations.

  25. Phil McAllister says:

    From previous public sector mapping involvement I would offer up for thought the following points from research that have stuck with me:

    * Within the wider population, there is still relatively little understanding of how online maps work. The interactivity is still not appreciated by all.

    * Design for the map to be the destination not the journey. Users generally do not want to go on a ‘treasure hunt’ to find the content (they may not even understand how to, point above). Using maps as a navigation tool to find content can be dangerous and should be carefully considered.

    * Relate the content to a user’s reference frame; find out where they are from and relate all content to that reference point. I.e. keep the name of the user’s town or village always plotted on the map as a frame of reference.

    * Do not limit available technologies to OS mapping vectors and tiles. There are occasions when different mapping solutions may be more appropriate. Perhaps rather than recommend a single source of mapping data look at recommending a mapping framework that allows for easier interchange of mapping data and APIs to suit purpose, e.g. OpenLayers

  26. [...] new developments in digital media and the use of state generated information. Their coverage of Trading Funds, and their suitability as a model for running organisations such as Ordnance Survey is a worthy [...]

  27. Pete Walker says:

    The Ordnance Survey monopoly and charging mechanisms has certainly curtailed things we would like to do.

  28. Alan Cox says:

    One thing that needs to be understood is that google maps is effectively beginning to replace the official government data sources for mapping even within government. That means the state will lose control of UK mapping to a US corporation focussed on profit making not the interests of the UK.

    At that point the Ordnance Survey becomes a historical oddity and rapidly hits near zero value. The current model will become unsustainable as the users of its data plummet.

  29. Alan Cox says:

    One other culprit here should be mentioned – the Met Office. They’ve at least tried to do some interesting things but having tried to integrate their forecasts and warnings into a piece of free software some time ago I gave up because the data available is not easily available in an importable form and the licensing and usage regulation was impossible to understand.

  30. Barry Tennison says:

    I agree strongly with this section about geospatial data.
    One example from my previous work as a Director of Public Health, concerning boundary data.
    I was astonished at how difficult it was (albeit some time ago) to obtain accurate up-to-date information on the EXACT boundaries of health authorities and local authorities. These are vital, not least to ensure that it’s clear which authority has to take responsibility for certain events and duties, and so to make sure that nothing “falls down the cracks”. These days, such boundary information should be freely available, at very least to public bodies but arguably to all citizens, as we can do so much more with it now (and even more in future).

  31. Barry Tennison says:

    From my professional field of public health, here’s another example supporting the comments of Christopher Roper and Harry Metcalfe on Feb 4. This is about small areas and boundaries. It may not be realised by all, but the astonishingly wide use of postcode as a geographical area/location identifier arose gradually because of its ready availability and relative freedom for use. But it is far from ideal for many purposes for which it is used, for example for mapping mortality or disease prevalence, since it is squarely and understandably based on Royal Mail business needs (locations of sorting offices, postpersons walks). So, postcode:
    * does not reflect natural and key adminstrative areas
    * does not aggregate well (for example, areas based on first-few-chars vary hugely in size and do not fit with natural groupings – examples are easy to find)
    * does not meet many of the needs for which it is the sole feasible current candidate, for example in my knowledge areas of disease patterns in space and time.
    What is needed is a set of small areas (similar to postcode) whose boundaries and geography are freely and readily available; which aggregate well; which fit with key data sources like census and other ONS data; and which will survive over long periods in their suitability (for example as the human geography changes). Some people have tried to use census enumeration districts for these purposes, but in my view we need a more fundamental study and then, crucially, the widespread nurturing and adoption of these basic building blocks by public sector bodies (at least).

  32. Ed Parsons says:

    It is only natural that much focus of this section is on the activities of Ordnance Survey, but as others have pointed out elsewhere in comments to other sections, the licensing and commercial exploitation of the Postcode Address File (PAF) by the Royal Mail has at least similar, if not a greater, impacts on ability of organisations and individuals to make use of government information. The fact that a website developer must pay every time the location of a postcode is looked up has a massive impact of the financial viability of many citizen focuses services which could be developed. This is yet another example of a licensing regime which actually penalises success.

    As Anthony Cartmell quite rightly points out a fundamental issue with Ordnance Survey licensing is their interpretation as to what is “Derived Data”. This can end up as a rather technical arguments, but currently if a OS data set is used in any part of the creation of new geographic information, that new information inherits the intellectual property owned by the OS and therefore must be licensed as OS data itself. So even if the new feature (such as the location of a recycling centre ) does not appear in any OS product, the OS can and will claim IP rights over it. Imagine the outrage if Microsoft claimed a share of the proceeds of the latest Booker prize winner because the author used Microsoft Word.

    As Christopher Roper and Harry Metcalfe point out perhaps ultimately in the long term a sustainable mapping of the UK may only come from fundamental structural changes. In an number of other European countries the activities of Mapping agencies in maintaining large scale detailed databases of properties (the expensive part of the OS operation) have been combined with the relevant Land Registry body, the primary customer for such information, the funding of the mapping activities coming solely from Land Registration fees.

    This remaining relatively “cheap” topographic mapping activities are then possible to continue funded from taxes, allowing the open licensing of less detailed mapping to who even needs it.

  33. I would like to see more explicit recommendations with reference to ‘derived’ data, i.e. information, particularly geographic, that has been created by reference to another source of information. Specifically, there have been instances with regard to information, such as ward boundaries, that has been derived through reference to an Ordnance Survey map. In these cases I understand that OS retain some intellectual property in the resultant information. Specifically, there are then restrictions as to your rights to display this information overlaid on a Google Map, using Google’s public API, despite their being no apparent disclosure of any OS mapping intellectual property. My understanding is that the terms that restrict this for Google’s public API do not exist in their commercial terms.

  34. Denis Payne says:

    OpenSpace API – I’m unclear as to why this comes up as a recommendation – this is not a data delivery service, and we cannot expect OS to invest in adequate tools and technology to offer the performance that Google/Microsoft do. For “OS to invest” read “UK plc to spend”

    If what is required is the delivery of data, why re-invent the wheel – why not just give it to Google/Microsoft?

    However, simple, effective/efficient access to current data via web service of some nature is certainly the way to go.

  35. Andy Mabbett says:

    Some consideration of/ reference to OpenStreetMap should be made. Google/Microsoft are not the only alternatives to OS.

  36. Later in the report you mention the “Access Layer” and access to raw data. The OpenSpace API allows access to OS web mapping, not raw data, something completely different.

  37. Jeni Tennison is absolutely correct, the National Street Gazetteer and National Land and Property Gazetteer are datasets with massive potential value.

    At present they are under-utilised as they suffer from even more restrictive licensing than OS data

  38. Define “general mapping and address data”.

  39. Yes.

    It is essential for local authorities and partners to be able to present data about public services spatially to the public and for this data to be reused by the public as part of co-creation, consultation and service design.