The MESH Confidence Assessment Scheme 
To help users assess the confidence of a map, the MESH Project developed a systematic approach using a multi-criteria questionnaire. The MESH Project collated many maps produced for different purposes, ranging over many years, employing many different techniques and from a variety of sources. In many areas the maps overlap and users need to know the relative confidence they should have in these maps. The problem of map confidence is multifaceted and any assessment runs the risk of being very subjective and dependent on the person undertaking the exercise. Clearly, if some comparison is to be made between maps, then the assessment should be as objective as possible.
The MESH Partners decided that a confidence assessment system should be devised and the confidence factors stored as new metadata elements so that they are accessible together with the discovery metadata describing the map. The metadata already compiled by the project lacked sufficient detail to make objective decisions about the confidence of various factors. The purpose of the MESH Confidence Assessment was to visualise an overall confidence score on the MESH webGIS using study outlines, and to link these outlines to the full set of scores so that the assessment process remains transparent. The overall scores allow rough comparisons to be made between maps whereas the full set of scores enables users to identify why one map may have scored more highly than another. A scoring system based on a multi-criteria approach also allows survey planners to anticipate the effect of changing various survey parameters on the overall performance of a survey. In other words, it may help as a planning tool. Similarly, some criteria will also be useful to cartographers to determine which factors will increase or decrease the confidence a user may place in a map.
The MESH Confidence Assessment scheme is a compromise between being comprehensive and being easy to understand and use. Many criteria have undoubtedly been left out and the exact scores suggested for each map may be challenged. The system is not designed to identify subtle differences between maps, but rather to give a simple and robust assessment. The exact score for any one field could be debated, but the overall score is little affected by tweaking the individual scores for the fields. Although the way in which a multi-criteria scoring approach is designed and operates is open to criticism, at least it is also open to inspection because the decision points are established and guidance is given to standardise scoring carried out by different individuals.
The MESH Confidence Assessment scheme has been built into two applications, each of which is best suited to a particular type of confidence assessment. For assessing multiple maps, we suggest using the MESH Confidence Scoresheet (a MS Excel Spreadsheet) for ease of data entry and comparison between maps. For a more interactive tool which is best suited to the assessment of a single map, use the MESH Confidence Tool which was built as a Flash application. This interactive tool allows the user to see the effect of changing individual scores and weightings on the overall score. For those interested in the methodology but who will not be carrying out confidence assessments, the factors and scoring system are set out in the MESH Confidence Assessment Guidelines. The scoring guidelines are built into each application for quick reference.
The MESH Confidence Assessment Tool, built as a Flash application. The scores and weightings of the factors are selected in the left-hand pane and the scoring guidelines are in the right-hand pane.


Links to resources:

MESH Confidence Assessment Guidelines

MESH Confidence Scoresheet

MESH Confidence Assessment Tool

All material variously copyrighted by MESH project partners 2004-2010

| List Access Keys | Mapping European Seabed Habitats (MESH) - Home | Overview | News | Work areas in the MESH Project | Partners | Contact Us | Search | Site Map | Partner Extranet |