- You are here:
- Home
- › Stephen Hale
Stephen Hale
Head of Engagement, Digital Diplomacy, LondonDiplodocus - who's there, and what are they doing?
People are using Diplodocus to talk about location based apps, how to centralise/devolve global web presences, social media guidance, copywriting, diplomacy hashtags, using social media during a crisis, and public diplomacy.
And we have members from the US, Israel, Belarus, Canada, Malaysia, France, Estonia, Bahrain, Denmark, Norway, Australia, the UN, the EU, the UK and Finland.
It's a steady start. It was always going to be a bit of a slow burner, but I'm already finding it useful. As more people join and begin to contribute, the more useful it becomes.
One of the first things we used the site for was to talk about the community itself. We even drafted some performance measures so that we can judge how we're doing. I've copied them below - you can hold us to account in 5 months time.
If you're not already a member, sign up now: Diplodocus: a community for international government people
1. Objective: Diplodocus has members from around the world
- KPI: number of different countries represented
- Target: 25 different countries represented within 6 months.
2. Objective: Virtual community leads to physical benefits
- KPI: number of physical meetings between members of Diplodocus
- Target: 10 physical meetings between members of Diplodocus within 6 months (measured using survey)
3. Objective: Diplodocus leads to change in behaviour
- KPI: number of times members of Diplodocus have changed their behaviour/approach as a result of using Diplodocus.
- Target: 10 members report a change in approach as a result of Diplodocus (measured using survey)
4. Objective: Diplodocus is a genuinely interactive forum for conversations
- KPI: Members contribute to discussions
- Target 1 : 80% of member make at least 1 contribution (other than signing up) within 6 months
- Target 2: 10% of members initiate a discussion within 6 months
- Target 2: Discussions receive an average of more than 5 responses
Stephen Hale
Share this with:
Evaluating our blogs
Foreign Office bloggers should focus on making sure that their blogs are integrated, personal, real-time, and 2-way. These are the headline findings of our detailed evaluation of the impact and reach of our blogs.
I mentioned a while ago that we were doing some work to evaluate the impact and reach of Foreign Office blogs.
Our aim was to get beyond our regular stats reports to provide a deeper analysis of the reach of our blogs and the impact they have on the people that read them.
Some of our findings were expected, some were unexpected. Some of what we found was an endorsement of what our bloggers were already doing. But there are clearly things that our bloggers could do to improve the reach and impact of their blogs.
We carried out the work internally (much credit to Rob and Shane), but we made every effort to just report what the data told us, and in no way game the results.
Here's a summary of what we found:
What we did
1. A survey
All our English language bloggers posted a blog entry asking their readers to complete a quick survey. We asked readers a few questions about themself and their blog reading habits. And we asked them to rate a set of extracts from Foreign Office blogs according to how informative, balanced, trustworthy, authentic, appropriate and interesting they found them to be.
2. Metrics
We took a good look at the data we already had, concentrating on traffic, inbound links, and the number and sentiment of comments.
3. Competitor analysis
We wanted to learn from what others were doing. So we thought a bit laterally about who we might regard as competitors or peers of FCO bloggers (Carl Bildt, Steve Clemmons, Norman Geras, Moby...) and compared what they were doing with what our bloggers were doing.
Some conclusions
Some of our findings are particular to individual bloggers. But there were some pretty clear themes in the data:
1. Blogs are 1 tool among many
Our most effective blogs are combined with wider communication and policy activity.
For example, the second biggest peek in traffic to the Foreign Secretary's blog last year was when he posted Forging coalitions with the Muslim world. That blog was complementary to a speech he gave at on the same subject, was combined with outreach to bloggers, and was promoted via Twitter - including real time coverage of the speech and press conference. As a result it generated mentions and inward links from other bloggers and leading websites, increasing the potential influence of the blog entry exponentially.
We found that that this kind of behaviour is common among the peers of FCO bloggers too, who might use Twitter to confirm hunches or check facts with their followers while they are writing blogs, or use physical events or conventional media to promote blog entries or continue the conversation.
Where we found this kind of multi-channel behaviour, we found that it was reflected by both increased traffic, and increased volume of comments. It seems that successful bloggers are rarely just bloggers, and successful blog entries rarely exist in isolation from other channels.
2. Personal insights and opinions make for interesting blogs
People expect, like and respond to blogs that only the author could write. In particular, they respond to personal insights. And readers comment on our blogs - and rate them highly and return to them - when they agree or disagree with the expressed opinion.
Our survey asked readers open questions about what they liked and disliked about FCO blogs. The strongest theme from the "likes" was around personal insights. And the most common answer mentioned insight into the opinion of the writer. When we asked them what they disliked, readers said they disliked blogs were bland or dull, or that didn't offer enough of the personality of the writer.
What do you like about FCO blogs?
What do you dislike about FCO blogs?
The blog entry that was rated as being the most authentic was $50 shopping spree by Grace Mutandwa, which uses personal anecdote to make wider points. Blogs that just reported events or described facts were rated as being less authentic.
The blog entry that was rated as most interesting was Marwa el Sherbini by Dominic Asquith, in which he expresses a clear personal opinion. In fact, this entry (published at the time in Arabic and English) was the best rated of all the entries we tested across our metrics of informative, balanced, trustworthy, authentic, appropriate and interesting.
3. Readers want comment in real time
A significant number of people read our blogs every day. We know that stories move on very quickly online. As do as do the opportunities for our bloggers to engage their readers.
More than 20% of readers of FCO blogs say that they read them every day. This is the kind of behaviour we might have expected from readers of technology blogs, where the majority of readers might subscribe to an RSS feed. I didn't expect the number to be so high for our foreign policy blogs, where we know that the number of people subscribing via RSS is a much smaller proportion of the audience. There is clearly appetite for real time content.
When we looked at our peers, we found that there was a clear correlation between real-time blogging, and reader engagement (if we take the number of comments received as an indication of engagement). Those who blog as things happen also receive the most comments. For example, Carl Bildt posted 53 entries to his blog in July - often offering comment on events before they happened - and received more than 400 comments.
4. Readers want conversations
Readers expect our blogs to be 2-way communication. And blogs that reach into specific communities of interest generate the most visible engagement (comments, and inward links).
Our survey results suggest that our blogs are stimulating real engagement. Our readers are having conversations as a consequence of reading our blogs. In particular, 66% of readers of FCO blogs said that they had discussed our blog entries offline. This appears to be an endorsement of the choice of blogging as a medium for our diplomats: by blogging they are stimulating conversations, on and offline.
We also asked readers what they expected in response to comments posted on our blogs. 34% of readers said that they didn't expect a response at all. Of those that did expect a response, 29% expected the author to respond personally, and 37% said that they expected a representative of the FCO to respond, although not necessarily the author.
We can see from the metrics that the blogs that are most commented-on are entries in which the author addresses issues that are already the subject of online debate. The most commented-on blogs from the Foreign Secretary last year were those about Sri Lanka, and we can see similar peaks when he blogged about Georgia, and Iran.
Traffic to the Foreign Secretary's blog, overlaid with volume and sentiment of comments
Some practical recommendations
We've turned these themes into some practical recommendations for our bloggers. Our advice isn't the same for every blogger, but we are using our findings to encourage our bloggers to be personal (say things that only they could say), real time (if it takes days to draft or check the facts then it's probably not a blog), integrated (with other things they're doing on and offline), responsive (responding to comments), and targeted (writing about things that people are already talking about online).
Stephen Hale
Share this with:
More on London Summit evaluation
I've blogged before about evaluation and our London Summit digital campaign. We've now published the detail: Evaluation of key performance indicators.
It's a full evaluation of the performance indicators we set for the London Summit website back in January. It uses survey responses and metrics from our own web platform and Google Trends to test how we performed against our own objectives.
The highlights:
- 4 objectives
- 13 KPIs
- 3 were fully met
- 4 were partially met
- 4 were not met
- 2 could not be assessed with the data we have
Details in the report.
Stephen Hale
Share this with:
London Summit digital campaign evaluation
So our website is now a historical record - the debate has moved on. There's still plenty to do of course, and for many the summit marks the start of a new debate. But our digital campaign is over, and we can take a step back and assess how we did. Did it deliver what we intended? And what lessons can we learn and share?
We'll publish all our evaluation on the London Summit website itself, including all the numbers, measurement against all our KPIs, and the independent evaluation of others. But it's been a couple of weeks now, and I think I have enough distance and data to reflect on what we did.
We started this work with a degree of confidence. We've been thinking about - and practicing - how to do digital diplomacy in the Foreign Office for a while now, and as a result we were pretty confident about our methods in January. The lead up to the summit seemed like the ideal opportunity for digital engagement. I published here the performance indicators we set for ourselves at the start of the process, but if I'm honest, I really didn't know whether we'd deliver them all, or how much of an impact our digital campaign would have.
So did we succeed? Well, I'm not sure yet.
There are bits that I think we did well. I think we made good judgments about the ambition for the London Summit website, and the tone of the content that appeared there. We did well to ensure that the site aggregated content from around the world, and provided space for a range of voices. Some of our partnerships with other web platforms and forums worked really well. We produced a lot of video and photo content, which helped bring the debate to life. We provided good authoritative content on the purpose of the summit, an efficient service for journalists and accreditation, live streamed video throughout the summit, and lots of structured ways for people to contribute to the debate.
We've learned a lot about how to deliver intensive digital campaigns. We had a great team working on the campaign, and we relied on the expertise of others across government, the FCO network and our partners. We published new content many times a day which enabled us focus our work on the narrative of the debate as it emerged.
We also learned a lot about the FCO web platform, about delivering content to a global audience with a huge concentrated peak of traffic. And we learned a lot about citizen engagement, how and when to encourage debate, and when to just take a step back and reflect what's going on elsewhere.
There were plenty of things that didn't work. As you'll see from the performance indicators below, there are some things that we set out to deliver, but just didn't achieve. Some of our ideas were never realised and some of our partnerships just didn't come off. We hosted and commissioned a lot of debate, and we listened to what others were saying online, but we didn't often actively participate ourselves other than in delivering messages, aggregating, or summing up.
Our website was a destination for authoritative content and specialist debate, but of course other websites provided alternative places for the popular global debate to take place online. And our success in encouraging debates in some countries, wasn't matched with all our target audiences.
Of course, this is all just subjective, and I'm possibly not in the best place to judge. So we'll publish all the bits of evaluation that we and others produce on the now frozen summit site, and I'll highlight the best bits here.
But I promised to report on our performance against the original KPIs. So here you are (we'll publish the full version of this on the summit site):
1. The focal point for engaging and shaping global opinions
- http://www.londonsummit.gov.uk most authoritative site for "London Summit" according to major search engines.
Met. Our content was well optimised for search so we were quickly at the top of natural search rankings for our key terms. Having said that, we found that other terms were far more widely used that some of the initial key terms that we identified.
- influential sites in every target country link back and quote from http://www.londonsummit.gov.uk
Partially met. Of the 23 countries targeted, influential sites (those with a Google page rank of 6 or above) from 12 countries linked to the London Summit site.
- all visitors find it quick and easy to find the info they are looking for
Our user survey will provide more data here.
2. Authoritative provision of in-depth briefings on Summit
- all unclassified policy papers accessible from londonsummit.go.uk in web friendly form
Met. We published everything we had, including communiques, transcripts, summaries, video. And we didn't resort to PDFs.
- only the highest-quality content goes on the site based on the content guidelines. If it doesn't help to achieve an objective is doesn't go on
Partially met. We wrote good copy, kept to our content guidelines, and had a clear process for clearing policy-sensitive content. But I know we sometimes published in a hurry, and corrected later.
- at least four expert bloggers providing authoritative real time content for London Summit
Not met. We ran an editors' blog, Tom Watson blogged from the summit, Stephen Timms posted tweets (#timms), the Foreign Secretary posted blogs, and government economists posts articles on the Vox EU debate. But we didn't have specific attributed expert bloggers contributing to the online debate throughout the campaign.
3. Effective operational functions for 2,000 journalists
- Media centre regarded by journalists as most respected government media site ever
I think we did ok, but our survey of users will tell us more.
- live streaming of all press conferences/keynote speeches
Partially met. We streamed the whole summit. We didn't live stream any other events, but we published a lot of same-day video thanks largely to our partnership with British Satellite News.
- the site is reliable (minimal down time) and meets AA accessibility at all times
Partially met. The site had 99.82% availability, the page templates all met AA standards and have been tested with real users for accessibility, and our key content was always provided in accessible formats. But our content did not always validate as AA accessible (for example, we did not caption, or provide text alternatives for all our content, and when were faced with publishing a video without a transcript, or not publishing it at all, we published the video).
4. Respected Platform for discussion and debate
- seamless integration with all partner engagement sites
Not met. Our referral stats show that user journeys between the parts of our web presence were not common. So although we did some cross promotion through RSS feeds and promos we didn't always deliver reciprocal links, so our users could not move seamlessly between our partner sites.
- clear evidence of link between pre-summit web debate and post-summit outcomes
Met. Our debate issues reflected many of the outcomes. It is less easy to measure whether the online debate influenced - or just reflected - the outcomes, but I hope our detailed evaluation (and the evaluation of others) will provide some data here.
- visitors return to the site, go to other areas of our London Summit web presence or subscribe to feeds/emails
Partially met. 27% of visitors to the site were repeat visitors, and 2,273 people subscribed to receive our email newsletters, but there wasn't significant traffic between areas of the summit web presence.
- the site (and related wider web-presence) becomes a best-in-class example of digital engagement
For others to decide
And some slected numbers (because however much I protest that evaluation isn't just about numbers, I know I can't get away with not mentioning them at all):
London Summit website (29 January - 6 April): 466,159 visits, 1,572,643 page views
London Summit YouTube: 149,580 video views
Live video streaming: 165,000 views on 2 April
London Summit Flickr: 1,065,825 photo views
VoxEU Global Crisis debate (ongoing): 160 articles, 300,000 page views
Yoosk London Summit: 327 questions, 36 answers from public figures
Chosun debate (Korean London Summit forum): 231,832 unique visitors (by 2 April)
I'll post more on the digital engagement, and the other evaluation as it's published.
Stephen Hale
Share this with:
Proving the value of digital campaigns (and why Twitter won't take off in Ukraine)
I met a delegation of Ukrainian government officials the other day to talk about digital diplomacy. It's one of the perks of my job that people outside the UK are interested in what we do. They provide a challenge that I don't necessarily get from my peers in the UK digital community. (Who else is going to tell me that Twitter won't work as a tool in Ukraine - because you only get about 3 Ukrainian words for 140 characters?)
We talked about the online campaigns that we've run recently in the Foreign Office, the way we manage and present web content, and some of the tools we've been using for digital engagement. I think I surprised them (and myself) by how excited I got when they asked how we evaluate our work in Digital Diplomacy Group. But the fact is I am very excited about proving that digital engagement works. And more than that: I think we have a responsibility to measure the actual impact of digital campaigns, rather than get carried away with the ease with which we can develop new tools.
Of course, web practitioners are notoriously lazy about evaluation because everything we do on the web produces numbers. Stats are almost always interesting, and it's easy to present them as evaluation. But they're not enough. The Foreign Office web platform had 2.5 million unique visitors in January. But so what? I know that I could significantly drive up traffic to the Foreign Office YouTube channel by posting a film of 150 ambassadors line dancing (I'm sure they'd be up for it). But traffic doesn't deliver foreign policy objectives. It just delivers traffic.
Our approach to evaluation was developed by Liam King, who is even more excited than I am about evaluation. It's not complicated - this is what we aim to do:
1. Insist on setting objectives and identifying target audiences for everything we do on the web.
2. Pick something that we can measure that will give us an indication of how well we met our objectives and and reached our target audience.
3. Measure it.
We do use stats, and we welcome independent evaluation (the Hansard Society are evaluating our blogs and our London Summit campaign at the moment), but we concentrate on providing evidence that tells us something about what we set out to achieve. This approach means that all the evaluation we do is useful for the people we're working with (because we are very clear about expectations right at the start), and it's useful for us (because we can use it to improve what we do).
I've pasted below the objectives and performance indicators that Liam and Paul set Digital Diplomacy Group in January for our work on the London Summit website. Our approach will develop, and we'll measure KPIs for each of our engagement exercises over the next 6 weeks. But the original performance indicators won't change - once the summit is over we'll be able to say with authority whether we delivered what we set out to.
London Summit website objectives and performance indicators:
1. The focal point for engaging and shaping global opinions
- http://www.londonsummit.gov.uk most authoritative site for "London Summit" according to major search engines.
- influential sites in every target country link back and quote from http://www.londonsummit.gov.uk
- all visitors find it quick and easy to find the info they are looking for
2. Authoritative provision of in-depth briefings on Summit
- all unclassified policy papers accessible from londonsummit.go.uk in web friendly form
- only the highest-quality content goes on the site based on the content guidelines. If it doesn't help to achieve an objective is doesn't go on
- at least four expert bloggers providing authoritative real time content for London Summit
3. Effective operational functions for 2,000 journalists
- Media centre regarded by journalists as most respected government media site ever
- live streaming of all press conferences/keynote speeches
- the site is reliable (minimal down time) and meets AA accessibility at all times
4. Respected Platform for discussion and debate
- seamless integration with all partner engagement sites
- clear evidence of link between pre-summit web debate and post-summit outcomes
- visitors return to the site, go to other areas of our London Summit web presence or subscribe to feeds/emails
- the site (and related wider web-presence) becomes a best-in-class example of digital engagement
I'll report back on how we did against these in April.
Stephen Hale
