Listening to Twitter- some key tools for your NGO or Council

Serendipity is not uncommon in social media. People find new and unexpected ways of using social media tools all the time. Of course, the same thing happens with PC-based software, but the web allows people to develop new uses for social media and then to spread these applications far more cheaply and effectively.

Twitter is a case in point. The 140-word microblogging service is still seen by many as an exercise in narcissism but at a deeper and more serious level it have found uses in areas as diverse as providing feedback on brands and products, distributing alerts for upcoming events, giving real-time responses to speakers in current affairs programs and even spreading information among activists demonstrating against dictatorial governments.

Both fed and been supported by the growth of a small forest of add-ons, Twitter is rapidly becoming a mainstream communication channel, which means it needs to be considered as part of the social media strategy for all NGOs, councils and other small organisations.

This presupposes of course that you have a social media strategy. The reasons for NGOs and councils to have a strategy and what it should do have been written about in numerous blogs, but the simplest argument is the most compelling – the hundreds of millions of people around the world who use various forms of social media on a daily basis.

What your social media strategy should do obviously will depend on your organisation’s objectives, projects, activities and users (both staff and customers/clients), but the basics are summed up in this post by Spurspectives. I’ve expanded a little on the standard components identified in that post to form this list:

• Adding a blog to your website (of course you can have multiple blogs, but don’t underestimate the demands of keeping even one up to date);
• Setting up a page for your organisation on Facebook;
• Sharing videos (for example through YouTube) and posting podcasts on your website about your organisation’s policies, projects and events;
• Connecting with colleagues and community leaders through LinkedIn and then joining LinkedIn networks relevant to your organisation;
• Using Twitter to post current news and updates about your organisation. These can include links to your blogs, videos or podcasts. You will need to decide who exactly has authority to post tweets on behalf of your organisation and what they will tweet about.

This list is a good start, but one key element of a good strategy is that you should not just see social media only as a one-way street to disseminate information about your organisation. Social media is about a conversation, or rather thousands of many-to-many conversations, and your strategy should be more about how to engage through these conversations with your users and those you are trying to influence.

This means that you need to listen to your target audience as well as broadcast to them. All of the social media components suggested above provide opportunities for feedback, but this is one area where Twitter shines, given that anybody with an account can respond instantly and publicly.

I don’t intend to provide a general intro to Twitter here – this has been well-canvassed in guides like the one on Victorian State Government eGov website or this local government-related blog from the UK. Instead I’d like to look specifically at how to receive and handle the rich stream of responses that Twitter provides.

Twitter is like a fast-moving river, with thousands of tweets posted every second; while you can use Twitter to search for your organisation, this is difficult if it has a long name. To help you can start using a hashtag for your organisation or its services in your own tweets. A hashtag is a short, distinctive is name preceded by the hash key (for example, #wikileaks or #cityrail) which used in tweets. If the hashtag catches on and other people start using it, searching for tweets relating to your organisation will be much easier.

Once you do this you can simply dip your hand into the “twitterstream” in every now and then to see what people are saying about you by using Twitter to search for your hashtag, or a filtering tool such as Tweetdeck, which allows you to display searches for multiple topics. You could then provide these as public feedback on your website by using applications such as Twitter widgets or Twitterfeed.

However, while Twitter and Tweetdeck can yield some interesting posts in real time, they don’t allow you to store the results for later use. For a more systematic approach you need software that allows you to filter and then extract the resulting list of tweets to examine outside of Twitter.

Tweetdoc allows you to enter a hashtag or other search term and then set date and time range and limits on the number of tweets to display. The resulting list is displayed in a PDF file which can be stored offline for later reference.

A similar service is provided by SearchHash. As the name suggests this concentrates on filtering by hashtags (though it does seem to work with other search terms) but is a bit more flexible than Tweetdoc in allowing export to an Excel document, which is easier to use as a basis for further research. Meanwhile other tools such as the Archivist provide a snapshot of statistics such as the number of tweets over time, the  top users, main sources, etc.

Be aware, however, that this is a fast-changing area. Search applications such as Searchtastic which were widely recommended only a few  months ago have already ceased operations. Others such as TwapperKeeper have removed their export facility, claiming that Twitter has alleged that it infringes its Terms of Service.

Setting up a Twitter account and then searching for your organisation’s hashtags or other topics related to your services is an important start, but you can do so much more with Twitter. In a future post I’ll look at how you can use it to help in running conferences and forums.

Cloud storage and privacy – the dark side of the silver lining

A report of the Senate Environment and Communications Committee released this month may have profound implications for small businesses, councils and non-government organisations that use cloud storage for personal information relating to their clients or customers.

The Committee’s report, titled The adequacy of protections for the privacy of Australians online was prepared in response to recent significant advances in online technology and its use for social media and cloud computing. The committee was particularly interested in the impacts on personal privacy and the protection of data.

The report contains nine recommendations. Key points include:

• Small businesses which hold substantial quantities of personal information, or which transfer personal information offshore, should be subject to the requirements of the Privacy Act 1988.
• All Australian organisations which transfer personal information overseas, including small businesses, must ensure that this information is given the same protection as that provided under Australia’s privacy framework.
• If an organisation overseas collects information from Australia, this information should be protected by the Privacy Act.
• All Australian organisations that transfer personal information offshore are fully accountable for protecting the privacy of that information.

The Privacy Minister Brendan O’Connor has highlighted in an interview with News Ltd that cloud storage was a particular concern.

“While some ‘cloud’ providers are located here in Australia, many more are located overseas,” he said.

“That gives rise to difficult jurisdictional issues, particularly where the laws of two or more countries could potentially apply.”

The Minister has indicated that the recommendations will be implemented.

“This is an important development that will prevent organisations from trying to avoid their obligations under the Act by transferring the handling of personal information to countries with lower privacy protection standards,” Mr O’Connor said.

This point is especially relevant as many countries do not offer the same degree of protection for personal data stored on servers in their jurisdictions against it being accessed by government agencies, or even shared with businesses.

If implemented, the recommendations will have obvious consequences for organisations large and small across all sectors that have made explicit decisions to save costs by moving client data to cloud storage. Even if they have received promises regarding the location of data storage facilities and their security, these assurances are very difficult to check.

However, these are not the only organisations that may be caught up if the recommendations are implemented. The buzz around cloud computing and storage – not to mention the potential cost savings - has encouraged many small organisations to experiment with online applications for their day-to-day computing, such as Google Docs, which rely on cloud storage. The files generated may include a range of personal data which could be covered by the proposals.

In addition, many organisations that continue to use “traditional” PC-based software also rely on cloud storage applications such as Dropbox for data backup or portability. So even if an organisation’s primary storage of personal data is located in its Australian offices, additional copies may be kept on an overseas server and would therefore be caught by the recommendations. Furthermore, the current exemption from the requirements of the Privacy Act enjoyed by small organisations would be removed, even if they do not store data overseas.

While the Senate committee report offers sound arguments for the proposed changes and nobody would deny the fundamental right of people to expect high standards of privacy to protect their personal data, the government needs to work closely with all organisations including small businesses,  NGOs and government agencies as well as cloud storage and software providers to ensure that implementation of these recommendations is not too costly or onerous.

Smartphones – back to the linear – but mashed – future of social media

In my last post I discussed my recent acquisition of a new iPhone 4. It only takes a short while to realise the revolutionary implications of these devices for the future of social media and Web 2.

Facebook iPhone interface

Checking tweets or Facebook and LinkedIn updates is a no-brainer. The linear nature of most social media interfaces also means that there is comparatively little difference in the user’s experience between a smartphone and a PC, despite the latter’s vastly larger screen real estate – and of course you can add comments and updates anytime, anywhere.

The same, linear, slightly retro feel applies to switching from app to app and to opening multiple files within the same application – just like the early versions of Windows, or the first time you fired up Internet Explorer (obviously the experience is a little different on tablets such as the iPad because of their larger screens).

On the other hand, the ability of smartphones to integrate different applications with social media is probably their strongest feature next to their portability and for ease of use beats computers hands-down. The most obvious example is the act of taking a photo to email or to post to a social media website, both of which are accomplished in a few seconds and just two or three steps on most smartphones.

This is a major advantage in terms of convenience and time saving, compared to using a digital camera, uploading the picture to a computer and then sending it. The quality might not be as good and you may not have the range of tools available on a PC to manipulate the image, but for most day-to-day purposes, who cares?

This mashing of applications on the iPhone can be used in less obvious ways. Recently I had to scan a receipt to email as part of an expenses claim. Again, it was far easier and quicker to use one of the many iPhone scanning apps rather than crank up a high-quality but slow scanner and turn on the computer.

 

Dragon Dictate iPhone app interface (recently released in an Australian accent version)

OK, the iPhone’s scan image wasn’t as great as the scanner could have done, but I didn’t need OCR quality and it was certainly good enough for my purposes. The same triumph of convenience over quality applies to a host of other things involving applications and social media which are easy to accomplish on the iPhone, from emailing voice recordings to posting movies to YouTube, or using the Dragon Dictation app to compose your next Facebook post.

 

 

As a result, I think we are at the beginning of the next wave of innovation in social media and the web generally – and as smartphones become ubiquitous and are increasingly the weapon of choice for going online, the potential for all sorts of combinations involving apps and social media seems almost limitless.

Web scraping – the Jekyll and Hyde of Web 2.0 (part 1)?

Two media reports this week have highlighted the way in which social media and web 2.0 applications can use “web scraping” in very different ways and for very different outcomes.

Web scraping is a specialised software-based process used to extract data from websites, where it is commonly displayed using HTML or similar mark-up languages. When displayed in this way, it is difficult for conventional software trying to “read” the data to make a distinction between relevant information and the surrounding “noise” of the formatting, a situation complicated by the fact that websites may display similar information using very different layouts.

After the data has been “scraped” from the relevant websites it is collated in a database or some other systematic framework and put to new purposes, possibly not ones that were envisaged by the original creators of the information. The most common examples are the websites which provide on-demand price information for a product selected by the user, based on real-time comparisons between the web pages of various online shops.

Web scraping can be viewed as a specific form of data scraping, or the extraction of data from the human-readable output of any computer, but for the purposes of these posts I will regard the two terms as synonymous.

First, a positive example of web scraping at work. The not-for-profit organisation OpenAustralia, which has already made a name for itself in making information about the federal parliament, parliamentary debates and individual MPs more easily accessible, has just released a new app for iPhones and Android smartphones. The app gives users the ability to locate neighbouring development proposals that may affect a property just by pointing thier mobile phones at the property.

Planningalerts phone app.

The app is an extension of OpenAustralia’s already-successful web-based planning alerts service. This allows users log the address of a property and be informed by email of development proposals within either a 200 metre, 800 metre or 2km radius, assuming the property is located within the boundaries of a council that provides this information online in a format that has been scraped by the software.

This is not without its complications. In an interview with the Sydney Morning Herald, OpenAustralia founder Matthew Landauer describes the process as “very painful and error prone”:

“The program clicks on links, fills out forms to do searches, and then when the program finds the web page with the development application it has to extract the unstructured information on the page and turn it into structured information.”

Only 85 of the over 650 councils in Australia are covered by the software, but these include many of the larger ones and others are being added to the system by “crowdsourcing”: members of the community adding the details of the websites of councils which start to place their DA information online.

www.planningalerts.org.au website

So far, so good. There would be widespread agreement that this is an appropriate “repurpose” of information which is publicly available anyway. Some councils may be nervous about the way in which the OpenAustralia phone and online applications raise the bar in terms of who gets notified about DAs, but hopefully most will take a positive attitude and cooperate with this initiative by making information available in a more standardised, software-readable format. In turn, this and other eGov initiatives may lead to councils and other levels of government making a wider range of data available in computer-accessible formats.

However, web scraping can be used in different and less benign ways. In my next post I will look at a new application which raises major questions about the use of web scraping and related techniques in relation to social media websites.

When Web 2.0 “goes bad” – part 2 – simple steps to help evaluate services and avoid problems

In my last post I looked at three examples of Web 2.0 services that had changed in unexpected ways and the impacts on their users. I also identified some of the associated issues with Web 2.0 applications, including data security, service closures or change in focus, the ability to extract data and unanticipated changes in fee structure.

Its time to look at some steps councils and small organisations can take in evaluating Web 2.0 applications they are interested in. But first, an extra disclaimer – the web is a volatile place and Web 2.0 applications are particularly subject to market forces and competitive pressures. In other words, there is no guarantee that even if you do thoroughly evaluate a service it won’t change in the future, possibly in ways that you don’t like.

So, what can your organisation do to help protect itself when looking at a Web 2.0 service?

1. Evaluate what the service does and how well it does it. Don’t be seduced by the hype – most Web 2.0 services have a free version which you should systematically evaluate (though these often involve a cut-down feature set). If you are seeking an online replacement for an existing application it should be easy to develop a checklist of your requirements; if the service promises something which you don’t already do, you still need to check that in fact it can deliver on this promise. Also check out how easy the service is to use, how quickly it loads and runs, what browsers it is happiest with and, if there are different service levels, what additional options they provide (see next point).
2. Confirm exactly what you are paying for. Beyond the limited free version most sites provide two or three service levels with increasing fees, usually paid for on a monthly basis. These service levels may be distinguished by different features and/or different limits on the number of users or concurrent projects. Make sure you are not paying for a higher service level than you actually need; if you require additional facilities for a once-off project, some applications will allow you to upgrade and downgrade on a month-to-month basis. Also find out how much notice you need to give to terminate a service – again, this can usually be done on a monthly basis.
3. Find out about storage, data security and backup systems. Check out if the service website provides information on how data is stored, whether there are adequate back-up systems and how secure your confidential data will be. If there is no information on the website, contact the service provider to request this information.
4. Clarify if you can extract data to backup and use outside the service. It’s important to sort out whether and how easily you can download this data and the formats in which it is provided, especially if you want to integrate information with other services or applications. How important this facility is depends on the nature of the service and the value of the data; for example, for online survey services you probably only need to be able to download the questions and the survey outcomes, whilst for online databases you would need to able to backup the whole database offline.
5. Setup your own backup systems.  Obviously you will also need to setup your own systems to backup downloaded data. If the Web 2.0 service you want to use is event related – for example, Poll Everywhere, which provides a mobile phone-based polling and voting facility for meetings – consider the possibility that the internet might be down on the very night you are holding your meeting and take along a manual voting system as a backup (I’ll review Poll Everywhere in a future post).
6. Assess the service’s maturity and popularity. Find out how long the service has been around and how often it has been updated – and whilst this may be harder, try to get a sense of whether it is attracting interest and users. Obviously an application that has been online for longer than a year or two,  has been updated a few times and appears to be popular is likely to be more stable and financially viable – though newer services are worth considering if they appear to be well-designed. Also be wary of older services that look like they have been neglected for a while and seem to have few users. These services could be struggling financially and the owners may already have plans to close them down.
7. Review service guarantees and support arrangements. See what if any promises are provided about continuity of service provision and carefully review the application’s user help and support arrangements. At a minimum, Web 2.0 service sites should provide a set of frequently asked questions and answers as well as an email address or form for support requests. Better sites will also provide product tours, webinars and/or tutorials on the service’s key features, knowledge bases, a list of tips and tricks, templates (if appropriate) and user forums.
8. Test the service provider’s response to support requests and feedback about product development. If you have any questions regarding the service after trialling it, ask them. Apart from getting an answer, this will also test the site provider’s responsiveness. Likewise, provide suggestions about the service and how it could be improved. Most service providers appreciate feedback and may indicate whether your suggestions are being considered for a future version.
9. Check out the views of users and reviewers. Some services will list prominent companies using the service and may even provide testimonials. Other Web 2.0 services may host user forums which can provide an insight to common problems. These may be moderated, however, so you should also search the web for references to the service in independent forums or blogs by either users or reviewers.

These are just a few suggestions for your council or organisation to consider in evaluating Web 2.0 services - please leave a comment if you have any additional ideas on how to review and choose web-based applications.

When Web 2.0 “goes bad” – part 1

What do a social bookmarking site, a web-based database and a social networking service mainly used in Vietnam have in common? The answer is that all of them have either recently changed, or are about to change, in ways that will disrupt thousands of their users – thus providing a timely lesson for everyone who depends on social networking or other web-based applications.

Services on the social bookmarking site, Ma.gnolia, stopped abruptly in January 2009 when its host computers suffered a database crash, irretrievably losing all the site’s data, according to Data Center Knowledge. The Ma.gnolia site, which was apparently hosted on two Mac OS X servers and four Mac minis, now carries an announcement that it will be reborn as a as a “by-invitation community bookmarking service”.

More recently the owners of Blist, one of the better-known online databases, announced that the site is morphing into Socrata, which will concentrate on “delivering social data discovery on government data sites around the world”. No doubt this has the potential to be a useful service, but the news must have come as a surprise to current Blist users who will apparently lose the original service in August.

Even bigger companies aren’t immune from these changes. Just ask the Vietnamese users of 360 Degrees, Yahoo’s social networking site. Whilst 360 Degrees failed to make an impact on Myspace and Facebook, it was an unexpected success in Vietnam, where in a country with strict government controls on information and dissent it became the dominant social networking service.

Yahoo announced that the service would close in 2008 but, according to the EarthTimes website, it is only now that the 13 July cut-off date is looming that many users are hurriedly trying to migrate their data to alternative sites or Yahoo’s Vietnam-only replacement service.

These three examples demonstrate why councils as well as public and private sector organisations need to exercise caution in choosing web-based applications to use in managing their operations or in providing services to residents, clients or customers. They also illustrate some of the potential issues that users need to consider in evaluating Web 2.0 services. These include:

• Data security: whilst the Ma,gnolia example of a catastrophic loss of data is thankfully rare, it does demonstrate that these things can happen. Potential users need to assess whether the services they are interested in provide adequate storage and backup facilities. More common concerns are whether and how online services maintain the confidentiality of the data that users or thier clients or customers store on their sites.
• Service closure or change in focus: again, whilst complete service closures are very rare, the Yahoo 360 Degree shut-down illustrates that they can happen. Even a change in focus such as that underway at the former home of Blist can amount to a service closure for dedicated users of the service being replaced.
• Service downtime: all websites can be affected by brief periods of downtime for essential maintenance or uploading software upgrades. These interruptions should be rare and advised to users well in advance.
• Ability to access data: users need to be able to access and download easily any data that they store on a Web 2.0 site, as well as contact information for their clients or customers.
• Upgrades and backwards compatibility: from time to time, all online services need to upgrade their services. These upgrades should be advertised well in advance, along with advice on whether users need to change any of their settings. Apart from the downtime involved, users will need to check whether there are any compatibility issues with new versions of software.
• Fee changes: users need to check out the different fee levels on web-based services – and how easy it is to terminate an agreement or change the level of service. They also need to be mindful that fees, service levels and conditions can be changed at any time by the provider.

Whilst these is an obvious need to proceed cautiously, these concerns should not be used as an excuse for councils and other organisations to not engage at all with social networking or other Web 2.0 applications. In my next post I will outline some practical safety checks for users to apply in evaluating these applications.

Councils, communities and Web 2.0

In my last post I suggested three recent posts on other blogs for further reading on the potential for Councils and NGOs to use social media and Web 2.0 applications. I selected these posts because I felt they provided particularly useful insights and I’d like to summarise and comment on some of the points they made.

Simon Wakeman points out that the advent of social media is only part of the changing communications and media environment that councils face, along with the decline of traditional media such as newspapers and the declining trust in institutions. This means that council staff will need to develop new skills to operate in this environment as well as have access to the new media tools.

Wakeman also believes that whilst councils are sensible in adopting a more centralised approach to their communications strategies with dedicated social media roles, they also need to review their policies and protocols and to adopt a more flexible approach to staff access to social media sites. Councils have an opportunity to put new policies in place whilst the volume of social media interactions is still relatively low.

Wakeman concludes:

“having social media explained properly helps people see the links between their service, social media and the community – and that can only be a powerful catalyst for councils to be more effective in delivering services for local residents.”

Bill Schrier takes this line a bit further. He points out that Government is by its nature “all about community”, bringing people together to do what they can’t do as individuals. He notes that Web 2.0 is:

“transforming the Internet into connected communities that allow people to interact with one another in new and distinct ways….

“… therefore, Web 2.0 – community building tools – seems tailor-made for government, at least theoretically.”

Schrier goes on to list over eighteen different ways in which Web 2.0 can be used to build online community engagement and some of the associated issues for councils, a taxonomy if you like of Web 2.0 applications. It would be difficult to summarise this list so I suggest you have a look at Schrier’s post. I’ll be using his framework to discuss some Web 2.0 applications in future posts.

Finally, Anne Stevens discusses the implications of Web 2.0 for the voluntary or community sector. She notes that there is a natural fit between specific-issue voluntary organisations and social media. People interested in a particular issue “will naturally be drawn towards others who share this interest and so community and collaboration online are an ideal way of leveraging this”. She also notes that Web 2.0 applications facilitate the sharing of information between multiple organisations working for the same cause (or single organisations with multiple locations) as well as with people who are isolated by geography or disability.

In future posts I’ll talk about some specific Web 2.0 tools as well as taking a look at some non-Web 2.0 applications that might be useful to councils and non-government organisations.

What are social media and Web 2.0 – and why are they significant to councils and NGOs?

Most people have now heard of Facebook, MySpace and YouTube, probably the most prominent examples of social media. The take-up of these applications is rapidly redefining the social landscape and changing forever the way we communicate.

The impact of social media is now being felt by companies, governments, NGOs and other organisations who are rethinking the way they deliver services and manage their own operations. These changes have in turn encouraged and been facilitated by the development of “Web 2.0” applications which are increasingly being used by some organisations to deliver services.

Wikipedia offers the following definitions of social media and Web 2.0:

Social media is information content created by people using highly accessible and scalable publishing technologies. At its most basic sense, social media is a shift in how people discover, read and share news, information and content. It’s a fusion of sociology and technology, transforming monologue (one to many) into dialogue (many to many) and is the democratisation of information, transforming people from content readers into publishers.

Web 2.0 refers to a perceived second generation of web development and design, that facilitates communication, secure information sharing, interoperability, and collaboration on the World Wide Web. Web 2.0 concepts have led to the development and evolution of web-based communities, hosted services, and applications; such as social-networking sites, video-sharing sites, wikis, blogs, and folksonomies.

In effect, the term Web 2.0 generically covers the range of developments in the web that facilitate the delivery of interactive technologies and applications which are now known as social media.

Whilst a small number of councils and NGOs have dipped their toes in the social media/Web 2.0 waters, it is interesting that the take-up has not been more widespread. This is partly because of the suspicion that many organisations harbour that social media is at best a waste of staff time and at worst a potential breach of security. This is both short-sighted,  in terms of recognising the changing expectations of both organisation staff and the wider community, as well as representing a failure in understanding the potential that these technologies can bring to small organisations to deliver better services and to interact with their clients and/or community members.

In later posts I will expand on this discussion as well as review some of the social networking and Web 2.0 tools that councils, NGOs and other small organisaitons might find useful. For the time being I’ll highlight some recent posts on blogs that also discuss these issues:

•  Simon Wakeman - Is social media a revolution in local government communications?
• Online Community Engagement – How Web 2.0 will transform Local Government
• LASA Knowledgebase – Web 2.0 for the voluntary sector