It might have not crossed your radar, but 2015 has been declared as the International Year of Evaluation, around which a series of events and publications aim to “to advocate and promote evaluation and evidence-based policy making at international, regional, national and local levels”.

In Local Authorities, appraisal, monitoring and evaluation exercises are a common source of requests for research and data. However, there is an increasing demand for evaluations of externally commissioned projects, as more services are being run by local community based organisations. These may link with activities being provided in any part of Local Authority responsibilities. The Whole Place Community Budgets programme is just one framework in which smaller service providers are being supported by local government.

Laudable though these efforts may be, a common worry from LA commissioners is that many local organisations they would like to collaborate with lack the capacity to engage with the funding and reporting requirements of essentially being government contractors. While financial reporting obligations are often familiar enough for established charities (which must be independently audited every year) and social enterprises (with annual Companies House submissions), demonstrating impact is a challenge for all stakeholders.

With small pilot schemes, demonstrating measurable quantitative improvement is very difficult to achieve, especially to the level of statistical significance that reassures researchers and funding panels. However, these projects can probably contribute good qualitative evidence in their evaluation, from testimonials of service users and staff, workshops, interviews and focus groups with stakeholders. Having a rigorous process for the analysis and presentation of this qualitative data can be a challenge to some groups, not as familiar working with qualitative data as they are quantitative data in a spreadsheet.

That was one of the reasons we designed Quirkos, as we felt there was a need for software to manage and analyse qualitative data, which was easy enough to be used by 3rd sector providers, commissioners and funding boards to explore and present qualitative data.

Yet there is also a need to share evaluations in an accessible format so that other departments and regions can learn from them. Mini-case studies are a good way to do this: a format that presents learning points as well as summary evaluations in one page summaries. I was involved in developing a series of case studies on commissioning health and social care services for minority ethnic populations. It neatly shows 10 examples of how largely qualitative insights demonstrated the effectiveness of changes to service provision.

Evaluations can be powerful tools for best practice, and integrating qualitative data can provide humanising detail as well as deep insight into the best ways to improving service delivery. There is no one approach for integrating qualitative data into evaluations, but there are tips for qualitative evaluations on our recent blog post article.

And a series of links to other useful resources and guides:

http://www.civilservice.gov.uk/wp-content/uploads/2011/09/Qualitative-Appraisal-Tool_tcm6-7385.pdf
http://ec.europa.eu/regional_policy/sources/docgener/evaluation/doc/performance/Vanclay.pdf
http://ncsu.edu/ffci/publications/2011/v16-n1-2011-spring/vaterlaus-higginbotham.php

Photo credit: London by Roberto Trm

The post Integrating qualitative data into evaluations appeared first on LARIA - Local Area Research + Intelligence Association.

For personnel working in local public sector organisations such as the NHS and local authorities, the need to consult with staff, service users and the public to implement service changes is more important than ever. We all acknowledge that the cuts in funding imposed by central government are putting significant pressure on consultation and research teams to do more with fewer resources and less funding. Whilst in the background, bubbling away, there is the ongoing menace of judicial review, striking fear into the hearts and minds of consultation teams across the country!

Whether we like it or not, the reality is that public consultation has never been more important and is certainly here to stay. Conducting a successful consultation depends on managing the process as effectively as possible and choosing the most appropriate approach and dialogue methods, which often means using a blended approach of both qualitative and quantitative methods.

Yet organisations often lean towards a quantitative approach when reporting consultations. Statistical data clearly illustrates whether a person is in favour, or not, of a particular proposal. Quantitative information will therefore always form the backbone of any public consultation, as it can easily be managed in large volumes and is relatively straightforward to reference when reporting findings.

However, the downside is that a quantitative approach doesn’t necessarily reflect the true opinions and views of the service user, as it doesn’t give them the opportunity to respond to the consultation questions in their own words. NVivo was developed to help provide a way of structuring qualitative data. Such information provides organisations with a valuable opportunity to really dig deep into their data, exploring issues, identifying emerging themes and making sense of unstructured information.

By being overwhelmed with qualitative data, or avoiding collecting it in the first place, important information can be lost. Exploring a rich data source enables public sector organisations to facilitate genuine (rather than tokenistic) discussion with their service users and with the wider public. It provides the ‘meat on the bones’ in terms of giving a fuller picture of what local people and communities are actually saying about services and gives organisations, like local councils, the opportunity to respond and act accordingly and to help build meaningful dialogue.

Ultimately, consultation data needs to be fit for purpose, stand up to scrutiny and support decision-making processes. Engagement and consultation teams must therefore evolve practices to report on both qualitative and quantitative findings. Only by drawing on the strengths of both approaches can Chief Officers and Elected Members make truly informed decisions relating to the provision of services to citizens and communities.

http://www.qsrinternational.com/solutions_government.aspx

Email: d.armstrong@qsrinternational.com

Photo Credit: Paul Krueger

The post The Value of Qualitative Research in Public Consultation appeared first on LARIA - Local Area Research + Intelligence Association.