Using randomised control trials in social care

Richard Cotmore discusses how randomised control trials (RCTs) can be used to measure the impact of services for children and families

Parent and child reading a bookWe use randomised control trials (RCTs) to find out if some of our services are having a positive impact on children and families. We do this by comparing the outcomes for 2 groups of people: one group that uses a particular service and one that doesn't.

The way we allocate people to each group is purely random – so if one group has a better outcome than the other, we can link it to the support they are getting through the service. RCTs are rare within a social care setting so it can be something of a culture shock for people working with children and families when we ask them to become involved in this sort of research.

There are different ways to organise an RCT, but in the example below we used a waiting list for the control group. Researchers from Bristol and Durham universities carried out an independent evaluation of our Letting the Future In programme, a service for children and young people who have been affected by sexual abuse. About half of the children were able to take part in Letting the Future In straight away. The other half were asked to go on a waiting list, so that we could compare the outcomes of children and young people who had and hadn’t received the service. After we had collected the data for the research, group 2 were all able to take part in the service.

A huge amount of work was required from our practitioners, their managers and our evaluation department to support the researchers who were carrying out the evaluation.

Here is what we learned.

The challenges of using RCT

When you’re doing a RCT, you need to stick to the model the trial is testing. This can have an impact on social work practice, as it reduces the scope for practitioners to exercise their judgement

For example, when working with parents and carers, one manager observed: “We had to decide on a specific number of sessions in advance, as flexibility wasn’t possible through the RCT. We’ll find out through the research whether more sessions were needed.”

Practitioners are normally able to use their judgement and change the number of sessions they offer depending on the needs of individual families. So this felt like a constraint on their professional decision making.

And, so that we could get the right number of families for the trial, we had to operate a waiting list. This was a change in NSPCC policy. We hadn’t previously used waiting lists, because we didn’t want young people to miss out on getting support elsewhere whilst they were waiting to access our services. We had to think carefully about the impact this change could have on vulnerable children and families before we could proceed.

How to manage randomisation

Running a waiting list while managing caseloads was complex. We needed to have enough referrals to make the RCT random, and enough spaces to take those who needed immediate support, but we also had to keep an eye on the “back end” to see who was finishing the service, making space for those on the waiting list.

It was essential that the practitioners who were working with children and families understood the randomisation process and could communicate it confidently to others: it was their responsibility to explain it to potential service users.

Such discussions were new for them and initially they could feel, as one practitioner acknowledged, “tongue-tied, embarrassed and apologetic”.

As part of the research, practitioners had to carry out an eligibility assessment and research assessment with each family, and making sure these happened in the right order, at the right time, was tricky.

Practitioners felt it was important to allow a gap between explaining the process to families and letting them know when they would receive the service. They could then process the results of the research assessment, check that the family was safe to go on the waiting list and prepare themselves to deliver the news to families.

The ethics of RCTs

Practitioners were understandably concerned about making a child who needed a service wait: it felt counterintuitive, given that their role is to support children and families.

It was important to explain the research clearly to families using the service, and to make sure the families on the waiting list had enough support - without crossing the boundary by providing them with a full service too early on.

We set up an independent process for those on the waiting list to be signposted to support – but no one used it. Practitioners reported that because families knew they would definitely be getting a service within a clearly defined period of time, they seemed happy to wait.

It was important to have ongoing communication between the researchers and the NSPCC, and there was a feeling that the collaboration was strong. As one manager observed: “Bristol and Durham (university researchers) were very good at listening. They took on board so many elements of feedback and tried to learn from the feedback and make changes as a result.”

Lessons learned

Introducing RCT methodology to the NSPCC was a new initiative and staff at all levels needed time to understand and adjust to it.

It helped to identify practitioners who were “champions” for the study, who were seen as understanding what was in the best interests of the children and had credibility with their peers when explaining how it would work.

At first, many practical issues were raised that appeared insurmountable, but once our practitioners were convinced the study was worthwhile, they helped work out how to overcome challenges.

Our practitioners learned to talk about the RCT to families and professionals. They found the materials provided by the researchers, including suggested scripts and process flowcharts, very helpful.

With this kind of support from researchers and colleagues, and with practice, practitioners felt more confident in discussing the RCT and the rationale for the waiting list. As one noted: “Waiting lists are normal in the health service. As I felt more comfortable, parents and children weren’t anxious”.

The other RCTs we have been involved with have operated differently. From what we have learnt so far, my tips for ensuring the success of an RCT are:

  1. plan well ahead with ample preparation time
  2. identify a sound strategy and be organised at every stage
  3. provide reassurance and support to all those involved
  4. prioritise face-to-face communication
  5. give information and support materials at every stage
  6. create positive feedback loops.

Read the full report of our evaluation of Letting the Future In, our first evaluation using a RCT.

Like this blog?

Let us know which blog you've read, what you think, share information you have on the topic or seek advice. 

Get in touch

More from impact and evidence

Impact and evidence insights

Each week we’ll be posting insights from professionals about evaluation methods, issues and experiences in child abuse services and prevention. 
Read our blogs

Letting the Future In

Letting the Future In helps children who have been sexually abused.
Letting the Future In service

Using evaluation tools and standardised measures in assessment

A practitioner's perspective on whether evaluation measures help practitioners in their assessment of a child
Read more

Using propensity score matching in evaluation

Denise Coster used an "experimental design" to evaluate the Coping with Crying film.
Read more

Evaluation in social policy

Leon Feinstein from the Early Intervention Foundation discusses randomised controlled trials, social policy and the future of evaluation.
Read more

Support for professionals

Follow @NSPCCpro

Follow us on Twitter and keep up-to-date with all the latest news in child protection.

Follow @NSPCCpro on Twitter

Library catalogue

We hold the UK's largest collection of child protection resources and the only UK database specialising in published material on child protection, child abuse and child neglect.

Search the library

New in the Library

A free weekly email listing all of the new child protection publications added to our library collection.

Sign up to New in the Library

Helping you keep children safe

Read our guide for professionals on what we do and the ways we can work with you to protect children and prevent abuse and neglect.

Read our guide (PDF)

Impact and evidence

Find out how we evaluate and research the impact we’re making in protecting children, get tips and tools for researchers and access resources.
Impact and evidence

Impact and evidence insights

Each week we’ll be posting insights from professionals about evaluation methods, issues and experiences in child abuse services and prevention. 
Read our blogs

Sharing knowledge to keep children safe

Read our guide to the NSPCC Knowledge and Information Service to find out how we can help you with child protection queries, support your research, and help you learn and develop.

Read our guide (PDF)