Vulnerable families and evaluations - 7 ways to address challenges

This is the second of a 2 part series discussing how to include vulnerable families in programme evaluation by Nicola McConnell

boy and father playingAfter completing our evaluation of Caring Dads: Safer Children, a parenting programme for domestically abusive fathers, I acknowledged the challenges posed by involving children and partners.

I flagged how important their data was in providing a realistic view of the fathers' behaviour. 

In this post, I wanted to discuss the challenges the evaluation posed when working with the fathers' families, and what can be done to address them.

Timing of baseline data collection

The evaluation design for Caring Dads: Safer Children was a “before and after” study, so data collection needed to happen at the programme’s beginning and end to measure any change in behaviour during the service.

For practitioners working with fathers’ families, collecting baseline data at the beginning often felt too early and intrusive because the families were only just getting to know them.

"I didn’t feel it was the right time to be asking those questions, even though they were relevant and significant."
Caring Dads: Safer Children / engagement worker

Asking partners to complete questionnaires at this stage could inhibit their engagement with the service and this posed a potential problem for the evaluation: would partners and children respond honestly in such circumstances?

One partner engagement worker said: “I didn’t feel it was the right time to be asking those questions, even though they were relevant and significant. The relationship wasn’t established well enough for her to feel comfortable and be honest – or for me to accurately assess if she was being honest.”

Teams also felt the amount of evaluation tasks needed within the time allocated for visits meant there was less opportunity to build relationships or provide support.

Risk of causing distress

During research interviews, I asked family members about their contact with the father, their views on the programme and whether the programme had made a difference to their family. 

Although several people voluntarily shared personal information, I avoided asking family members about past experience of abuse. That wasn’t the interview’s focus or purpose.

A greater concern was whether the questionnaires, which asked about the fathers’ behaviour towards their children and partners, could cause respondents distress, or open-up problematic issues for the child/mother.

While most who participated in the evaluation said they felt fine about the process, a few partners said the questionnaire about the father’s controlling behaviour described abuse that was shocking.

One woman said that when she read the questionnaire she experienced flashbacks of abuse from a previous relationship.

This incident illustrates the value of asking experienced practitioners to administer measures as they can provide support if necessary.

Gaining important information

Responses to the questionnaires could provide helpful information for practitioners such as disclosure of self-harm, anxiety or further incidents of abuse.

In these circumstances, practitioners were able to make referrals for additional support and work closely with referrers and group facilitators to ensure that any incidents were addressed in a way that was safe and appropriate for the family.

7 ways to address challenges

You can never anticipate all the challenges and complexities when involving vulnerable people within an evaluation.

These 7 pointers may help evaluators to be more prepared:

  • ensure practitioners or researchers working with families have a comprehensive knowledge of safeguarding procedures
  • tell evaluation participants they can withdraw from the evaluation at any time, or refuse to answer questions without giving a reason. For more information view our research with children: ethics, safety and avoiding harm page.
  • explain the limits of confidentiality to participants. All information will be kept confidential unless they tell you something that leads you to believe a child or vulnerable person is at risk
  • provide training for those not used to speaking with families in difficult situations so they can manage the risk of interviews causing distress
  • take time to choose and test standardised questionnaires in real life situations where practitioners are building relationships with clients, rather than a research setting
  • provide potential interviewees with a copy of the topic guide so they can make an informed decision about whether they wish to participate
  • stay focused on the purpose of the interview. Never make a participant feel obliged to share personal information.

Like this blog?

Let us know which blog you've read, what you think, share information you have on the topic or seek advice. 

Get in touch

More information

Involving vulnerable families in evaluation

Read part 1 of this 2 part series by Nicola McConnell on how to include families in programme evaluation.

Read more

Impact and evidence insights

Each week we'll be posting insights from professionals about evaluation methods, issues and experiences in child abuse services and prevention.

Read our blogs

Impact and evidence

Find out how we evaluate and research the impact we’re making in protecting children, get tips and tools for researchers and access resources.
Impact and evidence

Caring Dads: Safer Children

Helping fathers see the impact their violent behaviour has on their children and how they can make positive changes for the future.
Caring Dads: Safer Children service