The peer review process: an author’s view

Richard Cotmore, Head of the NSPCC’s Evidence Department, reflects on the introduction of a peer review process

Girl reading a bookLast year we introduced a peer review process for our research and evaluation reports. This means that, before a report is approved for publication it has to be reviewed by 2 subject experts who do not work for the NSPCC. The evaluator or researcher who wrote the report does not know who the reviewers are and the reviewers provide this service free to the NSPCC.

Having seen the process in action, I can see exactly how it can strengthen our work.

Benefits of peer review

I had a mixed response when peer reviewing was first discussed.

I always thought it was really important to have more external scrutiny of our work. It provides a significant additional safeguard and helps to address the risk of bias.

As well as peer reviewing our reports, I was pleased that we added external review and support to the earlier stages of the evaluation planning process. This was through:

  • our research ethics committee (the majority of members are independent – they do not work for the NSPCC)
  • the use of external consultants and advisory groups.

Initial concerns

I had 2 main reservations about the peer review process: would it be yet another hurdle to jump and cause unnecessary delay to our reports being published? And would the reviewers understand how we report?

The latter question is important. The house style of our reports involves writing accessibly: we write for an informed, interested audience but we do not assume research skills and experience.

I was concerned that reviewers would be disappointed by a lack of technical information; they might assume a lack of knowledge on our part, rather than a legitimate attempt to convey our reporting clearly for non-research audiences.

Several months on, however I can honestly say I’ve been really pleased with how it’s going and the constructive responses we’ve received.

Yes, it can be annoying to revisit a report you hoped everyone would love in its original form - but it’s far better to iron out any problems before a report is in the public domain.

Peer Review of Family SMILES Evaluation

I found the comments we received on the evaluation report for our Family SMILES programme to be particularly helpful.

One of the reviewers was so engaged, they typed up 8 pages of notes - almost a report in itself!

Family SMILES is a group work programme supporting families of primary school aged children, where a parent has a significant mental health difficulty.

The reviewer thought we had used an appropriate methodology, and noted that our findings were mixed - we only found improvements in some areas, and not all positive changes were sustained beyond the programme.

A deeper challenge

The reviewer was critical of how we interpreted the findings, however and this was more challenging for us.

They felt we put too much emphasis on the negative aspects of having a parent with mental health issues, and that we didn’t spend enough time portraying the very positive relationships children can have with their parents.

More broadly, they didn’t think we had paid sufficient attention to the external and systemic factors that can affect a parent’s mental health, such as poor housing and poverty. As a result they felt there was a risk that parental mental health problems could be misrepresented.

The reviewer’s contribution really made us think about how we talk about our mixed findings. Should our report focus solely on the service being evaluated or should it put the findings from the evaluation into context by including a broader discussion of the relevant issues?

The reviewer argued that, without addressing the broader systemic factors, a programme like Family SMILES would be unlikely to generate significant and long lasting change on its own.

This led us to realise that, while the programme could have some really positive outcomes for children and families, we needed to think differently about how it connected with wider support systems.

This fits with other work that’s going on in the NSPCC at the moment, looking at how we need to work with other organisations in a more joined-up way to support families with adversities.

A balancing act

Yes, it was challenging to revisit the report - but it’s definitely more rounded as a result of the peer review process, which is a very positive result.

I really appreciated how balanced the reviewer’s comments were.

They agreed that our positive findings for Family SMILES were potentially significant. But their review helped us reframe our interpretation of the less positive findings. This in turn has helped us think more meaningfully about how we can help families more effectively.

Look at the final evaluation report for Family SMILES to see our final interpretation of the findings.

Like this blog?

Let us know which blog you've read, what you think, share information you have on the topic or seek advice. 

Get in touch

More from impact and evidence

How to be a peer reviewer

Dr. Catherine Hamilton-Giachritsis considers the benefits of the peer review process for both sides, offering advice to anybody interested in becoming a peer reviewer.
Find out more

Setting up the peer review process

Mieka Webber offers advice on approaching and engaging subject experts to peer review research reports.
Find out more

Impact and evidence

Find out how we evaluate and research the impact we’re making in protecting children, get tips and tools for researchers and access resources.
Impact and evidence

Support for professionals

Follow @NSPCCpro

Follow us on Twitter and keep up-to-date with all the latest news in child protection.

Follow @NSPCCpro on Twitter

Library catalogue

We hold the UK's largest collection of child protection resources and the only UK database specialising in published material on child protection, child abuse and child neglect.

Search the library

New in the Library

A free weekly email listing all of the new child protection publications added to our library collection.

Sign up to New in the Library

Helping you keep children safe

Read our guide for professionals on what we do and the ways we can work with you to protect children and prevent abuse and neglect.

Read our guide (PDF)

Impact and evidence

Find out how we evaluate and research the impact we’re making in protecting children, get tips and tools for researchers and access resources.
Impact and evidence

Impact and evidence insights

Each week we’ll be posting insights from professionals about evaluation methods, issues and experiences in child abuse services and prevention. 
Read our blogs

Sharing knowledge to keep children safe

Read our guide to the NSPCC Knowledge and Information Service to find out how we can help you with child protection queries, support your research, and help you learn and develop.

Read our guide (PDF)