Evaluating process

An integral part of all investigations is the obtaining of an account from those involved, whether that is victims, witnesses, or suspects of crime. Quite often, the interview is the only source of evidence, especially in historical cases where there is little forensic evidence, or in ongoing live investigations where the evidence may not yet have been identified.

As such, obtaining accurate and reliable information is paramount to furthering understanding and facilitating effective decision-making during this stage of the investigation.

However, what some police officers believe they are doing during their interview does not always reflect what is actually occurring (Oxburgh, L., Gabbert, F., Milne, R., & Cherryman, J. [2016] Police officers’ perceptions and experiences with mentally disordered suspects, International Journal of Law and Psychiatry, 49, 138-146), and not all individuals involved in this process possess effective interviewing skills (Bockstaele, M. [2002]. Police interrogation and personality profiling. Brussels: Politeia). Thus, being able to evaluate interview performance is paramount in the maintenance of effective interview skills.

Conducting interviews, especially in complex investigations, can be challenging and while the current model of police interviewing in England and Wales (PEACE – Preparation and Planning; Engage and Explain; Account, Clarification, Challenge; Closure; and Evaluation – see Central Planning & Training Unit [1992a] The interviewer’s rulebook, Harrogate, CPTU; Williamson, T. [2006] Investigative interviewing, Abingdon: Routledge) is underpinned by robust and scientific research, there remain some challenges in evaluating interviews to understand human behaviour and the information gained during these processes.

Furthermore, despite the expectation that investigators will engage in interview evaluation to maintain best practice interviewing skills (the last ‘E’ of PEACE acknowledges evaluation as being a vital component of the interview process), there does not currently exist a single tool that assists with the full evaluation process.

The Forensic Interview Trace© (FIT©; Farrugia, L., Oxburgh, G., & Gabbert, F. [2019]. Effective evaluation of forensic interviews: The Forensic Interview Trace, Investigative Interviewing: Research and Practice, 10(1), 20-31) was developed as a technological solution to assist in evaluations following the interview stage.

The FIT© is software that was initially developed to assist investigators with the evaluation of their interview and their performance to predict the type and quality of information that may be obtained in future interviews with similar populations.

Grounded within key psychological concepts including those relating to memory, information retrieval, interpersonal communication and rapport-building, the FIT© has seen a progressive route to market through its development. However, the tool had received minimal pilot testing and required further evaluation and development.

The research team was funded by the Ministry of Defence (MoD) via Dstl (Defence Science and Technology Laboratory) under the Influence Programme (Chief Scientific Adviser [CSA] funded) by the Defence and Security Accelerator Fund, Phase 2, to further develop the tool from a prototype to a tool that is ready to be used by investigators.

This was achieved via a two-step approach.

Part 1 involved gaining feedback from participants in order to develop an understanding of their current interview evaluation techniques, as well as the usability, efficiency, and capability of software tools to assist in evaluation.

Part 2 involved an initial trial of the FIT© tool with participants to gain a unique insight in support of increasing its usability and capability. Such insights were used for final technical enhancements in the development of the FIT©.

A total of 32 participants from three police forces within England and Wales and six international security organisations engaged with Part 1 of the project; this consisted of a qualitative questionnaire that was split into two sections.

The first section focused solely on participants’ review and interview analysis and the second section focused on the development of a tool to assist in evaluation. The data was analysed using Thematic Analysis (Braun, V. & Clarke, V. [2019] Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health, 11(4), 589-597), which identified three main themes and eight sub-themes.

These focused on (i) current evaluation methods, including no evaluations conducted and the inconsistent use of interview evaluations’ (ii) barriers to conducting evaluations, including lack of resources and time; and (iii) technological solutions with participants highlighting perceived benefits of having a tool to assist them.

Following the completion of Part 1, three short video-training packages were developed based upon the main features of the FIT© focusing on rapport and engagement, question typology and effective evaluations.

These were disseminated to a total of 147 participants from five defence and security organisations (nationally and internationally) within Part 2 of the project. During this stage, participants were required to then use the FIT© to analyse at least one to two interviews a month for a period of three months. To date, a total of four participants have successfully completed the pilot that focused on user experience, while remaining participants are mid-way through the trial.

Thematic analysis on the returned data identified two main themes: (i) positive experiences of using the FIT©; and (ii) suggestions for further developments. In addition, participants were provided with an Evidence Profile Table and a Validation Profile Table. Here, participants rated the FIT© according to its comprehensiveness, relevance, plausibility, and overall content.

The majority of the participants scored the FIT© highly across all of these domains.

Findings to date suggest that participants find it difficult to complete evaluations of their interviews; this was evident in the current project given the varying levels of engagement throughout.

However, participants have also identified the benefits of using the FIT© designed specifically for the evaluation of their interviews. Such positive experiences related to the ease and efficiency of the tool in predicting the type and quality of information that may be obtained in future interviews with similar populations.

The suggestions received thus far have been implemented into final technical enhancements of the FIT©.

This project has allowed for the development of the FIT© prototype and has highlighted the significant and important nature of conducting evaluations of interviews as part of skill maintenance and development.

Recommendations based on the findings from the current project include continued collaboration between researchers and practitioners in understanding how tools and techniques can support interview evaluation, and the development of tools should consider enablers including visual feedback and barriers, such as time, and be developed using a bottom-up, user-centred approach to design.

The findings from the current project have provided encouraging results that show that the FIT© has the potential to improve the effectiveness of interviews in that the analysis may assist with skill development and better future interview planning.

While the current prototype is not yet ready for operational use, areas for further development have been identified. This includes the continued development of the FIT© by focusing on understanding product gap analysis, market qualification and developing a business model to investigate options for commercialisation.

If you would like to learn more about the FIT©, get involved with the trial, or simply receive updates as the tool progresses, then please contact Dr Laura Farrugia at laura.farrugia@northumbria.ac.uk

Dr Laura Farrugia is an Assistant Professor at Northumbria University and a Registered Intermediary. Her research focuses on vulnerability in the Criminal Justice System (CJS) with a strong emphasis on interviewing vulnerable victims, witnesses, and suspects. She has published extensively in this area and her research drives her practice as a Registered Intermediary where she advises police officers and courts on the most appropriate manner to interview or cross-examine vulnerable victims and witnesses. She regularly delivers training to national and international bodies on how best to communicate effectively with vulnerable individuals.

Professor Gavin Oxburgh is a Registered Forensic Psychology and Professor of Police Science at Northumbria University. He is a 22-year veteran of the Royal Air Force Police and is a recognised expert in communication, non-coercive interviewing, and human rights. He is the Assistant Director of the Northern Hub for Veterans and Military Families Research and was the co-founder of the International Investigative Interviewing Research Group, (a group he chaired from 2007-2019. He works closely with international bodies and was a key member of the UN-led International Steering Committee that developed The Mendez Principles on effective interviewing for investigations and information gathering.

Fiona Gabbert is a Professor of Applied Psychology, and Director of the Forensic Psychology Unit at Goldsmiths University of London. Her research in the fields of memory suggestibility and investigative interviewing has a strong focus on improving the usability, credibility, and reliability of evidence from victims and eyewitnesses. Her work has had an international impact on operational procedure and policy including the introduction of new evidence-based investigative interview tools and training resources to the field such as the Self-Administered Interview, the Structured Interview Protocol, and the Timeline Technique.