Measurement of Competence in Evidence Based Practice

Abstract

Evidence based practice (EBP) is a clinical decision making process which integrates the use of best available evidence, clinical expertise, with consideration of the preferences and values of the person with health concerns. The implementation of EBP is mainly through the 5-step process: asking focussed questions, acquiring research evidence, appraising the quality of evidence, applying evidence in practice and assessing the outcome of care. Although EBP has been the focus of optimal health care for more than three decades, the uptake of EBP is still influenced by a range of individual, organisational and environmental factors such as insufficient evidence appraisal skills and heavy workload. The barriers to and facilitators for successful implementation of EBP have been extensively researched, where the ultimate goal is to minimise the evidence-to-practice gap in health services. To achieve this, the initiative of knowledge translation which aimed at reducing barriers and facilitating the use of research evidence in clinical decision making, is viewed as the bridge that brings together the behaviour change of clinicians in applying evidence-based interventions to patient care. Moving evidence into practice is a persistent challenge to all health professionals, and the translation of evidence into nursing practice is no exception to other health disciplines. Research into the facilitation of EBP implementation has highlighted the critical role of clinical nurse educators (CNEs) in supporting clinicians and fostering an evidence-based learning culture in the workplace. Nevertheless, a recent systematic review has concluded that many CNEs did not perceive themselves as comfortable in engaging with EBP. This finding was however limited by the use of self-report instruments, suggesting the need for an instrument that can more objectively measure the EBP knowledge and skills of nurse clinicians. A further problem is the lack of a validated appraisal tool to assess the quality of EBP instruments using quantitative measurements. Moreover, previous reviews of such instruments have only focussed on their validity for measuring discrete elements associated with EBP such as attitudes to research utilisation and outcome-expectancy in EBP. No review has ever been conducted to identify instruments for evaluation of competence in evidence-based nursing practice.The overarching purpose of this thesis is to gain a better understanding about the EBP knowledge and skills of nurse clinicians and educators, whose role is to support and nurture an EBP learning environment in clinical settings in Australia. This thesis used a stepwise approach to address multiple issues associated with the measurement of competence in EBP. First, the development of an appraisal tool for assessing the validity of instruments (Chapter 3), then using this tool to systematically evaluate the quality of all EBP instruments used in nursing (Chapter 4). An EBP competency framework was then developed (Chapter 5) to guide the development (Chapter 6 and 7) and validation (Chapter 8 and 9) of a new EBP instrument called the Evidence-based Practice Assessment Tool (EPAT). A minimum level of competency in EBP was then determined with a group of mainly CNEs and nurse clinicians through a standard setting procedure described in Chapter 9. Finally, a factor analysis was performed with the EPAT and the final version was a 10-item assessment tool which consists of two 5-item subscales. The ‘EBP knowledge’ subscale includes five items related to EBP steps 1-3 (PICO question for which P=Patient/problem/population, I=Intervention, C=Comparison, O=Outcome; search for evidence, appraisal of treatment effect) and step 5 (evaluate quality of care) whereas another five items in the ‘Shared Decision Making (SDM) skills’ subscale are indicators for EBP steps 3 and 4 (appraise benefit and harm of treatment options, explain choices and identify values). Participants answered all questions on the EPAT related to a clinical scenario and the provided evidence summary. Using the framework of the Construct Validity Model, four types of validity were assessed to support the psychometric characteristics of the EPAT: the content validity, response process, internal structure, and relationship to other variables. The strong correlation between the two subscales confirmed unique but theoretically related constructs, which means ‘EBP knowledge’ and ‘SDM skills’ constituted to the measurement of competence in EBP (Chapter 9). The EPAT was deemed to have good content validity and response process (Chapter 8), good internal consistency and factorial validity; and there was a significant relationship to different levels of exposure in EBP education (Chapter 9). The EPAT also demonstrated good inter-rater reliability through substantial agreement among seven markers, also reflecting the reliability of criteria set for the marking guidelines. The EPAT is deemed to be a very feasible tool for use in different educational settings with an average completion time of only 33 minutes (Chapter 9). The results of participants, described in Chapter 9, demonstrated an overall low pass rate (35%) on the EPAT, which may be associated with the participant ability or the difficulty of most EPAT items. These findings are similar to other modified Fresno tests which only measure the first three steps of EBP. Moreover, this thesis provides new information in regards to the competence of nurse clinicians in applying evidence and evaluating the quality of care (EBP step 4-5). It is disappointing to see that the average pass rate for SDM skills of this cohort participant was only 24% whereas the evaluation of care was even lower at 14%, indicating the priority for future EBP education and training in nursing. Chapters 5-9 included in this thesis represent the development and validation of the first EBP instrument for measuring the actual EBP knowledge and skills in nurse clinicians, through the 5-step implementation model. The results of this thesis have provided an important contribution to the knowledge of measurement in EBP and reveal an overall low level of EBP competence among nurse clinicians. The newly developed EPAT enables a comprehensive evaluation of learner’s ability and paves the way for a more focused EBP education and training in healthcare.