Development of Measurement Scales for Teacher Professional Development in ICT Integration: Insights into Program Value, TPACK, and Student Learning Achievement in Primary Education

Abstract

Researchers have emphasised the importance of considering how teachers value a professional development experience when designing and refining a teacher professional development (TPD) program. Despite evidence that teachers’ perceptions shape the successful implementation and impact of TPD, few measurement instruments exist to assess the value that teachers obtain from participation in TPD or the professional growth that they believe it generates. Given the growing global demand for TPD programs to enhance teachers’ ability to successfully integrate Information and Communication Technology (ICT) into teaching, it is therefore necessary to ensure that such TPD are valuable from perspective of teachers. However, there is presently a lack of measurement instruments which can measure teachers’ perceptions of the value of TPD and the results of their participation. This study addresses this gap by developing two measurement scales - the Value of TPD (VT) scale and the Improvement of TPACK core via TPD Experience (ITTE) scale - using the context of a Taiwanese national ICT integration TPD initiative. Scale development was grounded in Rasch Measurement Theory, which establishes requirements for constructing instruments that yield meaningful, invariant measurement. Kane’s argument based validation framework guided the validation process. Validity evidence was gathered through a literature review, three rounds of focus group discussions, and a large scale questionnaire survey. Focus group feedback contributed to the refinement of initial item pools, and the Rasch analysis was used to evaluate item functioning and provide evidence of the unidimensionality and measurement properties of the VT and ITTE scales. Correlation analyses explored relationships among VT, ITTE, and students’ learning achievement; no significant associations were found, suggesting that the implications of VT and ITTE scores may not extend to student outcomes. The relatively small sample size and the lack of controls in generating the data for this stage of the study, means that these findings warrant cautious interpretation and further investigation. This study offers three significant contributions. First, it demonstrates the application of Rasch Measurement Theory in developing robust measurement scales. Second, the study’s VT and ITTE scales are generalisable to other contexts of TPD beyond the TPD program for ICT integration in Taiwan. The results collected through the VT and ITTE scales can be meaningfully compared across contexts, groups of teachers and means of administration. Finally, the development of VT and ITTE scales provides tools for researchers, policymakers, and system leaders to monitor and refine TPD implementation from teachers’ perspectives. Together, the VT and ITTE scales support more evidence informed design, evaluation, and continuous improvement of TPD programs.