Imaginary Knitting. Historical record linkage in the Caversham project

Abstract

The Caversham Project began 20 years ago to study the Caversham region of Dunedin, New Zealand, using systematic methods and comparative analysis. It is the largest project in social history or historical sociology in Australia or New Zealand, and one of the largest in the world. The databases of the project hold over a quarter of a million records gathered from over 80 sources. These records are divided between a text-base of oral history material, a GIS of land records, and a series of database files from various sources. This paper documents the strategies employed to improve the linking between database records. Both explicit and implicit criteria were employed in record linkage. Explicit criteria were those based on linking between the contents of records. Implicit criteria were those based on the relationship between groups of records. Implicit criteria were found to be a very useful means of linking records, as well as a powerful tool for locating errors containing anomalies (data integrity). Analysis of the linking process demonstrates that using implicit criteria can improve the linkage of records. Use of such criteria also enables questions to be addressed of the data about the hidden relationships that went into forming the source information. By making databases sensitive to the real world relationships that surrounded the source the records sprang from, a better model of the past can be created.