Practical Considerations for Evaluating Reliability in Ambulatory Assessment Studies

Ambulatory assessment studies have unique challenges related to obtaining adequate reliability relative to other types of research designs. For example, because of time constraints, limitations in the number of items administered at each occasion can negatively affect the reliable assessment of within-person change. Whereas some types of measures, such as objective measures of physical activity, have been the focus of a cumulative body of reliability research, a lack of reporting of reliability data for cognitive and self-report measures in many studies is a significant barrier for researchers who want to choose measures based on prior psychometric research. Several approaches to assessing the reliability of ambulatory assessment data are available, and further research with less widely used reliability metrics (e.g., stability in within-person variability across time) is warranted. Given the time and effort involved in conducting ambulatory assessment studies, proper consideration of reliability is needed so that researchers have sufficient power to take advantage of the large amount of data often available within a single ambulatory assessment study.