DeCoP: Dependency Controlled Pre-training for Time Series Representation Learning

Abstract

Masked time series modeling (MTM) has become a leading approach in self-supervised pre-training for time series data. However, existing frameworks struggle to effectively model dependencies that balance informative signals and noise, often leading to overfitting or missing critical temporal dependencies due to non-stationarity and limited semantic context in time series data. To address this, we introduce DeCoP, a Dependency Controlled Pre-training framework that enhances self-supervised time series representation by effectively controlling dependency modeling, while significantly reducing computational cost. DeCoP controllably reduces non-stationary noise and disentangles mixed temporal variations from coarse to fine levels, thereby improving representation clarity. Specifically, DeCoP incorporates Instance-wise Patch Normalization (IPN) for controlled dependency, which reduces noise and stabilizes data distributions, establishing a controlled foundation for dependency modeling. Furthermore, a Hierarchical Dependency Controlled Learning (DCL) strategy is employed to selectively control inter-patch dependency ranges, generating robust, generalizable embeddings that enhance model stability across varying time series patterns. Extensive evaluations on ETTh1 datasets reveal that DeCoP achieves up to a 3% improvement in MSE over PatchTST, using only 37% of the FLOPs required.