We study the dynamics of the linear and non-linear serial dependencies in financial time series in a rolling window framework. In particular, we focus on the detection of episodes of statistically significant two- and three-point correlations in the returns of several leading currency exchange rates that could offer some potential for their predictability. We employ a rolling window approach in order to capture the correlation dynamics for different window lengths and analyze the distributions of periods with statistically significant correlations. We find that for sufficiently large window lengths these distributions fit well to power-law behavior. We also measure the predictability itself by a hit rate, i.e. the rate of consistency between the signs of the actual returns and their predictions, obtained from a simple correlation-based predictor. It is found that during these relatively brief periods the returns are predictable to a certain degree and the predictability depends on the selection of the window length.