CRA will only allow an RRSP contribution to be designated as a Repayment to the Home Buyers Plan (HBP) if the contribution is made during the tax year or the first 60 days of the following year. Any amounts carried forward from previous years cannot be used as repayments. 

The following information is available in this CRA publication: How to repay the funds withdrawn from RRSP(s) under the Home Buyers’ Plan (HBP)

To make a repayment under the HBP, you have to make a contribution(s) to your RRSPs,  PRPP or SPP in the year the repayment is due or in the first 60 days of the year after. Once your contribution is made, you can designate all or part of the contribution as a repayment.

Thank you, not the answer I had hoped for but you saved me lots of time. 

I'm having the same issue this year. Is this a new rule? I have used unused RRSP contributions towards my HBP for my 2016 and 2017 returns with no issue. 2018 and 2019, I had contributed more then the annual minimum repayment amount so didn't have have an issue.