Hi Gracie89,
Yes I would advise you to keep any overpayment to one side. There are lots of stories of the DWP out of the blue demanding a repayment of an alleged overpayment made 3 or more years ago.
David
Nothing on this board constitutes legal advice - always consult a professional about specific problems