aiden
The assessor may have believed that the decision to refuse a home assessment was made by the DWP but it would not have been, the DWP do not get involved unless the claimant has failed to attend, so the decision would have been made by the Assessment Provider.
Obviously, I don't know what else the DP letter said but arguments that you are not Fit for Work would not have influenced those assessing you, PIP is not a work related benefit so no account of this would have been taken.
You can complain to the Assessment Provider
Complaining to the PIP Assessment Providers Gordon