‘Even Republicans that I speak to really wonder if he’s doing more harm than hurt than help in these campaign stops’