‘The replacement may end up not having a Nazi tattoo or abuse women, but he’ll still share the socialist agenda’