When your enemies criticise you, this doesn't automatically mean that they are lying, you know.
The Baptist heard about those rumours 100 miles away. He was worried by that, but there was no need for him to be, imo.
He DID like to take his wine and food with people that had poor reputations and...