Butch and Vincent

Ok I’ve watched this scene both with and without the Trivia Track several times. In Sally Le Roy’s bar ,where Marsellus tells Butch to take the dive, why is it when he and Vince see each other at the bar Vince calls Butch a “Palooka” I understand the reference to Joe Palooka that he was a boxer who was supposed to take a dive but didn’t , but why does Vince call Butch this.Butch did nothing to provoke Vince, so why does Vince outright insult him, is there a backstory or something? I’m goin berserk tryin’ to figure this out if someone could clear things up I’d really appreciate it.