Should Hector Have Killed the Ruffians?

One thing I like about playing D&D is that it puts me into the shoes of a paladin and gives me many opportunities to ask, "In this situation, what would a paladin do?"

In my most recent gaming session, my character, Hector, was at a tavern, watching a bard from his adventuring party perform. At one point, a group of tough customers asked, or rather, demanded, that the bard play a particular song - a song the bard didn't know. The bard made his apologies, but the ruffians continued to insist that he play the song, and they very nearly became violent. Fortunately, the party was able to persuade another patron of the tavern to teach the bard the song, which the bard then performed, assuaging the ruffians. The next day, while travelling to another town, our party ran into the same ruffians, who stopped us on the road, eyed up our equipment, and then let us pass without further incident.

It was abundantly clear by both of these encounters that these ruffians were bad news. They almost attacked our bard for not playing a song he didn't know, and they at least considered the idea of attacking and robbing us on the road. In both cases, Hector was ready to step in, sword in hand, if things got ugly. Fortunately for the ruffians, things didn't escalate that far. But now that I've had some time to think about it, I wonder if Hector shouldn't have done something about these thugs anyway.

These fellows were very clearly thugs. It was obvious, from both our encounters, that they teetered on the brink of violence and for no good reason. Since things never actually got violent between them and our party, the ruffians survived, and I'm quite certain that, in their fictional world, they will go on to harass, attack, kill, and rob other fictional characters, and, as a paladin, Hector probably shouldn't just stand back and let that happen.

But what choice does, or did, he have? He didn't have any legal authority. Trying to talk them out of being criminal thugs wouldn't have done any good. And, since they didn't attack anyone in his presence, he couldn't fight back in defence. If he wanted to stop them from attacking others, he would have had to take some initiative and attack them first. But attacking first is almost always morally wrong. I just wonder, would it have been wrong in this case?

I don't have a good answer for what Hector should have done. It's a moral dilemma that I'll probably still be puzzling over for a while. How sure do you have to be that somebody's going to hurt someone else before you decide not to give them the chance? How aggressive against you or one of your friends does a person have to be before you decide to stand your ground? Morally speaking, Hector may have done the right thing by keeping his sword sheathed and letting the ruffians live, but I'm not 100% sure that was the right thing to do, and I'm even less sure that that's what a paladin would have done.