Criminals are people who the player can capture. If the player defeats the criminal, they will offer a special item to the player. The player can either choose to return the criminal to justice for a pre stated reward, or accept the special item. If the player has all the special items, the criminal will automatically return to face justice. Criminals often have names like Mr. X. Farthingsworth etc. Governor's daughters will tell the player about them and the barkeeper in taverns will guide you to them if asked. If the criminal is returned to justice, governors will state that when you talk to them, and may even promote you if it is important enough.
Fighting a criminal is exactly like fighting an Annoying Captain, complete with the barmaid smashing a bottle over his head.