What should I do in this touchy situation?

I promise I am not a bigot, so hear me out before you condemn me. I am a landlord in Biloxi, Mississippi, and have long served a very traditional, southern community. Recently, a ual moved into the apartment I own. He is a decent fellow, being an istant minister at the local Catholic or one of them other ones, church, and he always pays on time. However, his loud music and rowdy buddies sometimes disturb the other tenants. Further, most of my tenants agree he is just a little too "colorful" and if he could just straighten up a bit, he might be acceptable. However, I have heard him and his friends respond poorly to such suggestions. Would it be acceptable to evict him if this continues? Once again, I don't want any problems, I just want my town's way of life to go on as usual.