Should controversial speakers (on whatever subject) be allowed on college campuses?

Why should speakers not be able to come to college campuses?? Especially if people want them to come and speak in front of their peers or group to help give their group better understanding about whatever subject the speaker specialized in. I guess you might have some people who dislike the person and their viewpoint which cause issues or it might not but just because there are some individual might cause some trouble but it should be taken out on the speakers just because they might have quite of few people that might disagree on their viewpoint on things.