It is not uncommon in church to hear that church is not a building, but rather a group of people who are learning to worship, serve and love God together.
Yet the building must signify something important for church goers and those that practice any faith, as it is the location where that education, worship, serving and loving happen. It is where a gathering of people with common beliefs meet, connect, pray, and likely give business to centers and restaurants nearby. That is to say, the presence of a building where faithful people gather has the implication that the same people are gathering in that neighborhood, living there, volunteering there, buying food or engaging in activities there just outside of that place of worship. This might sound highly beneficial to a given neighborhood, but it is not news to anyone that the opposite perception is prevalent, especially when it comes to mosques.
It is not the building that makes others feel fearful. It is not the additional traffic that induces fear, though it is notably a more common protest than feeling afraid, as indicated by Pew. When the neighbors' fear does affect the decision to add a mosque or other place of worship, it is fear of the unknown - unknown beliefs, unknown people, unknown culture.
What is the answer to this fear? Is it better not to invite such buildings into areas with people unwilling or unprepared to welcome a new group of faithful people to their neighborhood, or is it more worthwhile for that group of people to join the neighborhood and allow people's fears to be replaced by understanding over time, regardless of how bumpy or lengthy the road may be?