A coalition of families and civic groups calling itself New York Neighbors for American Values held its first rally near ground zero Wednesday.
The families support the mosque and want to remind the public that Muslims were killed in the attacks on the World Trade Center.
But they also admit it's difficult to disagree with other 9/11 families who want the center built somewhere else.
"We feel pain too, we just feel it differently," mosque supporter Donna Marsh O'Connor said. "We're not opposed on every issue, but on this one we really are. And I'm sorry, but I can't let this one go."
O'Connor lost her daughter on 9/11. But as an American, she says it's important to affirm American values about freedom of religion. But she admits it's painful to disagree so publicly with other family members.
Jim Riches, who lost a child on 9/11, said it's not about American values and freedom of religion. He says it's about a little sensitivity.
"We just would like them to be a little sensitive to our feelings," Riches said. "We'd like them to sit down with us, compromise and maybe move to another location not 500 feet from ground zero."
The coalition also criticized what it calls irresponsible politicians for using the controversy as an election issue.