The police department has engaged in "unlawful religious profiling and suspicion-less surveillance," the groups including Muslim, Jewish and Christian organizations and the NAACP wrote.
"Unsurprisingly, the NYPD's surveillance program has had far-reaching, deeply negative effects on Muslims' constitutional rights by chilling speech and religious practice and harming religious goals and missions," the letter said. "It has frayed the social fabric of Muslim communities by breeding anxiety, distrust, and fear."
The NYPD has said repeatedly it does not profile and its programs are legal. A spokesman had no comment on the letter Thursday.
The Justice Department has said it was continuing to review complaints received from Muslims and their supporters after a series of Associated Press stories revealed widespread surveillance by the police department.
In February, civil rights lawyers filed papers seeking a court order barring further surveillance of Muslims without evidence of crimes and a court-appointed auditor to oversee police activities that were "flagrant and persistent." They argued the department broke the rules of a decades-old court settlement known as the Handschu decree that restricts how the NYPD conducts surveillance. The department said it has not violated the degree.
The decree was put in place in response to surveillance used against war protesters in the 1960s and '70s. The decree was relaxed following the Sept. 11 attacks to allow police to more freely monitor political activity in public places.
A federal judge heard arguments from both sides this month and said he would rule later.