Initially, authorities thought it was the case of a jilted lover, angry at an ex-girlfriend.
But it became clear later the killings - two of four that Gelman is accused of committing during a rampage that lasted 28 hours - had a much more complicated motive.
Victim Yelena Bulchenko, 20, had a longtime boyfriend, and some of her friends didn't know the suspect. They said the only relationship the Ukraine-born Gelman had with the woman was in his mind. If he snapped, they said, it was because he couldn't have her.
"He did not know her. He was not her boyfriend," said Gerard Honig, Bulchenko's boyfriend of two years who said he lived with the family. "She said he was creepy. He was weird, he was a stalker."
Authorities are still piecing together the case. The attacks that swept up strangers, as well as people in the suspect's life, prompted questions about what could unleash such a sprawling and sudden burst of violence from someone with no known history of it.
"It's a very bizarre case. It's hard to understand what the motivation was," said Louis B. Schlesinger, a forensic psychology professor at New York's John Jay College of Criminal Justice.
"We don't have enough facts yet," he said.
Gelman is accused of killing Yelena and her mother, 56-year-old Anna Bulchenko, who was at the home when he arrived, after killing his stepfather over an argument about the keys to his mother's Lexus, authorities said. He waited at the Bulchenko home for hours, police said. Yelena arrived home around 4 p.m. to find her mother in a pool of blood, and Gelman chased her outside and stabbed her 11 times, authorities said. He later ran over a pedestrian and attacked random people, police said At the time of his arrest, he muttered "she had to die" to police, but it wasn't clear to whom he was referring, police Commissioner Raymond Kelly said.
Gelman has been indicted on charges of murder and attempted murder but has not issued a plea, and was being held without bail.
His attorney had no comment.
Gelman said at the time of his arrest that it was a setup, and speaking to tabloid reporters from Bellevue Hospital where he was being held, he said: "Sometimes my mind isn't right."
Bulchenko's friends were surprised when they found out who was accused of killing her. Some had never heard of him. Those who did say he was obsessed with the woman, who worked as a dental assistant in Brooklyn.
"She wanted nothing to do with him," said her friend Aleksandra Ilyayeva. The New York Post reported that Gelman had a makeshift shrine dedicated to Bulchenko where he used to spray paint.
Honig said he knew of Gelman, but that he and the victim were barely friends. He didn't say how the two met or how long they knew each other.
Gelman lived with his family just blocks from Bulchenko in a close-knit community of Eastern Europeans. His family emigrated from Ukraine.
"I think he was just kind of obsessed with her, he'd harass her," Honig said. "But it wasn't something - she didn't seem in fear."
Experts say domestic violence killings are usually committed by someone who knew the victim intimately - they had a relationship or children in common. People who obsess from afar don't usually get physical.
"It's often considered more frightening, with the letters and flowers, but it's generally less violent," said Michele Galietta, a psychology professor at John Jay College of Criminal Justice.
There are too many unknowns right now to speak specifically about the case, she said. It's not clear if drugs, which contribute to aggressive behavior, played a role. It's also not clear if there was a history of mental problems.
Americans have become increasingly familiar with certain types of multiple killings, such as school shootings and the devastating domestic slayings social scientists call family annihilation.
But in the Gelman case, "here, you have someone who kind of can't be neatly slotted into one of those narratives," said David Schmid, a University of Buffalo English professor and the author of "Natural Born Celebrities: Serial Killers in American Culture."
After the stabbings, police say Gelman drove away and rear-ended a Pontiac, then stabbed the driver when he confronted Gelman, police said. The driver survived.
Police said Gelman left the man bleeding on the street and drove off in his Pontiac before running into 62-year-old pedestrian Stephen Tanenbaum, who died from his injuries.
He later abandoned the car and disappeared until just before 1 a.m. Saturday, when police say he hailed a livery cab and attacked the driver, then fled. Shortly afterward, he approached another car and stabbed a man inside multiple times in the hand before hijacking the car, police said. Both men survived.
Just after 8 a.m. Saturday, passengers on a subway in Manhattan noticed that a man on the train matched photos of Gelman they'd seen in newspapers. They notified police as Gelman jumped off the train at the West 34th Street station, crossed the tracks and hopped on another train, where he sliced a passenger, police said.
Officers were in the driver's compartment of the train looking for him on the tracks when he made his way up to the driver's door and pounded on it, "claiming that he was the police," Kelly said.
One of the officers threw open the door and wrestled Gelman to the ground, knocking the knife from his hand, Kelly said.
It's not clear when Gelman's next court date will be. He has not entered a plea.