The fire began early just after 3 a.m. at 135 Statesir Place in Middletown.
The Monmouth County Prosecutor's Office says improperly discarded smoking material in a metal waste basket ignited the blaze.
The victims were identified as 63-year-old Sheridan Banovich; her sister, 60-year-old Denise Dusold; Banovich's daughter, 23-year-old Deidre Banovich, and the young woman's boyfriend, 25-year-old Anthony Cadalzo.
Banovich's husband Stephen, also 63, was the lone survivor of the blaze.
The police chief also said the home did not have working smoke detectors. The home had some minor renovations and painting done recently, and the detectors were never put back up.
The damage to the home was extensive, and the fire moved so quickly and with such force that firefighters were unable to reach those trapped inside.
When Newscopter7 arrived on the scene Tuesday morning, the fire had already been knocked down. But its impact was clearly evident.
Firefighters say the blaze quickly spread throughout the house.
"It was terrible," neighbor Kristen Dressler said. "We woke up at about 3 a.m. hearing screams, and my husband heard popping, cracking. He thought maybe it was kids playing outside, but quickly realized it was something really serious."
"My wife saw it from the bedroom, because she heard somebody hollering," neighbor Ted Magee said. "So I guess it was him, trying to yell to his wife to get out. But it was too late at that point."
When firefighters finally put the flames out, they found the victims inside. Neighbors were overwhelmed at the loss.
"We'd go to their house for Halloween," one neighbor said. "They were really, really nice people. Always said hello. His sister-in-law would walk the dog all the time, so we'd always say hello. She was very nice. It's just really sad."
Mourners made a makeshift memorial in front of the charred remains of the home Tuesday night.