Matthew Badger spoke out for the first time to ABC News, breaking his silence four months after the fire and describing his heartbreak.
He is the estranged husband of his daughters' mother, and he wasn't in the home at the time of the tragedy.
In a revealing interview, he talked about the days leading up to the fire and the questions he has now.
"I was with them for an entire week in my apartment," Badger said.He smiles when he thinks back to the precious memories.
"It was dancing," he said. "We had our own Christmas tree and that photograph was when we had opened our presents they were all very happy."
It was the day before the fire.
The big old house in Stamford was consumed by smoke and flames It was obvious Christmas Day chaos as firefighters tried to reach the victims.
The fire spread too quickly, tearing through the house and trapping and killing the people inside.
"You have heavy fire right above your head. Back out, back out," a transmission said.
When the smoke cleared, a family had been destroyed.
The girls - 9-year-old Lily and 7-year-old twins Grace and Sarah - and their maternal grandparents Lomer and Pauline Johnson were all killed in the fire.
The girl's mother, Madonna Badger, and her friend Michael Borcina escaped.
Authorities say Borcina left a bag of fireplace ashes near the back of the house.
That was the spark.
In an exclusive interview with ABC's Claire Shipman, Michael Badger still has questions any parent would ask after living through something like this.
"It was very difficult to see, why did this happen?" he said. "I mean, it doesn't make any sense, and I'd just seen them the day before."
Matthew Badger is speaking out now in part because he has started what he's calling the Lily Sarah Grace Fund.
Named after his daughters, he's trying to raise money for underfunded art programs in public schools.
The investigation into the fire continues.
Get Eyewitness News Delivered