Little Brandon Holt, from Toms River, died Tuesday after his condition steadily deteriorated following the incident, which happened around 7 p.m. Monday in the backyard of the younger child's home.
Prosecutors say the 4-year-old went inside his home and found the .22-caliber rifle, which accidentally discharged and struck Holt, who was about 15 yards away at the time.
Holt was flown to Jersey Shore Medical Center, where he was initially reported in stable condition before being downgraded to serious. He was then listed in critical condition in the intensive care unit before it was reported that he died around 5 p.m.
"This never should have happened," the victim's uncle, Daniel Walkins, said. "It's horrible."
The shooter's parents are the ones who called police. That boy's identity has not been released.
Toms River Police Chief Michael Mastronardy called the shooting a "tragic incident" that has affected the whole community.
"Right now, we're keeping this child in our prayers, and I ask everyone to do the same," he said.
The shooting occurred in what neighbors say is a quiet residential area of the Ocean County community, at a home near the edge of a cul-de-sac.
"I'm sad for the children involved and their families, but I'm angry with whoever owns that gun and allowed a little child to get hold of it. A 4-year-old can't load a gun," said Debi Coto, who lives a few doors down from where the shooting occurred. She said the families of the two children involved had not lived in the neighborhood for too long, and she did not know them well.
Coto said she was home when the shooting occurred but did not hear the gunshot. She said it initially wasn't clear what had happened, but then neighbors saw police bring out the gun.
"I had just been telling my sister how nice it is to see kids playing together and enjoying themselves, and then this happens," she said.
Coto said the 4-year-old's mother seemed "very upset" about what had happened.
Investigators won't say how the child got the rifle or who it was registered to.
The shooting came just days after a 4-year-old boy in Tennessee grabbed a loaded gun at a family cookout and accidentally shot and killed the wife of a sheriff's deputy.
Investigators said Wilson County Deputy Daniel Fanning was showing his weapons to a relative in a bedroom of his Lebanon home on Saturday when the boy came in and picked up a gun off the bed. Sheriff Robert Bryan said the weapon discharged, hitting 48-year-old Josephine Fanning, who died at the scene.
The accidents happened amid debate over gun control laws in the wake of December's Connecticut school massacre.
(The Associated Press contributed to this report)