The dog known as "Bones" was taken over the weekend.
He was sent to Toledo to be rehabilitated after his owner was charged with killing his roommate on the Upper East Side back in October of 2011.
Police first thought the dog was involved because he was covered in blood.
The shelter has 15 other dogs, but Bones was the only one taken.
Get Eyewitness News Delivered