Built in 1865, the property has had only four owners during its history—and still boasts many of its original features.