Aaron Phelps, pioneer settler and first postmaster of Milford, built this Greek Revival residence between 1836 and 1838. It was one of the first frame structures erected in the village. New York immigrant Ira Hibbard purchased the house and transformed it into a tavern in the early 1840s. It served the Milford community for many years as a tavern, stagecoach inn and mail drop on the Pontiac - Howell stage line. During the 1860s, the tavern was converted back into a private residence.