This building was erected for Jackson dry goods merchant Horace Ismon in 1853. Ismon had begun his mercantile career in Paw Paw during the 1840s. Although established in Jackson, he continued his Paw Paw ventures, engaging in the woolen trade, building a tannery and grist and saw mills, and financing the Paw Paw Railroad. Ismon was a founding director of the Jackson County Bank in 1872. Having no children of their own, Ismon and his wife Clara, mentored youth interested in the mercantile trade.
A lavish gala attended by three hunded guests and future governor Austin Blair celebrated the opening of the Ismon Building on February 22, 1853. The building has housed a variety of enterprises, among them Horace Ismon's dry goods store, a pharmacy, and a home health service. From 1927 to 1972, the Fairy Garden restaurant occupied the upper floors. It believed to have been the first Jackson establishment with air conditioning (installed in 1933).