The house was built in 1851 by Asa Tift, a marine architect and salvage wrecker, and became Ernest Hemingway's home in 1931. The house still contains the furniture that he and his family used. The cats about the home and grounds are descendants of the cats he kept while he lived in the house, including many extra-toed (polydactyls), like the one Papa Hemingway loved.

In the living room of the house there are some of the furnishings that Papa's wife Pauline collected while she lived in Paris and had shipped to Key West when she and Papa bought the house. Her chandelier collection, for example, replaced all the old ceiling fans. Her chest-on-chest is a 17th century Spanish, made of Circassian walnut. When traveling, wealthy Spaniards could take their papers and valuables with them safely stored in its removable and lockable top. Pauline used the chest as a writing desk. Papa himself loved art and on the far wall of the living room you can see a view of St. Paul's Church painted by local artist Eugene Otto that was a part of Papa's collection. A large lithograph shows Gregorio Fuentes; Fuentes was the cook and mate on Papa's fishing boat Pilar for more than 20 years, and became a good friend. In the room across the hall from the living room the is a red leather Cardinal's chair by the door - it is reported to have been used as a prop in the Broadway production of The Fifth Column, Hemingway's only full-length play.

Given the Hemingway's love of Key West, it is not surprising that many people believe that the couple's spirits still inhabit their beloved home and grounds there. It is purported that Hemingway often quipped to his friends in Key West that it was there, in that house, that he would be spending his afterlife. Many believe he is.