The departure of the British at the close of the revolution did not end the upheaval in the life of the Mansion. Serving as an inn for New York City-bound travelers, ownership of the house passed through many hands. Finally, in 1810, the Mansion was restored to its original purpose as a country house by the French emigrant Stephen Jumel and his wife Eliza.
Stephen and Eliza added new doorways and stained glass to the facade of the Mansion. As regular visitors to France, they furnished much of the house in the French Empire style. Many of those objects, including a bed said to have belonged to the Emperor Napoleon, remain in the Mansion today.
Stephen Jumel died in 1832, and Eliza, then one of the wealthiest women in New York, later married the former U.S. Vice President, Aaron Burr. Their marriage lasted just two years. Eliza retained ownership of the Mansion until her death in 1865. After a twenty-year court battle, which was finally settled by the U.S Supreme Court, the property was divided and sold.