Not even 15 minutes outside of New Paltz along Route 44/55 is the Mohonk Preserve. It encircles the Mohonk Mountain House property, and you can get day passes or annual passes, (highly recommended if you plan on going more than once or twice). There is great mountain climbing and bike riding and hiking along the well maintained carriage trails.
Best of all, just down Clove Road, there is a swimming hole called Split Rock. Whoever named it was a great fan of truth in advertising! It is a very large rock that is split with a waterfall feeding into it. The water is between four and five feet deep - deep enough for swimming, but DON'T dive in!