A woman living on the coast regularly rows across to visit a poet renting a house, and it might have fantastic/paranormal events

This might be a fantasy or ghost/paranormal story taking place sometime in the early 20th century? I recall a woman living on the coast… either with a sibling or her family. They may have been wealthy, or lived adjacent to wealth, and she would take a rowboat or small sailboat across the water to visit a poet renting a nearby home/cabin.

I feel like maybe it was set in Long Island NY or New England. It could be slightly earlier than above, or slightly later, but probably not contemporary to now.