I'm not familiar with Vermont but Maine's a big State, it depends on the time of year and how far North or west you want to go. If I where coming to Maine to fish I guess I would probably hit up Baxter State Park, there are all kinds of small streams and brooks in the area. The Park also has allot of small walk in lakes with canoes available for a small rental fee, its a perfect area for float tubes. The West branch of the Penobscott is on the southern end of the park and the East Branch on the North. The only thing is reserve your camping area in advance
Here's a couple of shots from south branch pond campsite.