I've used from 3 wt's up to 8 wt's for Smallies, based simply on the waters size & the size flies I want to use. Generally these 2 considerations limit the size of the fish, and even the occasional bruiser is not a problem, no matter what rod wt I might be using. That's fishing!
There's really no right or wrong to it & often it's just a matter of personal preference. For the majority of the places I've fished for them, I go with a 8'6" 6 wt that I've had for many years. (Cabela's Gary Borger Signature model, no longer made.)
No matter what you decide to try, there's always going to be conditions, situations & fish size that won't fit the rod. It's always a compromise no matter how much thought we put into it.
Frankly, you would probably be fine with what you have. Of course, sometimes we all need an excuse to buy another rod!