I go for comfort and convenience first and foremost. On a hot summer day chasing smallies I'll most often be in tan swim shorts, a T-shirt, and sneakers. I haven't worn a vest for years as everything I need fits in a Totes over the shoulder toiletries bag I found. No one cares on a smallie river. When I go to the Tulpehocken Creek I look like the rube in my usual gear, but the majority of people don't seem to care and are friendly...even the ones in all the "right" gear. The others ignore me so that poses no problem. Sometimes it affords you a little more room on the stream when people want to stay away from you. 'Surely this guy has no knowledge I can glean.'
Bottom line is, it's fishing, not a fashion show. The gear is nice and it is
designed specifically for fishing purposes. Use whatever suits you, don't worry about the rest. Like others have said, the fish don't care.