There's tons of reasons to fly fish and I certainly won't get into them all, but here's my two cents.
First off, 99% of fish that can be caught on conventional tackle can be caught on a fly rod. That's half the fun right there. I enjoy going after anything that bites, but my niche lies in catfish, drum, and carp. Rough fish mostly. I like to change it up sometimes and catch bass and panfish too. The fight of a fish on a fly rod is amazing. It's hard for me to describe, but it's awesome. It can also get challenging, but the rewards are great. Any kind of fishing can be challenging sometimes though.
I know Bass Pro had some good sales going on a while back and they still might be. Don't quote me on it though. I haven't checked around. BUT, they do have some decent combos for around $100.
I would recommend starting with a 9 foot 6 weight rod. You can use a 5 weight too, but if the fish are good-sized, a 6 wouldn't hurt. I wouldn't go over a 6 until you get more familiar with the rhythm of fly fishing. You'd be surprised how big of a fish you can comfortably land on a 6 weight. It will handle bass, panfish, carp, cats, etc.
Before you hit the river, I'd recommend hitting up some ponds or small lakes if you can. Bluegill and bass are always usually pretty cooperative and great for cutting teeth on. Plus, they're a riot on a fly rod.
Most lines these days have welded loops on them. I use them and they're great, but I believe that doing it the old-fashioned way with nail knots is the way to go when starting out. Learning all of your knots right off the bat is a good insurance policy. I found out personally that loops do fail. I had a fish take my loop over a rock. I tied a quick on-the-water nail knot and was back fishing in a couple minutes.
The other fellas here have great advice and they'll steer you in the right direction.
---------- Post added at 12:05 AM ---------- Previous post was Yesterday at 11:53 PM ----------
I noticed a couple of things and I'll add a bit more.
Backing is usually made of Dacron and comes in 12,20,30,or 50lb. 20lb and 30lb being the most common.
Don't try to cast a plug on the end of your tippet. The weight of a casting plug will either break your plug off, or you'll doink yourself in the head. Remember, you're casting the line, not the fly. Most flies are almost weightless.
For tools, I'd get a pair of forceps for removing hooks and line nippers for cutting leader, trimming knot tags, etc. A lot of folks pay big $ for fancy nippers, but a cheap pair of fingernail clippers from the dollar store works wonders. I've used the same pair for years.
If you want to use silk line, expect to pay $$$. The ones I've seen are quite a bit more than top dollar lines. Plus, they require extra care and attention. But, they last a long time from what I've been told when given proper TLC.