I'm not sure how to answer this with out sounding condescending or disparaging, but here it goes.
(the following is not meant to suggest that you are bad at fly fishing-- actually just the opposite-- just some offerings on what might be at work behind the scenes. This is a different game than dunking worms, or throwing power bait.
Firstly, I doubt that your dad was pulling them in by the dozens and you didn't get a single hit. I suspect what happened is that you maybe missed a few (or even many)---
This is likely due to a fly that is too big (ie-- they couldn't even really get the bug into their mouths-- what size was the wooly worm?) There is a fine line with bluegill fly sizes-- too small and they will inhale and likely be killed every time, and too big will get hits, but no hook ups. I have had 'gills actually smack at a size 2/0 bass slider-- they WILL hit BIG bugs.
What was the water like? Structure and cover in the water? did you feel any kind of resistance on the fly and possibly mistook it for a snag or drag through some weeds? Sometimes they hit subtly, sometimes voraciously. Sounds like they might have been a little gentler this day. Easy to miss them. I have watched a gill take my fly in crystal clear water and missed him... They can be VERY subtle.
As for suggestions:
Try the foam bugs that you have. The rubber legs will incite some strikes, and they will float so that you can see the take. As for the leader and the thickness-- just tie a double surgeons knot into the one you have, and use a length of 4# mono to connect to your leader (as 'tippet').
Other flies you could try are really anything with rubber legs (girdle bugs, rubber leg buggers, hopper patterns, spiders), some smaller poppers (size 6 or 8-- don't go much smaller than sz. 8 if you don't want to kill a bunch of fish.), you might do very well with some nymph patterns-- toss em out and (in still water) make the slowest retrieve you possibly can.
Hope this helps-- stick with it-- you will get fish before you know it. Just pay attention to what you are doing, and when you get the fish, you'll know how to do it again.