To add on to the other comments, the main reason you wouldn't use a swivel is because it will take away from your presentation. As mentioned earlier, a dry fly is meant to float on top of the water. A swivel would probably drag it under right away and hose your presentation.
Many flies (both dry and nymphs) are as small or smaller than the swivel. So, the fish sees your swivel and not the fly. Bigger bugs you can probably get away with it, but the best bet is to learn how to tie a good clinch knot or hemostat knot.
One of the things that took a while to understand is that presentation is the difference between catching fish and not catching fish.