Your casting is probably spooking the fish, or at least putting him on his guard. make you cast and let the fly sit.... minutes if you have to. Let him swim off and come back. He needs to calm down and forget that there is a "lure" there.
Then try little twitches and see if that will get a strike.
PS. Not to say that there is anything wrong with your casting. Sometimes even the most delicate cast scares the fish off. Trust me, I know.