I don't think we should ever handle a fish if we can help it, especially in very cold Winters. Land the fish in a net and remove the fly with the Waterworks Ketchum Release tool. The regular tool works down to size 18 and the Midge tool 18 and below.
No need to remove your gloves.
Waterworks Ketchum Release