To me, it looked like the fly was just floating down the current, making a huge arc and coming kind of back around to where we were standing.
Well, you've pretty much summed it up. "Swing" is primarily a term used when fishing in moving water, and just refer's to the arc as you've said. It's a technique used with a rather short length of line most of the time, otherwise you'll lose control over the line & fly and bites with subsurface flies would be difficult to detect. In some situations, "mending" of the line, which is manipulating it to maintain control & stay in contact with the fly is needed. When you see an angler flip the line to remove a bow & again create a rather "straight" line of direction, that is what is known as "mending".
However, there are other terms that could also be applied as you swing the fly. Most times it would be a "dead drift" where the fly just rides along in the current as a dead baitfish or insect might & any additional motion imparted to the fly is done by the current.
You could apply some additional movement too, by pulling the line, or by moving the rod tip so that the fly may twitch as it moves along, but not retrieve the fly in a manner where you shorten the line, until you reach the point where you need to recast.
Swinging is different than casting & stripping line to retrieve the fly and impart motion to it, which is what you likely do when there is no current.
Hope this helps explain the terms.