The tighter you make the leg channel, the more friction the pad experiences when it tries to rotate. This isn't an issue when you butterfly, because the shape of the pad, the force of your body, and the contact with the ice all combine to rotate the pad around your leg. But when you lift your legs up, the only thing acting to re-center the pad on your leg is gravity.
If you want the pad to rotate back when your leg lifts up, I'd suggest loosening the calf straps, and maybe the professor strap as well.