Gloves are good and good ones aren't restrictive at all in my experience.  Find some uninsulated, unlined leather gloves with external stitching that fit, rub a bit of conditioner into them and ride.  Also, it'll be nice to keep the skin on your hands when you fall off rather than leaving it on the road.