I've swapped the barrels several time and even taking it off and putting it back on means I need to re-zero.
First, if you aren't confirming kills, you're just making your job harder. That said, your trajectory has changed significantly with the 22 pellets. It has gone from being fairly flat, to greatly curved. At 50 yards, I have watched my drop go from 1 inch to 4 inches using rifles producing similar power between 177 and 22 cal. Of course, pellet weight is a big factor, ie, heavier results in more drop. Basically, you are now shooting a much more curved trajectory, so figuring out POI at different ranges is even more critical. It's often the case that a heavier pellet will impact higher closer to the target than a lighter one, while at long ranges it will impact lower.What I would do is set up targets at 10, 25, 35, and 50 yards, and then figure out just what is going on, and then figure my hold over accordingly. Since you say the 22 seems more accurate, this is imo likely what you are experiencing.