holding said Ag too firmly, not relaxed but TENSE as you try get on target with too much intensity and steadiness while looking threw an optic.IMO
What position are you shooting from? Do you shoot firearms, and if so do you have the same problem?
What kind of groups are you getting each way? I wonder if for some reason you are adding more variables with a scope that make it be harder to be consistent. Is your cheek weld the same with and without the scope? That is about the only thing that I can think of that would change your hold on the rifle from a bench. Are you getting a good repeatable cheek weld? Is your scope zeroed to point of impact or is it zeroed to preserve the point of aim? How high is the scope mounted? It may be a combination of things.