I moved your post to the PCP gate, so you can get more answers to your question.
What you are describing is called a shot curve. It can be eliminated with a regulator.
...Towards the middle and end of your string you have more valve dwell and you may be belching more air out behind the pellet after it has left the barrel causing poi shift.
My bet is mechanical bending of the tube when the pressure changes, influencing the barrel.... If not that, then the differing pressure pulse is changing the barrel harmonics and causing the POI shift.... At the beginning of the shot string, the pellet is getting a short blast of very high pressure.... at the end of the string, even though the velocity is the same, it is getting a longer, slower push of lower pressure air.... more of a "shove" instead of being "hammered".... Bob