Shims! Those are no fun. It's always like doing the job over and over if you don't have the right shims you need, and you have take them out just to figure out what shim you need, whether you can move one from one to another position when possible. Then wait for ordered shims and do it all over again. Went through this on my first gen DOHC Suzuki 4 cylinder bikes.
Do you have the cam follower depression tool? I bought the tool and it was a pain in the neck rotating it into place. Stiff springs.
Yes. I agree it can affect valve timing, especially at low rpm, which is wot you are seeing.