You have a system specific problem of some sort.
Ive just just a test, with a heavy payload ti put it under stress.
FG HUD Active so you can see my stick and pedal positions.
Hover:-

Cruise :-
Both with SAS/AFCS active.
If you have something influensine your inputs then that will also have an effect on the AP when its active too.