In my opinion it would surely be easier to learn AE if you knew PS first.
There are some concepts like Layers, Masks, Alpha Channel, Adjustment Layers, and so on that are very similar between the two Programs.
You don’t have to be an expert in “heavy image manipulation”, but you’ll notice that AE walks a lot together with PS (even the communication between both is very good, and sometimes doing something in PS and exporting to AE is much easier than trying to do inside AE), and it’s better to know them both, if even if you’re just into video stuff.
Have fun!
https://cargocollective.com/danielcoutinho