Citadel spray primer isn’t even primer to be honest … it’s just spray paint. Non-adhesion is a common issue with it outside of Games Workshop models (they use a different plastic mix than most board game pieces do). That’s probably the problem right there.
Get real primer. I use cheap $2 primer from WalMart and it works perfectly. Others swear by automotive primers. Some people use Krylon Fusion as it bonds itself to plastics even better (but is not actually primer) - I’ve used it with good results but still stick with the cheap-o Walmart brand primer.
Once painted and you’re happy with the results, use a clear acrylic coat and your paint job will remain protected for years.
Alternately, use a clear non-acrylic so you can, if you want, remove the paint at some point in the future (acrylic is a pain to remove).
It’s a good idea to give the pieces a light scrub when washing them, but not always necessary. This is just to remove any leftover mold release residue (which is in essence an oil and will prevent stuff from sticking to it).