What is the speed like without the premultiplyAlpha? (I assume your test doesn’t need to convert from RGB to RGBA, so that’s all that really suggests itself, besides the decode itself.)
Does it gain anything from something like FastPremult in this paper?