Lily pollen can be a pesky nuisance! I would advise pulling the pollen off the anthers as the buds open. Preferably with a tissue to avoid staining your fingers. If you do have an accident, DO NOT, I repeat, DO NOT rub it! especially with a wet cloth! Just gently brush it with a soft paint brush or cover the stain with sellotape and gently pull off.