Howdy GreenisGold420,
Some people can have a reaction to the tiny hairs that are all over the plant (and not just on cannabis plants either). If You have ever been around okra plants, they are covered with tiny hairs and one must wear long sleeves and gloves when picking it. An other thing to consider, how long ago was the mite problem? Maybe You are having a reaction to residual pesticide?
I had a similar problem some time back, I asked my Doctor if was psychosomatic, and Her said "No, first You must have a 'psycho' for it to be 'somatic'" LOL.
Seriously though, I would try a damp wash cloth on Your arms to remove any of the hairs or chemical residue. It could also be an allergy to some thing that You are using. What are the details of your grow, nutrients, growing media, etc.?