Personally I'd wait. But then I get the jitters from green fruit. Keeps me awake. Get enough magnification to see into the crystal balls on the end, make sure you have a few yellow and brown. Better yet, harvest in stages and see what you like the best. Then you'll know for next time.