Who are you to decide what a "suitable number of players" is?
Three people for a B, for example, or barely even a single party for an A. More often though, it's when I know that a mark has only been barely announced, and only a very small fraction of the total hunting parties have arrived.
Basically, I'm making a subjective judgement call, but it's a judgement call I'm comfortable with. If somebody accidentally pulled it, then nobody should be upset if I pull it off early in the fight so it can be pulled closer to an earlier announced time. If somebody deliberately pulled it early, and especially if they weren't the one to find it and announce it, then I have no sympathy if they wasted a cool-down.
In either case, I'm trying to make things as good and as enjoyable as I can for the largest number of people, I doubt any GM is going to punish me for that.