If you could always tell which one is an illusion right away, he would be terrible. And those illusions still hurt you, too, which is part of why he's frustrating. It doesn't help that we keep getting randos that burn their abilities on the illusions during team fights. Obviously it's easy if it's the situation you described, lol. But those precious seconds before one Samuro does something different from the others are fight-determining.
If it's Ranked, then it's not as much of a problem because I can be last pick. But it's annoying in Unranked when I pick Stitches or Dehaka and then they go Samuro.