Scarlet Witch is a mutant and has more ties to X-Men than Avengers. I doubt Fox would allow that.
I absolutely hate Wasp. The cartoon ruined her for me. She Hulk is cool but isn't she really just Hulk with boobs and a vagina? Its sort of a been there done that thing.
Scarlet Witch and Quicksilver are both fair game, as long as you do not mention that they are "mutants" and Magneto's children.
Which is fine because at first they did not know that they were his children.