Because apart from "getting them together" there's not a whole lot to care about. The "plot" in Avengers was okay because like you said it served its purpose, but that doesn't necessarily make it good. I warmed up to Avengers a little while after I saw it because it was just a harmless action flick that did precisely what it sat out to do, but it left me with little else.
Never in a million years would I say that it's even remotely as good as ASM. ASM had me gripped, I was into the characters, into the story, and thus into the action and emotional elements. Avengers' action was just kinda there, although well directed and structured, but I felt zero attachment to it. I'm not saying Avengers needs to be some emotional film with any kind of depth to it, but right there's my argument in the matter, by its nature it had nothing to it at all whereas Spider-man did.
And yes I get it that Avengers was basically this big sequel to several pre-existing superhero films, but I even found those to be pretty empty. I rewatched Thor the other day, and while I still enjoyed it, it just felt like some half-baked setup film, like okay there's Thor, you know who he is now yadda yadda now wait on The Avengers! I also felt that way about Captain America. Iron Man 1 was okay but Iron Man 2 was terrible. I didn't feel this way about ASM. It felt like its own entity, a complete, standalone film-- and much stronger because of it. I would say that about the Raimi films as well.
For me you have a bunch of entertaining but throwaway Avengers related films (including Avengers) but then the Spider-man movies (both old and new) and Nolan's Batman films are simply way ahead of them.
