You are remembering it wrong OP, I watch and make comparisons for a hobby, I know for sure the three machines are not that far apart, and rarely GC and XBOX completely trump it. PS2 usually is just about the same, sometimes it has "enhanced drawing distance", at the expense of pixel crawling effects, due to the lack of Mip-Maps (which requires more processing power, mind you!), sometimes there are extra special effects on any of the three machines (Frame Buffer effects favour immensely PS2, due to the 48 GB/s embedded ram), usually textures are better in color and resolution on XB and GC , but many times they become blurry in the distance, PS2 stays pixelated but sharp through most games. It's true that some GC and XBOX games had some futuristic/alien graphical effects which you rarely see on PS2 (or maybe never at all), but if you factor in that the sysyem eventually had its own share of secret weapons graphically, and that the two best looking games of that generation, Gran Turismo 4 and Silent Hill 4, are on PS2, there is no doubt it aged almost as bad and as good as the other two.
Maybe, I say maybe, you aren't enjoying the games in the proper way: are you playing PS2 on a CRT with a good cable? That makes or breaks the deal for PS2, quite a lot honestly, I've tried all combinations (PCSX2 + OLED TV, PCSX2/PS3 + LCD TV, PS2+ VGA CRT MONITOR, PS2 + HD CRT), and honestly I think only old school 480i CRT TVs really show what the system is capable of.
The other two gaming system might look a bit more flat on CRT, but they lose less warmth and vibrancy when displayed on other kinds of tv sets. That and better framerate are the reasons to buy the other two systems, more usabilty all around, but visuals-wise 99% on the games were almost the same on the three machines.
SOURCE: Silent Hill 3 is my love (since last year) , I play it every day/week on a CRT, and the culprit of my love is wondering every day "Why are people stuck in 2003/2004, when everybody thought PS2 never had a chance in high profile graphical effects compared to GC/XB, and they didn't even give a glimpse to modern games on PS2 like Path of Neo (which had lots of missing effects on XB an Pc) , Tourist Trophy or Medal of Honor Vanguard?", and then I rewatch the SH3 pre-final boss cutscene, where Valtiel climbs down the wall, and looks better than many of the games in the last 20 years.