He's not a failure of a villain, and I don't know what kind of expectations you have from a country if Latveria is a "hole in the ground" for you.
First, let's talk Latveria.
It originally was a tiny Eastern European shithole, poor and ravaged by years of war. Then Doom rose to power and cleaned it up, eventually making it a super power. In the current Marvel Universe, Latveria is one of the world's strongest countries economically and militarily, it easily measures up to the real world's US or China, and has only the fraction of their size and (human) manpower.
But Doom did not do this based in slavery. He created an army of robotic servants that keep the peace and infrastructure of Latveria smooth. His people love him, and in most depictions of Doom, he cares about his country and his people as well. He never endangers his population directly, only using his own Doombots (and himself) for his operations.
Now, Doom himself. Yes, he's a mustache twirler and has an ego the size of Jupiter, but he's the epitome of the mustache twirler and egomaniac. Doom was one of the pioneers of that particular type of villain within comic books. He is driven by his hatred for Reed Richards at first, but eventually this turns into his belief that his sole purpose is to govern Earth and lead it to glory under his rule. This goes to the point that he is willing to work with his enemies, even the man he so utterly hates - Reed Richards - to preserve the planet from destruction in the case of bigger threats. After all, if Earth doesn't exist anymore, what would be there for him to rule?
His ambition and his desire for revenge made him become a super genius in basically all fields of science. He is on the same level as Richards in basically all regards. But he also dabbled into the fields of the supernatural, becoming a sorcerer almost as powerful as Dr. Strange. Speaking of which - he teamed up with Strange to go into hell and beat up Mephisto to save the soul of Mama Doom.
And most of all, he just doesn't take any shit from anybody. You can be a peasant or a cosmic entity, Doom will shittalk you, and then continue to do whatever Doom pleases. Heck, when Loki visited him, he poisons the food just for shits and giggles.
Doom isn't a failed villain, over the decades he's been built to be one of the most powerful men on Earth, and rightfully so. He is insanely intelligent, has vast amounts of knowledge, has nigh limitless amounts of tech gadgets, is one of the most powerful magicians of the planet and heads one of the most advanced and powerful nations in the world, a nation that he himself brought to where it is now.
And to top it off, he
knows all that, and that's why Doom doesn't take shit from anybody.
