Either I'm totally misreading this situation, or they amped up the level of AI cheating when I wasn't paying attention.
So, through various maneuverings, I managed to get an imbecile child appointed as the Fatmid caliph (playing as Axum). He's beset on all sides, a hundred gold in debt, and only has some 6k troops to his name after a serious of disastrous losses against the Seljuks, including the invasion of Jerusalem that he's still fighting. So, naturally, I declare war; it's a Holy War, looking to snatch one of the more southern Egyptian duchies. Immediately, and I mean IMMEDIATELY, he goes from -100 something gold to 250. His levies start rebuilding at a rate of something like 1000 every week, if not faster. I still manage to run him off at first, pushing his guys around, moving towards defeat in detail... for a while. But while I was off chasing a 4k stack down, another 3k moved in on the territory I was occupying. I go back for that, and literally in the month or so it took me to get there, the first stack that I was chasing hits 6k. I can't handle that, so I move up some mercenaries. In the time it takes THOSE guys to get there, before my eyes, the newly formed doomstack hits 12k. How does that even work? If I want to pick up new vassal or liege levies, I have to disband everything, then reform them, THEN drag them all together. This guy literally just reinflates his guys. I don't get it.
Yeah, you need to watch out for the mistake you made there.
The moment you declare war on him, he's now defending the realm against a heathen/heretic (which also allows nearby rulers of same faith to join the defensive side due to holy war), which boosts all of his vassals' opinion of him (+50 opinion).
Then, the specific vassals whose holdings you declared a holy war for are going to get another even larger boost in opinion towards their liege (+100 opinion)
On top of that, the AI might have been more likely, under even more threat, to give vassals titles or requests they asked for.
Thus, he now has new access to levies and tax that he previously couldn't have the chance to waste. He can both raise those individual levies if he so wishes, but more likely from your description, he auto reinforced them.
If you go to your forces/military page, it should tell you how much of each vassals' forces you have access to. As you suffer attrition, casualties, etc, your armies, which are made of different levies, will deplete. But every month, your standing armies suffer a certain amount of attrition, but at the same time, those depleted soldiers that "died" and are not a part of your standing army are now instead at the vassals' holdings recovering up to the max levy (including your percentage of those troops) (which is why sometimes after getting all of your levies you'll see there are more levies to recruit after a few months). Unraised troops will reinforce far quicker than raised levies.
When vassal opinion goes up, they're willing to give you a greater percentage of their troops, and it will be a sudden increase in percentage as the months ticks. So now your opponent should both have more available troops to actually raise, and has more troops from his vassals for passive reinforcement.
On top of that, increased opinion makes it more likely that vassals or perhaps even other rulers that dislike you might send the ruler a gift. And with his increased troop counts, they wouldn't see his cause as hopeless, but still judge him to be the underdog because of Seljuk numbers adding on to the "Character's enemies" troop count, iirc.
Add to the fact that this is the Fatimid Empire, which will have a ton of relatively powerful vassals (and those vassals whose land you're going far were less likely to have had their lands damaged by the other wars)....
Yeah, you screwed up there.
Btw, you can raise levies individually, per each of your individual vassals. Rather than raising all your levies at once, you can click the raise levy button next to the individual vassals on the war menu, and even specifically raise them in a county due to a holding that vassal holds in some other land (so the Duke of Croatia, if he had a holding in England, could raise said troops in England, although the counts underneath him would be restricted to their own holdings. When you raise all levies, it auto raises them in the capital holdings). On top of that, if you just click raise all levies, after your army has been depleted for a while, there should be some reinforcements you can raise.
Also, yeah, you should've avoided the holy war. Just attacking the realm with claims (and you can declare for multiple claims at once, so stack them up if you're going for the duchy), you would've only increased the opinion of whoever had his holdings claimed by you, not every vassal that thought you were an infidel.