Necalli had two interlinked story arcs. The first began by addressing his motivation: Devouring Nash's wandering soul after he was killed. The Illuminati (perhaps not even aware of his existence) either accidentally interrupted him with their final experiment that temporarily brought Nash to life, or Necalli was drawn to Nash's soul because he hadn't properly "moved on" as the Illuminati worked to perfect their technique. Necalli was never part of their plans, as this occurred only in Nash's head while dead, sort of an afterlife "dream". But this was merely to establish that Necalli is always around, taking the lost or wandering souls of warriors. This is touched again on his final appearance when confronting Ryu: throughout the story Ryu was being tormented by the Satsui no Hado which essentially puts his soul at risk, thus becoming a lost soul like Nash's. Necalli was only driven to devour Ryu if he succumbed to the dark power, which is why when he finally mastered Satsui no Hado and restored the strength of his soul, Necalli was no longer necessary.
The second story is Bison's secret plan. Bison was well aware of Necalli's legend (as told by Dhalsim): putting the world in crisis would make Necalli manifest in the real world. Bison has pretty much always wanted Ryu's body for its power and the ability to contain it properly. The Satsui no Hado is established as linked in some way to Psycho Power by several characters over the course of the series, and of course Bison wants it and a body that can use it. By baiting out Necalli, Bison knew that Ryu (as well as himself) would be one of his most tempting targets and in turn would force the suppressed Satsui no Hado to completely take over. Bison later defeated Necalli to establish that if Necalli devours Ryu and absorbs the awakened evil power, than Bison could easily then take it for himself directly from Necalli. However, Ryu overcame the temptation and evolved to an even stronger power, which triggered the final confrontation with Bison. Bison was excited to learn there was an even more powerful energy than Satsui no Hado and wanted to see it for himself, as well as finally take Ryu's body, a vessel strong enough to contain it, for his own. So Necalli, being a neutral force of nature, was merely a pawn in Bison's plan to make Ryu "ready" for Bison's eventual ascension one way or another.
As for Balrog's prologue story (the only time Necalli actually appeared in one except for the Warrior Priest in his own prologue), this event occurred before operation C.H.A.I.N.S so canonically this is the first time Necalli appeared before street fighters in the real world. What we don't know is who Necalli was actually drawn to in this scene: Urien, Balrog, or Ed. As his later appearances would be around dark energy users, my bet here is that he was after Ed, being an echo of Bison. Urien's dialog here was pretty flippant and had a hint of urgency, he probably thought Necalli was just some sort of assassin (and given how weird Illuminati experiments were, didn't think anything of the mud man thing).
For me it makes most sense that Necalli was drawn to the real world by the threat of Psycho Power (and its cousin Satsui no Hado). Bison defeated Necalli so all he could do at the end was track down the vulnerable Ryu, until this was no longer necessary.