Why does nobody listen to me when I say that Bran is going to turn out to be the Great Other who controls the Walkers or some crap like that.
That's been my working theory since undertaking my re-read. It seems to make sense in the "Ice v. Fire" context, as it wouldn't exactly make sense that the base of operations for the last bastion against The White Walkers would be deep within their territory.
The CotF seem to have no incentive to protect the world of men, seeing as they've been almost driven to extinction at the hands of humanity, along with most every other creature which might threaten their dominion. While they might have been allied with the First Men during the original Long Night (at which point time The Pact still held), the burning of the weirwoods and domination of the forests the Children once called home in the Southron lands, in addition to the slow expansion of wildlings into the Northern territories, might have motivated them to enlist the aid of The Other in culling humanity. In the process, they've begun exploiting the weirwood network, which itself is a neutral force to be used by either side.
The wights and White Walkers themselves are more of a tool for The Other and the CotF to exploit. They're like a biological weapon that they can deploy in a certain region but can't directly control, which is why the cave the Bloodraven is in is warded to keep them out.
It makes me wonder if there's another faction of the CotF currently residing on The God's Eye: one that might be more benevolent towards humanity.