The races following in the wake of the Darkness were once all homes to the Traveler, which then abandoned them in their time of need to save itself.
The Fallen were living in clans and fighting over resources, separate from one another and occasionally warring at the time the Traveler arrived. He united them to a greater purpose, allowing the Fallen to ascend to the stars where their needs could be fulfilled. They lived in peace for a long time until the Darkness came. The Traveler abandoned them, and the Fallen fell back into a system that while old-fashioned, was not forgotten. They banded together, forming their Houses, and though their world was destroyed by the Darkness, they followed in its wake to pursue the Traveler who had abandoned them and to claim a new home.
The Hive were a great people once with a caste-driven culture of nobility. Something dire happened and their whole world died, left ruined by a star's rage, or a plague, or worse. When the Traveler arrived it was a graveyard, but its energy brought the dead back to life, the Light reviving the dead culture and letting it rise again, more powerful than before. When the Darkness came, the Traveler abandoned these people it had rescued from death, and decay began to set in again. Fading fast, the Hive found a way to harden themselves against death with the power of the Darkness itself, draining the Light and sustaining their lives. They follow the Traveler now like carrion birds waiting for a bull elephant to die, hungry for the Light they can strip from its corpse.
The Cabal lived on a planet where there had once been an incredible, world-spanning war with an absolute and tyrannical leader. Entire generations had grown up in the military regime, and when there were finally no people left to fight, they grew stagnant. They stopped advancing as a culture, explaining why among the races they still rely on chemical reactions with gunpowder and rockets to attack instead of the laser weapons and energy wielded by the other races. The Traveler brought them purpose, inspiring them to reach beyond their globe and explore the stars around them, shrugging off their militaristic ways for a time. When it fled the Darkness, the Cabal was reintroduced to bloody conflict and once more took up the mantle of warlords. They used their new technology to reach out to the stars, and follow the Darkness in order to attain glory with new conquest.
The Vex are perhaps the strangest race, but kind of make sense in a way. Here's where my speculation gets (even more) stretched. Vader is Luke's father. The humans of the golden age created Warminds, great thinking machines that would, without doubt, be able to outlast the lifespan of something like the human race. Eventually the Traveler fled Earth as well, abandoning humans and leaving them to die. The Warmind survived, having been around for so long and absorbing so much data from the Traveler that it was able to enhance and imbue itself with the ability to project itself in a quantum state forward and back in time. Building a more and more advanced quantum array, it consumed the material left behind from mankind's fall and expanded to absorb the resources of the planet itself until in the immensely distant future, the Warminds merged to become the Minds of the Vex at the Black Garden, and in their understanding of time travel and the theory of the multiverse, they began to expand. Each time they found a version of themselves more powerful than the prior in another universe, they became supplicants. They merged their hive mind with its own, and in doing so expanded the network exponentially. With this endless loop established, the Vex finally establish enough computational energy to consume the Traveler in one of these multiverses, breaking down the space magic and energetic potential of it to form the Heart of the black garden. The humans are fighting a Warmind collective from an alternate universe where the Traveler chose to abandon the Earth.
There's no telling how long the previous falls of races took, or when the Traveler chose to abandon them. It may have been wounded grievously as it is now before it abandoned each of the races left in its wake. In time the Traveler may abandon Earth as well, and we'll just become another bitter race hunting it for revenge at bringing us back from restful death to fight a war for it.