The biggest issue in terms of getting a wide audience, I think, is simply that it's not a "comforting" show. It was never in a million years going to get NCIS ratings, because it doesn't offer that kind of...strings-free, uncomplicated escapism.
It's a show where there aren't heroes to cheer for or villains to hate; pretty much everyone is just a human being acting out of understandable motives with the information that they have. Every time you think you're about to get a handle on something, the show shifts the terrain beneath you, complicating things again. It's emotionally murky and uncomfortable in a way that likely hits home for lots of people in a way that the more fantastical conflicts of something like Game of Thrones probably don't.
It also doesn't glamorize espionage. These aren't super-cool spies with awesome gadgets. They're lonely, paranoid, miserable people who are in the process of being chewed up by the remorseless thresher that is the intelligence trade. They're incredibly sympathetic characters, but they're not "cool" characters. They're no Don Draper to (misguidedly) idolize here.
That, plus the methodical pacing, unromantic view of both the 1980s and American government and foreign policy, and so on pretty much means it was always going to be a relatively niche show.
