I don't really think you can take any as just being randomly there. Even the "random" ones kind of build up the world in a way. I think the Littlefinger scene did a lot for his character, especially when you add it up with later scenes. It shows how he interacts with "his people." The rest of the lords and all that kind of get scenes to show how they interact when they have power, and usually with the people they are lord over. Those are good scenes. Littlefinger isn't a real lord. He manipulates things, but really he only has power over his whores in his brothel. I think by showing the dirty work that goes on there, they're showing where he is in the world compared to the others around the show, and how even though he's built up a lot he's really not anywhere.
In this show sex is used a lot of times for power. It shows who has the power over who. Sometimes it's women seducing and sometimes it's the men forcing. In this scene Littlefinger forgoes the "seduction." Basically, symbolically, I think it shows how he clings to power where he can. It shows how strong willed he is compared to some others that do wield what the world perceives as real power. I mean compare him to Stannis, who gave in pretty easily. His power over sex and the sexual is very different from all the other characters. He commands it, cold and calculating, orchestrating from the sidelines. It's just like how he works in the world around. Basically, I think while the rest of the sex scenes delve into the characters themselves having sex, with Littlefinger it's a bit different. It's telling about his character that what should be his "sex scenes" have him not having sex, but directing others and literally owning others' sex in a way.