Au contraire: few people are would be able see what the brand of socks is until you're both in the bedroom about to be disrobed. Like boxers**, there's little point buying branded pairs to attract a partner because the only time they'll notice is when you've already sealed the deal. No one is going to poke around your dirty nondescript underwear trying to figure out what brand it is (just don't, you know, wear neon green christmas elf briefs or something equally silly that can be seen in the dark).
FWIW I do buy expensive socks but, again, by expensive I'm thinking icebreaker or rapha not hugo boss nor calvin klien
**Unless you're staring in a 90s MTV video where the imperative is to show off your LV/CK/ boxers.