Aside from this one fatal flaw, it does have a better interface than Plex, even as a pure media player. It is faster. DLNA allows you to browse both by metadata and folder structure; Plex only allows the former. And it has broader compatibility.
Now, if your only choice is between Plex and PS4 Media Player, I would pick Plex, because song order is just so essential. But if we are talking about all available solutions for doing in-home music streaming, I don't think Plex is particularly good. Whereas PS4 Media Player would be great if it just fixed this one issue.