I suppose to put my feelings on this into words and try to explain things, I'll say that Kumiko and Reina's relationship is one of the more unique relationships I can think of in an anime series in quite some time. It's this weird sort of friendship, sort of rivalry, based around each of them having sort of these weird feelings for each other that are mixed between conflicting thoughts (such as the way that Kumiko both finds Reina's views on some things to be completely opposite of her own and wonders why she cares about them, but then at the same time stresses out so much about what Reina thinks of her and is always trying to clear the air). At the same time, despite them interacting with music in vastly different ways and having these conflicting images of one another, we can also see in the latest episode especially how they're both on the same wavelength on a lot of issues, and share similar worldviews on certain subjects, which is also what leads to the powerful moments in the second half of episode 8. We see that Reina can have the same level of bluntness that Kumiko has ("You know how you have a terrible personality?"), we just haven't seen that same element from her because we aren't in her head the way we are with Kumiko. Kumiko gets swept away not because she wants to make out with Reina, but because she finds herself having someone who she can interact with as an equal and be herself around. Reina sees through to the core of what makes Kumiko who she is, and says that she wants to be around that. This is a powerful thing for a person to go through, and this is something that makes me want to see more of these two interacting going forward.
To reduce this very interesting and evolving relationship to being "yuri" to me is a real tragedy.