As someone who has spent the past 15+ building houses, renovating houses, finishing houses (including painting) it's essentially this. While I don't necessarily think it's one of the "hardest trades" there is certainly a knack to it. It takes time to become proficient with the skill, like anything else really.
The same goes for painting, anyone can paint I guess, but can everyone cut in a razor sharp line against the ceiling? Or roll a wall out properly? That's not even mentioning the other more difficult aspects of painting like staining wood, clear coating etc.
Many drywallers/mudders can paint as well but you typically see that in small jobs or handyman type scenarios. When it comes to anything larger like subdivisions or custom homes it's just not efficient. For a general contractor it's better to get a crew that just does drywall, a crew that just frames, a crew that just does cabinets etc. The quality is much more consistent that way.
If op is confused about
paint roller for smooth finish and drywall being separate trades, wait till he finds out that drywall and murders are separate trades usually! "Boarders" and "mudders".