The show knew we were waiting for the clegane bowl possibilities and was all "trial by combat is now no longer allowed, because we knew you'd rather watch 30 minutes of an awkward altar boy and his priest accuse a woman and a gay man of not being christian instead of a cool battle scene"
Oberyn nominated himself for a trial by combat just so he could kill the Mountain.
The Hound is a wanted fugitive and no one knows where he is.
There's quite a big stretch to have him face his brother in a trial by combat. Plus, any time this show mentions something really obvious in the first few episodes with a neat and tidy conclusion based on a character's wishes, it never happens.