You always want to keep the higher rarity version, pretty much. But merging in a lower-rarity hero only gives you SP.
So if you enhance the 2* by merging in the 1*, you will get like +20 SP on the 2*. It's not worth much.
If you enhance the 1* by merging in the 2* (or another 1*), you will get some SP and also you will get +1 pseudo-level, which is 2 random stats. It's still a 1* though. And even with the max of +10 pseudo-levels the 1* will probably still be weaker than the 2* at level 40.
So, you never want to get rid of your highest rarities. But you also gain very little by merging in lower rarities. Instead you should consider either (1) setting lower-rarity heroes free to get feathers or (2) upgrading the lower-rarity heroes to the same rarity as the one you're using so that you get the stat boost. In general I expect that it only really makes sense to merge 5*s into 5*s.