While they're on Calmira, they mention that Naofumi is 63, Raphtalia, is 65, and Filo is 67. Raphtalia and Filo are higher because of last hit mechanics since that was shown earlier in the series which is about what you'd expected for a difference in level. While we don't have Ren, Itsuki, or Motoyasu's levels, we do have L'arc and Therese's levels which are 56 and 52. The other heroes would have to be near or below that considering what happens. My previous comment about a hero xp boost instead of what we got would change these levels overall to something closer to where Naofumi is now. Then you consider the tendency of the lameduck heroes to want to take all the kills themselves and soak up the XP there levels would be even higher. That would have an effect on how certain things occur since we know that both level and skill matters in this world per the Eclair sequence after Calmira.
The others think Naofumi's cheating but what's suggested is that the source of Naofumi's strength is actually taking this seriously and trying to learn what's going on and how the world is setup. Remember, Naofumi even went as far as to investigate the other's claims of power-up systems existing and found out that was the case. He's actually taking advantage of everything he can.
But like I said, the author could have used something else but I think they'd have to have changed other things to accommodate it.
As for the Raphtalia thing, I think it would have been way too soon for that sort of progress and as I said, part of it is Raphtalia's own resolve and showing Naofumi how serious she is. I will say that the novel might be a better choice for people that need more of Naofumi's internal dialogue to see how much he's suffering. I think the manga does a fine job of it but I'm more of a visually orientated person than anything. The WN also adds an interesting point when Raphtalia is getting the mark again. In all versions, Naofumi says "You don't have to do this, you know" but her response in the novel to him is "I'm getting this because I don't think you trust people without this mark, you shouldn't lie." A bit interesting that she doesn't believe he's changed yet.
While it isn't important to the characters' relationships directly most of the time, It's important to the setting and some of the things that occur. Naofumi helps Raphtalia take revenge on the slave owner that hurt her and her friends, it's part of the problems found in Melromarc which had story ramifications, it's a result of the prejudice and xenophobia in the setting, etc. So there's a pretty good reason to have it overall.
It's meant to show Naofumi is a morally grey character who did something bad out of necessity and I can see it being a pretty consistent reminder of that.