It was fun! However, when your phantoms have other agendas, attempting to organise them solely through the use of gestures is not recommendedSounds like fun. I've never seen such a thing myself, invading in Anor Londo is usually met by a ganking committee - straight up 3 on 1, and being invaded (in Dark especially) usually means two blue phantoms coming at you together. After hundreds of invasions in Anor Londo, I've never seen anyone go 1-on-1 unless there were only two players there!
Not that I'm complaining, it's fun winning when the odds are against you. There's no way of knowing what imaginery rules anyone is playing by, if any. It might be fun if there was a visual cue of some sort when invaders arrive telling other plays "Honourable fighter" or "No rules, fight to the death" or something like that (which could still be used deceptively just like the message system).
I've seen Youtube videos of regular PvPers in the Burg (in that open area where you first see the dragon); they'll touch a Red Soapstone sign, and while waiting for their dueling partner, battle with a random invader. If the guy arrives and the fight's still going on, he'll stand up near the firebomb bridge and wait patiently. It's great, as while many players will each have their - as you say - "imaginary rules", this seem to be a pattern everyone abides by at that level.
The idea of a real honour system is nice, but probably very difficult in practice. It could be a true outline of that player, based on playstyle or even what gestures they use more frequently. If you left it for them to decide (and deceive like with the message system), I think it would lose substance. A player bowing and then fighting "dishonourably" is much more relevant to the Dark Souls experience =)