Then bugs would be out to get me. Yeah there is history there.
You don't need to spill that history out here or anything, but if you think over that history, can you perhaps see a reason why they are concerned?
It might be an overreaction on their part, but I have a hard time believing someone would say something like that over minor instances of regular angry moments, but that's just my take on what information you have given us.
I think it's important to not take it personally, having a family with someone means you are responsible for that family. You protect them, you help them learn and grow and encourage them in different ways throughout life, you be there for them to catch them when they fall, no matter what, it's not just you at that point, everything you will do and say will have an impact on them, good, bad, big, little etc. etc.
That's what being a man and having a family is all about.
So I would think long and hard, it's okay to have things to work through, it's part of being human, but it's important to try your best to work through those things, especially with those you love, so that you can grow and change into a better version of yourself.