Yeah, it's just not clear what the characters know or are supposed to know. Even just a note that says "the characters are speaking English now" would be enough to just alleviate those problems.
Watching Space Bros is always really weird because the scope of the show is somewhat international. While I appreciate them keeping everything in Japanese because they mostly just use Japanese Voice actors anyway (not a fan of Engrish), I can't help but cringe every time the people from other nations do and say things in a way that's very Japanese.