The invasion was an important development several reasons. It definitely weakened their bid for conditional surrender by removing a potentially valuable bargaining chip that would allow the Japanese to keep other territories gained via war with China 50 years prior. The possibility of a conditional surrender seemed unlikely anyway, but it was undeniably hopeless after that. Japan was going to shrink.
However, it also was the main reason why the US decided to strike the military infrastructure in HS and NS. They needed US troops on the ground in Japan to keep Soviet troops from occupying Japanese territory. I argue this was as much in the best interests of the Japanese as much as for the US. (By comparison, look what happened to the soviet-controlled Japanese territory in Manchuria and Korea following the war: two areas now with their traditional culture/way of life all but destroyed, society-wide poverty, gulags, low human rights and still 60 years later without a free election). The longer Japanese drag their feet with surrender, the greater likelihood that Communists would control part of mainland Japan. Or, so was the thinking at the time. .
The idea that the Japanese 'were going to surrender' may be true, but until you actually capitulate and agree to terms, the war continues. Killing continues. Destruction continues. It is not my understanding that they were trying to surrender and the Allies refused to accept. Some might view the refusal to officially surrender as an act of continued aggression, tantamount to an attack.
We're all gamers here, right? Think of it like Starcraft. It is courtesy to 'gg' whenever your chances of victory are hopeless. If you refuse to 'gg,' the victorious army must press on. This stubborn behavior hurts both players, but it only costs each other time. Zerg and Protoss don't have feelings, sense pain, or have families. Obviously, War isn't a game, but similarly, when it's time to gg, you gg, or the show goes on. Japan should have gg'd in 1943. After that, victory was assured for the Allies. EVERYTHING after that was unnecessary. In fact, if you believe that just the inevitability of eventual surrender qualifies opposing military actions as 'unnecessary,' you could argue with ease that the whole war was unnecessary. (Admiral Hara Tadaichi believed the war was lost on Dec 7, 1941).
As far as I know, the US did not continue to bomb Japan after all active troops had surrendered. For this reason, I believe that had Japan surrendered after the Soviets declared war, the US would have stopped. Yet, Japan did not surrender, so the US did not stop. 'going to' doesn't count for much in Total War.