The back button is the oddball. Conceptually, it's simple enough; it goes back. But the meaning of "back" varies. In Internet Explorer, it works like a browser back button. In most applications it backs out a level; for example, if I'm viewing a specific contact in the People hub, pressing back once will take me back to the list of all my contacts. Pressing back from the top level of an application will take you out of the application entirely, and back to the home screen, e-mail, or hub, that you were in when you launched the application.
This leads to some subtleties. Start at the home screen and then run an application. Pressing either back or Start will take you back to the home screen, but with a difference. If you pressed Start from within the application, pressing back will take you back to the application. If, however, you pressed back from within the application, pressing back again will do nothing. Intellectually, this does make sense; it's the same as you get in a browser. Go to your home page, then visit another page. If you click "back" to return to the home page, pressing back a second time won't do anything. If you press "home" to return to the home page, however, pressing back will take you back to the site you were visiting previously.
While this behavior may make sense in terms of a browser metaphor, in practice I found it kept on surprising me. The big thing was, if I spent long enough in an application, then I tended to forget where I was before, so pressing "back" to return took me somewhere essentially random.
Even worse was the behavior in the browser. Let's say I click a link in an e-mail; that starts the browser. I navigate through the website for a while, before wanting to return to my e-mail. Now, I can do this by pressing "back," but a single press won't take me back to the e-mail. It'll just undo the browser navigation. I have to press back-back-back-back... to return to the first page I visited in the browser, and only then will pressing back return me to the mail. If I don't care about the browsing I was doing, this is a little annoying. If I do care, it's actively destructive. All those presses of back take me away from the page I ended up at. But I might not want to leave that pageI may want to refer to it later or something. I just want to go back to the e-mail, without having to go back through the webpages.