Don't count frisbee golf as part of your training regimen. The OP has information about your Basal Metabolic Rate - your BMR - and your non-training activities should just be lumped in with that.
I highly suggest counting calories when losing fat, at least when starting out. I jot things down in a notebook when I do this. Count everything. It sounds daunting at first, but knowing what you eat is a big part of making the change. The only time I never counted something I ate was with non-starchy vegetables. The calories in those are so minimal and I wasn't about to write down every vegetable I ate in a salad - though carrots, potatoes, and especially dressing are still important. That's why I just advise to write everything down - you don't know what you're eating until you do. And if you don't know what you're doing now, how are you supposed to change it, right?
If you want to do more than a little P90x, perform some calisthenics. Mr. Snrub added some information on different bodyweight movements you can perform any time of the day. Pick a movement and grease the gear - do it a lot. Most movements can be done just about anywhere, especially pushups. Pushups while watching television, pushups in between typing, pushups in line at the grocery store, pushups on top of someone doing pushups. Increase the frequency while avoiding high reps in your sets and you'll be on your way to improving your muscular endurance.
You're at a stage where eat less and do more is the best advice. As far as critiquing your current diet, I'll leave that to other members. I'm happy with mostly whole foods - tons of meat, fish, eggs, vegetables and dairy. I eat oats, fruits, rice and sweets on occasion. My diet is always high in protein, 170g - 200g of whole protein is my minimum intake per day. That's whole protein, too, not whey shakes.
I'll add this - if you don't want to pig out all weekend, make Sunday supper your free meal of the weekend. Eat strictly up until that evening's meal - and then eat just about whatever you want. It helps keep you mentally fresh and improves adherence to your diet throughout the week without damaging the long-term results.