If you really want to keep your time under 500km, then it's going to limit your options if you are trying to get a 507 run in at the same time.
In this route below, I've added a sweet causeway near Kirkfield, which is really neat with water on both sides of you. I've also taken you up along #2 Deep Bay Road to Minden - it may have some light construction on Deep Bay Road, so let us know what the road conditions are like on it. There are gas stations and fast food places in Minden where you should be able to find an open washroom. Then on your way up to Haliburton I've routed you past the Minden Wild Water Preserve - usually there is a portable toilet there if you want and it's a great stop for a quick break by the rapids. Then take Glamorgan Road #3 south to Gooderham, and then the #507 south. Once in Buckhorn you can head home like you originally planned or make a quick side trip east to Northey's Bay Road which is quite nice. All these things add around 30minutes onto your trip, but at least it makes it more interesting, keeps it under 500km and gives you some more options if you want to include any of them.
You should also be able to find gas/washrooms in Haliburton & Peterborough.
Have a great ride!
goo.gl