Just some words of caution as well. if your riding ANYWHERE other than property YOU own, you are required to have your bike fully insured.
Also I would caution against, "riding on dirt road along side of railway tracks". Those roads are all PRIVATE property, belonging to the railroads. The CN and CP police do NOT have a sense of humor. You could find your bike on the back of a tow truck, going to an impound yard.
Obviously, riding the dirt roads you seen as you headed to Newmarket, any road your going to be on will require a BLUE plated bike, for road use. Under the Highway Traffic Act, even an "unassumed road" is still a "highway", (legal definition), as such the bike requires a BLUE plate and of course insurance. Unfortunately, as others have pointed out, Ontario is not a very Off road friendly place to live. Once you join OFTR, you will find LOTS of trails, just most are at least 60 minutes + from Toronto proper.