Technically, it should say somewhere on the bill of sale that it's subject to finance approval. Whether it says deposit is refundable or not is irrelevant, obviously it doesn't hurt though.
Have you finalized anything with the bank/finance company? Did they give you information regarding the term? rate? monthly payment? Just because you signed the bill of sale, it doesn't mean you're locked in. Whether they gave you 3% or 29.9%, $100 or $500/month, it is up you to determine what is affordable. You are in no way locked in unless you signed and accepted the finance terms and conditions and even then, you can still cancel but obviously at this point you'd be giving up your deposit.
If this was a cash purchase then the dealer is correct. You are obligated to pick up the bike or you forfeit your deposit. They can send you a registered letter and take you to court (do-able but very unlikely) but they'll probably sell the bike 10 times over before any of that happens. At this point, it's just a way of locking you in when in reality, you have every right to cancel.
I PM'ed you my #, I'd be more than glad to talk it over and give you any advice I can. Call me.