If you are self-employed, then you started doing business when you started doing business, not when you registered a name or for a GST/HST number. 

That having been said, you may not be able to expense all startup costs. If you took a knitting course to learn to knit so you could sell handknit sweaters, then that would be considered a capital cost that you would not be able to claim as an expense. But, If you bought a sewing machine to make the sweaters, you might put that as a class 8 CCA asset, while knitting needles might be expensed on Line 8811 – Office stationery and supplies.

If your business is incorporated, then you can't put startup costs on your personal tax return.