Yes, in your case you can send your return to the Sudbury Tax Centre, if your permanent residence was in Toronto. Please be aware that as a non-resident, you can also fax your return. Also, if you do send your return to the wrong address, the Canada Revenue Agency (CRA) will then send it to the correct place themselves, so there will just be a delay in having your return processed.
Sudbury Tax Centre
1050 Notre Dame Avenue
Sudbury ON P3A 5C2
Canada
Fax: 705-671-3994 and 1-855-276-1529