If you ever submitted (or had someone submit for you) an income tax return to the CRA, then you would have gotten a Notice Of Assessment telling you that they have processed your return. These would be mailed out to you, unless you have chosen the electronic only option in My Account. But, you would need to have filed a return at least once to have a My Account, so that would be another thing that would tell you that you have already filed a return before.
You would answer "No" to the question if you have ever submitted (or had someone submit for you) an income tax return, even if you didn't do it every year.