Shouldn't my balance be adjusted once I pass the Climate Action Incentive Credit amount?

When you tell TurboTax that you live in a province that has the Climate Action Incentive (CAI), then that is added to the return when you enter the province. The amount may be removed or increased depending on the information that you add to the return, but it can also stay the same, which is why you may not see a change.