I suspect this is a manifestation of the anomalies we observed in Guardian Forces.

If your connection to Zodiark is severed than rectifying the drain upon your aether and mind should be possible. Whether your memories would naturally restore themselves is another question entirely, but I do not think stopping the deterioration should be out of the question at all, especially given our new goals.