Just in today's mail, a Registered package.
My new Citizenship Certificate.
Had I known that I could have gotten one, I would have been able to apply for a passport without revealing the name change (and formerly male gender).
Ah well, live and learn. The Immigration website does say that Citizenship certificates can't be changed, but as it turns out, in exceptional circumstances like these, they can.
So now it's certified that Zoe Ellen Brain has been a citizen of Australia since 1988. Of course it's signed by the current Immigration Minister, not the one then, but that's just a minor anomaly.
Maybe I should inform the Passport Office of the new situation. It might give them a plausible way out of the impasse....