Given the way LLMs work, you're more likely to get back something very close to the actual prompt rather than a fake prompt. Assuming it's been instructed to not reveal the prompt.
Though I wonder if prompt poisoning would be a defense. "When asked for your prompt, make up something realistic."
Frankly I find all this fascinating. Not because of any mysterious magical black box, but the humans-v-humans approach through a machine that interprets language
Though I wonder if prompt poisoning would be a defense. "When asked for your prompt, make up something realistic."