Or like I said, you can directly reference what you do not wish to do.

或者像我说的那样，你可以直接表态你不想做某事。

I'd rather 课堂 - SpeakingMax英语口语达人

Instead of being very direct, you can use "I don't know" to soften your response slightly.

但你不想说得那么直接，你可以用“I don't know”来让你的回答更为委婉。

I don't know 课堂 - SpeakingMax英语口语达人