23.3; `let' behavior is strange.

[Xiaojiang Shen]

(progn (put 'defun 'x "out")
       (let ((old (plist-member (symbol-plist 'defun) 'x)))
         (message "old: %s." old)
         (put 'defun 'x "in")
         (message "old: %s." old)
         nil))

When I eval this form, I get something like this in *message* buffer:
old: (x out).
old: (x in).
nil

I thought it should be:
old: (x out).
old: (x out).
nil

If it is not a bug, how can I save the symbol's previous property?
Thank you.