“During the few times Lee traveled extensively after the war, he was often confronted by crowds of admirers and on lookers. As was the respectful custom, he would raise is hat above his head to salute the crowd. Returning from one such trip, the celebrity kidded with his daughter: ‘They would make too much fuss over the old rebel.’
“A few days after he came home, one of his daughter remonstrated with him about the hat he was wearing. He replied: ‘You don’t like this hat? Why, I have seen a whole cityful come out to admire it.’”
Source: Robert E. Lee's Lighter Side: The Marble Man's Sense of Humor, edited by Thomas Forehand, 2006.