The answer to this is very much dependent on the content and context of the material. If you have said something really bad in a legal sense, then you should definitely consider deleting it, or deal with the consequences. Assuming we are not talking about those naked pictures of you at a stag/hen party, what do I think?
I do rewrite old articles if I’m sufficiently motivated by a drastic change in my understanding of the subject. The articles on my website are more fact-based, so if there is incorrect information, it should be revised. I tend to write version-specific articles, so I don’t usually add features to old articles, but write a new features article, then link the articles in a chain. That prevents an article becoming massive over time, while still clearly showing the progression of the feature set for those that care about more than the basics.
My blog posts are more opinion than fact, so I am less inclined to revise them. If my opinion alters, I prefer to write a new post discussing my current opinion and why I changed my mind, with forward and back links between the posts. I think this is much more informative for readers, as they can see how you’ve developed over time. There is little value in trying to rewrite history. 🙂
I remember having a conversation with Chris Muir about this a long time ago and he was very much of the opinion that a blog post is a point in time snapshot of your opinion and thoughts, so it shouldn’t be retrospectively changed. His words have obviously influenced my opinion in this matter. 🙂
Ultimately, the Way Back Machine will reveal that terrible secret (you thought Buffer Cache Hit Ratios were a great way to tune a database) and you will be exposed. 🙂