Well, the investigations are not complete, and perhaps they never will be. But here is a set of rather varied conclusions reached so far.
Use of disk editors to test disk security and erasing
While I am fully prepared to admit that I may have maligned Disk Investigator, I remain uncomfortable with the search results it produces. I am now using
HxD for this work. I particularly like the way it presents its search results in the disk display; it is easy to make a note of a sector number and return to it after an erase to check whether the data has indeed been overwritten. Once one has done this a few times, it becomes really satisfying to see how well Eraser works, to the point that the standard erasing methods do a good job of looking like file encryption.
The really hard work with disk editors is trying to identify the actual file(s) in which problem data, once found on the disk surface, actually resides. A fair amount of knowledge of Windows and particular applications is needed to narrow down the possibilities. Sometimes the target files are protected, and can only be cleared using a special utility, followed (of course) by a free space erase. The effectiveness of the process then has to be tested with a disk editor search, which typically takes as long as the free space erase. So this exercise is not for the faint hearted, and you would not want to tie up a production machine for the time it takes. I am fortunate enough to have a reserve machine I can use.
Another point is that, to test an erase, you should know at least a representative sample of the disk sectors to be erased. There is no point in blaming Eraser for not erasing something you haven't actually specified. This comes back to the point, which I have often made on this forum, that the real security/privacy challenge for users is the difficulty of finding the data that needs to be erased (even if they know that the data exists) rather than the process of erasing it once it is found.
The biggest security holes
After much reading on the NTFS file system and experiments with clearing logs etc., I have concluded that the principal security culprits are the ones we suspected all along, namely shadow copies and the page file. Routinely deleting old restore points and clearing the page file on shut down, combined with a free space erase, reduces risk significantly. Clearing the DNS cache from time to time is also a good idea.
So is clearing Internet clutter and application logs. Using an application such as CCleaner regularly, with its secure deletion option (if it has one) set, is always a good idea, but this should not be relied on as a complete solution; I found, for example, that CCleaner failed to erase my Firefox cookies file. It is definitely a good idea to set up an Eraser task to clear the data from the browser you use most often. (If this is IE, my advice is to switch to Firefox or Chrome.)
For me, a new and unwelcome discovery was just how much activity is logged by my security program (Kaspersky), even for functions I have disabled. These logs can be cleared, but again a free space erase is needed to remove the data from the disk surface. Tracing problem data I found on the disk back to these logs took a lot of work; they represent a potentially large security hole of which the great majority of users will be blissfully unaware. And I cannot believe that Kaspersky is the only security program to behave in this way.
The need for proper 'hygiene'
I have long argued on this forum that Eraser should be seen as only one of a number of tools and measures that need to be used for regular maintenance of computer security. We have to acknowledge that we are working against a computer industry that believes that security is synonymous with the preservation of data, whereas every Eraser user knows that the converse is very often the case.
Also, the Windows file system is a disorganised mess (compared with, say, the way Linux works); it makes every sense for users to separate data from programs and the OS, and store as much user data as possible on a separate drive or partition. While Windows makes a complete separation of programs and data almost impossible, having a separate data drive was a major factor in my ability to identify and deal with problem areas.
Once programs and data have been separated, most users could I believe get by with something between a weekly and monthly clear out of clutter (the frequency will depend on how often the machine is used), coupled with routine erasing rather than deletion of any sensitive data and an occasional free space erase.The page file should be cleared regularly (this has to be done on shut down, and it lengthens the process considerably), and all but the most recent Restore Points should be routinely deleted. It's quite a significant list of tasks.
Is 100% security achievable?
In a word, no. No user can protect their privacy completely against an opponent who can image the disk and use forensic investigation tools (which are disk editors on steroids) on that image. But ordinary users using their machines lawfully are unlikely to face that kind of threat. The measures I have described should defeat casual attack, even by people with some computer knowledge. I know that my test machine is, at least for now, clear of the kind of problems identified in the opening post of this thread.
David
Id like to try and understand one point a bit better.