Eraser interrupted with error when wiping cluster tips

[Tom1234]

Eraser was erasing cluster tips and I had Task Log form opened. I was scrolling the form when in the same time eraser was adding new entries into the Task Log form. At that time disk wiping was stopped with last entry in the form indicating error "cannot access a disposed object" LogForm. I googled and it looks like it is some kind of microsoft .net error for UI elements like forms. Eraser did not crash, it just stoped wiping the cluster tips. I think it didn't get to the stage to start wiping the free disk space because directory and files for that were not created yet in c:/.

My question is this: is there a chance that some of the files that had their cluster tips wiped, at that moment when eraser abruptly stopped, were left in some kind of corrupt state? longer size with garbage at the end or something else?

thanks!

 

[Tom1234]

related question, what would happen if eraser was erasing cluster tips and there was power outage or just a system crash. would that be handled correctly?

 

[DavidHB]

related question, what would happen if eraser was erasing cluster tips and there was power outage or just a system crash. would that be handled correctly?

It would be the same as any write activity when a power outage occurs (caches would be lost, and there is always the possibility of data corruption). Mostly, in my experience, all would be well. Strictly speaking, no application can 'handle' a power outage, because it will just cease to function (unless the machine is attached to a UPS, of course). But writing to cluster tips (when the OS permits it, of course) is, I believe, no more 'dangerous' than any other form of writing to files.

As regards your original question, I believe that Eraser deals with each file in turn when wiping cluster tips. So it follows from what I said that file corruption resulting from an Eraser crash is unlikely. I'd hesitate to say 'impossible'; there are a lot of variables to consider, though the .NET libraries ought to be able to handle most exceptions without corrupting data. All I would say is that I have done a lot of Eraser testing and had more crashes than most people, and have never encountered file corruption as a result.

Was the crash you experienced on the C: (system) drive? I routinely find that Eraser (any version) cannot compete a cluster tip erase on the system drive under Vista or Win 7. This seems to be a result of vast numbers of permissions refusals becoming more than Eraser (or, as you say, .NET) can handle. Maybe when Joel gets round to implementing the Eraser engine as a service, the permissions issue will go away. In the meantime, my advice is not to bother erasing cluster tips on Drive C:; there are just too many protected system files in the Windows system. The corollary of this is, of course, that it makes sense to store sensitive data (in fact, all data) on a non system drive. This will be even more true when more of us start using SSDs as system drives.

David

 

[Tom1234]

yes, crash was on drive C:. it's a laptop so it's the only drive. it wasn't actually crash, eraser just stoped with drive wipe and from the circumstances (me scrolling the log form at the same time eraser was adding new entries to the log form, and error in the Task Log dialog indicating LogForm as the cause) i suspect eraser just didn't like me viewing and scrolling the log form at that particular moment. so then, you think under such circumstances files would be properly truncated to their original sizes and closed?

 

[Tom1234]

as far as the power outages and the system crashes are concerned, the thing is that cluster tips erasing takes really long time so there is chance that something may happen. for example here in pacific northwest we get few power outages a year lasting from few seconds to few hours. and since erasing cluster tips may last entire day there's a good chance that there may be a problem. databases solve such a problem with "transactions" that guarantee that for example financial database is not corrupted no matter what. i think they make a backup copy of a file that is beeing modified so next time they run they can restore the state. and they use "file commit" to make sure that backup copy is physically written onto the drive and not in the cache. i guess this is a feature request for the next version

 

[DavidHB]

I am starting to think that when you say "erasing cluster tips", you are referring to the whole free space wipe process. That can take a day, depending on the speed of the machine and the size of the drive, but erasing cluster tips on its own should not take that long. Anyway, as I said, I'd advise you to uncheck the option to erase cluster tips when you are erasing free space on the C: drive. The degree of compromise to security is probably minimal - much less than the likelihood that you have left personal data undeleted (it is really hard to remove everything you need to).

With a laptop, power outages are not much of an issue; even if the battery runs down, the machine should hibernate or shut down gracefully. In any case, the standard file management routines built into Windows are designed to have the kind of transactional security you were describing for databases; if it were not so, we would see far more file corruption than we actually do. Eraser uses the low-level routines built into .NET, so there is (unless Joel advises otherwise) no need for the feature you were requesting.

I'm not particularly a LInux fan, but I much prefer the way that Linux separates data from programs (and partitions from physical storage devices) to the mess that is a standard Windows installation. In their laptops (or at least all that I have used), Acer wisely partition the physical drive into system and data drives, though they do leave it to the user to move the data (and the standard locations such as My Documents) to the data drive. Now that newer laptops have hard drives of 250GB or more, I'd recommend this approach to anyone using a laptop as their sole machine or for mission critical tasks. That way, if malware takes down your system drive (I see it happen often), you can format it and reinstall Windows with your data untouched. And it is much easier to back things up and use utilities like Eraser if you keep all your important data on a non-system drive.

Your reference to truncating files suggests a degree of confusion. Erasing cluster tips does not change the file size (or the file data). All it does is overwrite the unused space between the end of the file data and the end of the last block allocated to that file. That space is inaccessible under normal circumstances (you can of course read it with a hex editor); the reason for overwriting it is to remove any bits of a previously deleted file which might remain in the unused space.

I hope that these explanations are useful.

David

 

[Tom1234]

David, thanks for the clarification,

About the power failure, I understand that can not happen on a laptop. I was thinking more the general case like power failure on a desktop or some other failure on a laptop or desktop. And the process of erasing cluster tips actually took whole night on my laptop. I know because it was still leaving new messages about various files in the Task Log form. I don't think it got to the "clean free space" stage by the time it ended in error.

Now about the actual process of the cluster tip erasures. So you guys are using some low level file write routine to write at the end of the file? And that means that there is no point in time when the actual file size is increased to erase the remainder of the cluster? If thats the case then that's fantastic, no need to worry. I guess I incorrectly assumed that you would open file and start appending to the end of the file until you reached the end of the cluster and then just truncate the file to the original size, which may possibly leave corrupt file in the case of a crash. I guess I assumed wrong.

Thanks again for the clarification

 

[DavidHB]

About the power failure, ... I was thinking more the general case like power failure on a desktop or some other failure on a laptop or desktop.

I'd stick with the view that Eraser would do no more damage than any other write operation in that situation. If power outages are common, it makes sense to protect mission critical systems with a UPS; basic UPS capability (which will at least get you a clean shut-down) is reasonably affordable these days. Hardware failures come in many shapes and sizes, so it's difficult to be specific about those. And, just to add to your joy, the component that is most likely to fail without warning is, in my experience, the hard drive. That is what backups are for ...

And the process of erasing cluster tips actually took whole night on my laptop. I know because it was still leaving new messages about various files in the Task Log form. I don't think it got to the "clean free space" stage by the time it ended in error.

That's slow, I think, even allowing for the fact that the drive is probably only 5,400rpm. If your machine is generally slow, that calls for investigation. I've had a couple of slow drives (on other people's machines) to deal with over the last couple of weeks. One of these had been infected with a rootkit (thank you, Microsoft Security Essentials), and the other was failing and had to be replaced.

Now about the actual process of the cluster tip erasures. ... I guess I incorrectly assumed that you would open file and start appending to the end of the file until you reached the end of the cluster and then just truncate the file to the original size, which may possibly leave corrupt file in the case of a crash. I guess I assumed wrong.

Joel is the expert on this, but, as far as I know, the process is pretty much as follows:

• read the last block to get a working copy;
• in the working copy, identify the end of the file data, and overwrite from there to the end of the working copy;
• write the working copy back to the original block location.

This may be a poor description, as Eraser works with file streams rather than the complete file constructs you and I are familiar with. What is certain however is that Eraser does not at any time change either the size of the file data or the amount of space allocated to the file in the MFT or FAT.

David

 

[Joel]

Okay, that was a rather long read... lol.

I've checked the Eraser source code; exceptions that are thrown and catchable by Eraser code are properly dealt with; files are reset and closed. I say only exceptions caught by Eraser code - because if the .NET framework buckles under us, there's nothing we can do but hit the ground face first. In the case of the log message, it is a .NET exception. It does seem to be caused by a race condition: so if you can find a way to reliably replicate the crash, please share and I'll see if I can fix it.

Your reference to truncating files suggests a degree of confusion. Erasing cluster tips does not change the file size (or the file data). All it does is overwrite the unused space between the end of the file data and the end of the last block allocated to that file. That space is inaccessible under normal circumstances (you can of course read it with a hex editor); the reason for overwriting it is to remove any bits of a previously deleted file which might remain in the unused space.

That's not exactly it, either. While at the bird's eye view that's what's going on, Eraser does have to open the file, deliberately extend the file size to the nearest cluster size (so making the file size a multiple of 4096 bytes, for example), erase the gunk after the old file size, and then truncate the file back to it's original file size. A power failure when Eraser's extended the file but not reset it could corrupt the file. Eraser does take necessary safeguards though: it will refuse to wipe the cluster tips of system files, as well as files which are open in any program. This could be one of the contributions to the very bloated erase logs...

Now about the actual process of the cluster tip erasures. ... I guess I incorrectly assumed that you would open file and start appending to the end of the file until you reached the end of the cluster and then just truncate the file to the original size, which may possibly leave corrupt file in the case of a crash. I guess I assumed wrong.

Joel is the expert on this, but, as far as I know, the process is pretty much as follows:

• read the last block to get a working copy;
• in the working copy, identify the end of the file data, and overwrite from there to the end of the working copy;
• write the working copy back to the original block location.

This may be a poor description, as Eraser works with file streams rather than the complete file constructs you and I are familiar with. What is certain however is that Eraser does not at any time change either the size of the file data or the amount of space allocated to the file in the MFT or FAT.

Now, having said my earlier paragraph, Eraser does change the file size for the instant the cluster tips are being erased (it is, after all, just a number in the MFT). However, the amount of space allocated in the MFT and FAT does not change (since the MFT/FAT works at a cluster granularity, which we are not appending or removing any clusters)... I agree this is all a little confusing and that my explanation can be better, but let me know what exactly requires clarification and hopefully I can clear it up.

 

[DavidHB]

Thanks for the clarification, Joel. It seems that, in the situation you describe, the word 'truncate' is not so much wrong as potentially misleading ...

As I've said before, I do agree about the bloated logs (and wonder whether they are a factor in the unexplained crashes). It would be nice to go back to Eraser 5 at least to the extent that the logs tell you a lot less of what Eraser cannot do and at least something of what it is doing. Perhaps a log referring to each element of the task and saying whether it could be completed provides a sensible structure. For instance:

Starting Task Erase Free space on Drive C:
Erasing cluster tips
Eraser erased cluster tips, but could not access 15,467 protected or locked files
Erasing free space on Drive C:
147GB of free space erased; erasure completed
Erasing unused MFT entries
233 unused MFT entries erased
Task completed

or

Starting unnamed task initiated from the context menu
Files in folder ZZZ1234 deleted. Folder deleted.
File ABC5678.nnn deleted
File DEF9012.mmm deleted
Task completed

Does this make sense? If so, I'll start a task in Trac (unless there's one there already).

David

 

[Joel]

This crash has got nothing to do with the level of logging Eraser does -- logging mechanisms should be robust, and if it is not, it's something I'd have to contend with... which I am Race conditions (like this one) are impossible to trace since they occur randomly.

On your suggestion, yes it is possible. But I still think it will be lost in all the other messages. Messages like these cannot be given a higher priority level than the cannot erase cluster tips for file XYZ messages: it just wouldn't make sense. One Informational entry at the end of the erase can be made, but if the user filters the log to Warnings and above, he won't see it. I think there was such a request before, but I don't know how to get about implementing it...

 

[Tom1234]

Hello Joel and thanks for answering. So are you saying that in this case there would be no corruption of files on my computer? Eraser did not crash it just ended wiping cluster tips task and added final error message to the Task Log ("cannot access a disposed object" LogForm).

When error happened I had Task Log form opened and I was scrolling through the error messages. At that time Eraser also started adding number of messages to the form and caused additional scrolling itself. So maybe the best way to replicate the problem would be to slightly modify the code to rapidly add dummy error messages to the Task Log and at the same time scroll the form manually.

I googled a little bit and it seems that there are low level windows APIs that deal directly with sectors on the disk. This way there would be no need to temporary increase file size so file corruption could not happen. Is there a chance that that may be used in future releases? Or alternatively some kind of transaction mechanism that would backup file size and/or last overwriten cluster and then in the case of crash restore it?

Have a question. When free space is wiped is it in the final pass set to 0 or is it filled with some pattern of bytes?

Tom

 

[Joel]

Yes, there is no corruption. There are not plans to use any low-level APIs: The low level APIs bypass the file system driver, so if a file is in use then the file will be corrupt.

 

[Joel]

And the pattern is the same one used for the rest of the unused space erasure.

 

[DavidHB]

Joel - Thanks also for your response about logging. This is off-topic, so, when I can, I'll start another thread on the subject in the Programming forum.

David

 

[william070707]

While starting a new thread,please give the thread link here,so that we can go directly from here :mrgreen:

 

[DavidHB]

Will do but the post requires some thought.

David

 

[DavidHB]

the new 'spin-off' topic on logging is here.

David