Eraser - Possibly Insecure?

D

DHT420

New Member
This is pretty long and verbose and I apologize for that, but I thought you might be interested in a test
of Eraser I performed.

The latest PortableApps version of Heidi.ie's Eraser (v5.82) was subjected to a test in order to determine
it's effectiveness as a secure file eraser. The test was performed using a simple, reproducible formula.
Five text files, labeled consecutively TEST1, TEST2, TEST3, TEST4, and TEST5 were created. Each of these files
contained a different random 20-character alphanumeric string that was repeated 10 times in the text file.
These files were created on a laptop and uploaded to the test computer via a Western Digital 120GB Passport.

(example string: dMFO8p1qBSdEVhIRcL5m)

The files would be uploaded to the computer from the Passport individually for each test. The test file
would be placed in the C:\ drive of the test computer. The test file would be opened in Notepad, so that
the test's file string could be copied to the Clipboard. Then, Eraser would be run and the file would be dragged and
dropped into the Eraser task area. The file would then be erased according to the preferences of the test.

After the file was erased, a program called "Disk Investigator" would be opened. Disk Investigator is a forensic tool
similar to a powerful hex editor that can be used to recover deleted data and most importantly: search throughout
a hard disk for a specific string. In the "Search" box of Disk Investigator, the string that had been copied from the test
file to the Clipboard earlier was pasted. Disk Investigator then searched the C:\ drive to find the string in question, and if the string was found it was stopped and the result recorded.
(Disk Investigator can be found at http://www.theabsolute.net/sware/dskinv.html.)

By this test method, if Disk Investigator fails to find any of the random strings after using Eraser then
these files can be considered to be unrecoverable and securely erased. If strings are found by Disk Investigator
after using Eraser, then the ability of Eraser to securely erase files is called into question.

Environment:

Eraser's default File Erasing settings:
Overwrite Cluster Tip Area: Yes
Overwrite File Names: Yes
Overwrite Alternate Data Streams: Yes

Computer Info:
The test computer was an HP Pavilion running Windows XP SP2 with an Intel Pentium 4 processor, 504MB of RAM and a 32.3GB hard drive. The computer was not connected to the Internet, and the user was logged in as an Administrator. The hard drive was not in any significant state of fragmentation (Manually defragmented prior to testing). It had no significant utilities running while these tests were being performed, and there are no backup or data restore utilities installed or running, including System Restore.

Tests:

First Test:
Erasing Method: 'Only first and last 2KB'
Passes: 1
Strings found: Yes
Conclusion: This file was not satisfactorily deleted.

Second Test:
Erasing Method: 'Pseudorandom Data'
Passes: 1
Strings found: Yes
Conclusion: This file was not satisfactorily deleted.

Third Test:
Erasing Method: 'US DoD 5220.22-M (8-306. / E)'
Passes: 3
Strings found: Yes
Conclusion: This file was not satisfactorily deleted.

Fourth Test:
Erasing Method: 'US DoD 5220.22-M (8-306. / E, C and E)'
Passes: 7
Strings found: Yes
Conclusion: This file was not satisfactorily deleted.

Fifth Test:
Erasing Method: 'Guttmann'
Passes: 35
Strings found: Yes
Conclusion: This file was not satisfactorily deleted.

*The numbers of instances of strings being found for each search were typically in the high twentys, whereas there were only ten instances of each alphanumeric string in each text file. Also, Disk Investigator typically found the strings when it was about 8% or 9% through the scan.

Conclusion:
The finding of these strings after Eraser had deleted the files in question is no different than the contents of a sensitive document being found after deletion by Eraser. While complete recovery of a document deleted by Eraser may not be easy, there is a great risk that sensitive information may still be present after Eraser is used.

I cannot postulate why these strings have been retained by the hard disk even after Eraser was used to delete them. Disk Investigator scanned only the hard drive, not the RAM, and I cannot picture a scenario like this happening because of cached data in the pagefile. In the tests, the file was deleted IMMEDIATELY after it was copied to the hard drive from the Passport, so there really was little time for any pre-existing programs to have created a copy of the file if such programs
were installed.

Anyone is welcome to do this same test themselves, and probably should if they are concerned. I encourage someone else to see if the same thing occurs on their computers(or even external storage media) in order to determine if this is a singular problem or an important security risk.

PS: I have done less intensive testing with other file shredders (Omziff's file shredder, Spybot S&D's file shredder, and dsDel) and I have noticed the same thing. This was both on a Toshiba laptop running Vista AND the test computer used in the above tests.

PPS: I did another test where I created a text file in the C:\ drive with the single string "LOLCHEESEBURGERSALSOCATS". After using Eraser with the Pseudorandom 1-pass erasing method on
the file, I restarted the computer and then proceeded to run Disk Investigator. In little time, DI found the string.
 
Re: Portable Eraser - Possibly Insecure?

Couple of things I want to add:

The test computer was not encrypted or running an encryption program.
The test computer was running in Normal startup, not in Safe Mode.
When Disk Investigator found strings, it gave you the ability to look at their location
on the hard drive and the other data around them. Typically there would not be a single
string but a few rows of them, JUST like in the text files that were deleted by Eraser.

Also, I think its fair to say that the testing method represents a much more simplistic and cautious
file management than daily use, so if Eraser fails to perform under these simple conditions
perhaps the failure would be more catastrophic under daily useage conditions.

It seems there are more people on these forums that have been able to view data deleted
by Eraser using Winhex, which is similar to Disk Investigator. It doesn't seem like this is a unique case.
 
I've done some testing myself now with similar results. But you have to search for the test strings before the wipe as well. If I copied the file containing the strings "fresh" over to the test partition it would be there always twice. After running Eraser on that file it seems that it indeed is wiped clean. But the other ghost copy or whatever you call this will be still there. So this tells me two things: Eraser does not completely fail and wipes the original file, but Eraser is not aware of the other 1:1 copy of that file hiding somewhere on the partition thus leaves it intact.

To test Eraser in normal conditions I picked another random existing file (that was put on the system not via copy&paste) and some unique strings inside it, searched for it with Disk Investigator to make sure it only exists once on the partition and ran the wipe afterwards. No traces are found after that.

The only thing killing the second copy is a free-space wipe.


Interesting. :lol:


// Eraser 5.8.7-beta4 normal installation
 
It seems like with the daily, regular use one can expect from people looking to securely eraser sensitive info using Eraser, that their data would be compromised in a way. Aside from a LiveCD wipe or daily free space wipes, regular in-Windows usage under normal circumstances seems insecure.
 
Thank you for your analysis DHT and whishful, your help in improving Eraser is greatly appreciated by the Eraser devteam and the community at large.

There are a few things that are important when considering file deletion applications. You mentioned that strings can be found on the disk. I do not doubt the finding because there are two things currently that prevent a complete Erasure of data:
  • Journaling file systems. On Windows, NTFS, on Linux, ext3+ (ie ext3, ext4 etc), XFS (IIRC) etc.
  • Pseudorandom number generator

Allow me to explain (the descriptions may not be 100% accurate, it's a rough picture I'm getting at here and I'm also just recalling from memory.)

Journalling file systems are file systems which records every transaction which takes place on-disk, for example, every read and every write (then there are metadata cases and so on). This is done such that data written to disk will go to the journal before data is actually written on the disk, so that in the event of corruption or hardware failure, if the journal is written but data is not, the operation can still complete and leave the disk in a sane state. If data doesn't go to the journal, then at least your disk isn't corrupted (just data loss). That being said, that means that in a journalling file system, two or more copies of data will exist on the physical disk. Te first being the copy you want to create, the others being copies in the journal. Why I say two or more and not just two is due to the implementation of the file system.

Eraser will have access to the true copy. But since every operation is recorded in the journal, the write request is also logged thereby leaving another copy somewhere on disk. This copy cannot be erased by Eraser since it is a file system structure (and is locked by the OS - on Windows anyway) but when reading sector-by-sector, disk recovery programs can find your data there. This is unavoidable and therefore the current recommendation is to weigh your sensitivity/reliability ratio. If sensitivity is more important, disable journalling altogether.

Next regarding the Pseudorandom number generator. Data to users are nothing but numbers to the computer - that's the missing link here for most of us. Remember that your data is also a unique set of numbers. The number generator (or, in short, PRNG) will therefore attempt to create seemingly random sequences of numbers. When this goes to disk, it is inevitable that some sequences are similar, if not identical, to your sensitive data. This is unavoidable. Imagine having a data sequence 12345; the PRNG is also capable of generating a sequence 12345 albeit at an extremely negligible risk.

So, can we say that both problems are of equal concern? No, definitely not. Journalling is quite a big issue and is more serious than the problem of the PRNG creating the exact same data sequence. One thing that you can help us with is in future, report the file system the drive is on.

Thank you once again and I hope my post clears a few things up.

Regards,
Joel
 
The file system is NTFS.


So the USN Journal is the problem? Wouldn't it be possible to have Eraser do a sector search itself for the copy and wipe it on restart or via raw disk access? I have no idea of the technical aspects and how Eraser would write to disk going around the OS but to me it seems like the only real way. Or at least an extra option to scan for the copy file, report any doubles and offer to wipe the free space?

With the information I have now I am doubtful if using Eraser on files isn't much more than a waste of time without knowing if there wasn't a copy somewhere on the HD. A simple deletion with the Windows recycle bin and some time later a free space wipe then has the same effect as using Eraser before.
 
I would believe so. Your strings are definitely small enough to stay in the journal for quite a while (I believe it has a fixed size.)

No, Eraser can't do a sector search itself and write it on restart. That will require reverse-engineering a patented technology and I don't have the time nor the resources to do that. NTFS-3G is a driver for Linux that does that, but then it won't work under Windows. Even if it does, I'll have to write a NT native application to do that Erase which will really cost a lot in terms of time and resources. I don't think we can scan for the copy in the journal, either, because it's quite opaque to me and not all parts of the file can be present. The journal is a pretty big beast on its own. I haven't done research coz I don't have time on my hands now, so the current two solutions are to either use FAT as your file system, or disable the journal. Google around for that, if memory serves it's fsutil usn deletejournal C: or whatever drive.

By the way, if the journal has a copy of the file, free space erasures may not help at all since the journal is considered occupied space.

Good luck!
Joel
 
One more thing - regarding your first/last 2KB erasure, it is not meant to be effective in your case. It is helpful only for files with specific data headers and adversaries are trying to find all files of a certain type. In your case, you know the string you erased so the test here is invalid. Just FYI.

Joel
 
Joel,

Thanks ALOT! You have been extremely enlightening with reguards to this issue. It's great to see an extremely adept developer who can ALSO easily explain things to laymen such as myself. I really appreciate your input.

Seems like journaling is the definite culprit here, because as you said the chances of the PRNG outputting the exact same 20-character string is highly unlikely. Seeing as a lot of Windows users have NTFS-formatted drives, I can see that there is some cause for worry. Of course; awareness of the issue is always the first step to a solution!

Thanks again, and good luck with Eraser's continued development!
 
What else would make a second copy of a file if not the USN journal? There are no indexing services active nor is there an "$Extend\$UsnJrnl:$J:$DATA" anywhere on the partition. Fsutil says the same thing. I went on and created a journal and deleted it to make sure the error message "The volume change journal is not active" was true.


One thing we didn't consider with these tests though is the (ridiculously small) size of the test file. Putting even a few characters around the test string will prevent the file from being doubled. I didn't test where the size limit lies exactly but embedding the 64byte string inside a file which then is ~2000byte will never cause a second copy. Not to mention anything beyond 1.000.000byte.


New conclusion: Eraser does a good job in almost all cases except for very tiny files which could be stored twice on the disk.




Can someone confirm and explain my findings?
 
System Restore may store a copy of your stuff, I don't know. Oh - the other location in NTFS is the MFT itself, of which there are two copies. Small files reside within the MFT (resident files).

Joel
 
You said the USN journal is a fixed size. Is it also a fixed location? Is there a way to wipe it similar to free space, by filling it with random data?
 
You may have misunderstood. The remnants in my case were note kept by the USN journal but more likely one of the MFT copies, as Joel said, since no USN journal exists on the tested partition. Wiping the free-space and also including non used MFT should clear these references. I am not sure what happens once you used fsutil to delete the journal but I would guess that it does not overwrite, only delete it. So wiping the free space would ultimately take this data with it as well. Please correct me if I'm wrong.
 
This is a fascinating discussion and I am impressed with the knowledge available on this forum. Thanks for sharing and educating us.
 
From my observations, whishful is right.

Joel
 
Eraser doesn't work as expected. please take note

Hi guys,

I'm writing this because of the many forum posts I have found about tools being able to recover files after wiping the free space with Eraser. At first I thought this was because many users are inexperienced in this area, but then again: where there is smoke there is fire. I'm 100% sure I have been able to recover several key files which should have been gone (regardless of system restore - which is disabled just to make sure)
In my search for an erase tool I stumbled across Eraser. So I decided to put it through the test. I suppose the moderators are familiar with Recuva (latest version), if not please ask for a link. I'm also using getdataback (v3.03) but scans with that tool are still in progress as I type. As a last tool I will be using Winhex.

Well now, I've got one external usb hard drive enclosure and one normal Windows XP sp3 computer.

Regarding the USB drive:
Early tests with Recuva show the usb drive has been erased properly. Properly means Recuva could not find any deleted files present so nothing was recoverable. I'm still waiting for the getdataback results and a second scan with Recuva. Winhex scan will have to wait as this is extremely time consuming.

GetDataBack update!:

The main issue here is that GetDataBack found orphan files or lost files (I believe they are the same, correct me if I am wrong). These files are mostly small files, thumbnails of pictures. I do not know for certain whether they are resident (to the MFT) or not but they CAN BE RECOVERED. There were also a few big AVI's but most of them were unrecoverable (I could only read the file attributes). One AVI was playable by Windows Media Player but I did not note down its size (darn).


Regarding the Windows XP sp3 computer:

Before erasing the free space with Eraser, I created a text file with the name 'secret.txt' and the text 'this is a secret' in it. Then I deleted this file via the recycle bin. Then I ran Recuva and could find a file name DC1.txt. This was in fact my 'secret.txt' file I had deleted earlier. I also could find a few pictures which I had deleted a week ago. All had names like DCx.jpg.

Then I decided to run Eraser and erase both the free space of my system (C)and secondary partition Z: (will clear this up later on). Then I had Recuva rescan (deep scan) the recycle bin. Darn, the text file was still recoverable. The pictures where gone though. Their file names (DCx.jpg) still showed up in Recuva but the files were not recoverable. I recovered the text file to the Z: partition (it is bad practice to restore an item to the same partition as where it has been deleted, we all know this). I then deleted the text files once more (again via the recycle bin). Then I re-erased both C: (system partition) and Z:. When Eraser finished I ran Recuva again and now I could recover 2 instances of my ‘secret.txt’ text file: one on the Z: and one on the C:.

I don’t know what went wrong, Erase seemed to be working as it should. I’ll update this post further on this week.

Cheers,

Jeroen

***** Update
Joel, I really appreciate your work and I know you must be very busy. But could you please take the time to read this paper and comment on it?
http://www.berghel.net/col-edit/digital ... v_8-06.php

Most of it has to do with the MFT. I think Eraser might miss the small files that reside within the data structure of the MFT.
 
Re: Eraser doesn't work as expected. please take note

Alright guy's I'm on a roll here.

I've just wiped the free space of the C: of my girlfriend 's PC. I am using a remote connection but this shouldn't really influence things. Recuva found nothing out of the ordinary. I'm doing a second deepscan of C: (just to be sure) now and I'll screenshot what Recuva finds to show what I mean with 'nothing out of the ordinary'.
Sreen shot added to post:
As you can see, nothing special. Mainly Eset NOD32 files, created after the wipe. Some other stuff too.

Then I am going to repeat my 'secret.txt' exercise:

As a reminder, I type the text 'this is a big secret' in the text file (but in Dutch then ^_^)

- create it on the desktop
- delete it via the recycle bin
- check whether Recuva can recover it
- run Eraser
- check with Recuva whether it can still find it and/ or recover it

I hope you will all find this interesting:).
 

Attachments

  • recuva output..jpg
    recuva output..jpg
    183.9 KB · Views: 2,458
Re: Eraser doesn't work as expected. please take note

Alright, I've done some more testing and I can now tell you in more detail what the problem is:

When Eraser erases free space following issue might occur:

-A file that fits in the MFT-itself (I believe such files must be smaller than 1.4kb) that has NOT been deleted by eraser's on demand erase will not be cleared after a free space wipe.
-The reason for this is that MFT-entries that have become obsolete are not being reïnitialized/wiped. Eraser does not check the MFT for such entries.

note that it should (someone please confirm) clear the MFT-entries of the files it erases itself regardless of the file's size!

Notes: at current I am usure whether this applies to larger files too (> 1.4kb). If eraser erases them, their MFT-entry will likewise be gone. I do not know what happens when you delete a large file via the recycle bin and then do a free space wipe (or for that means any other tool that does not clear the MFT-entry). Most likely the physical file will be gone but the MFT-entry describing that file may still exist.

I hope Joel or other admin can comment on these findings.
 
Re: Eraser doesn't work as expected. please take note

Both Joel and me are quite busy at our jobs but we have agreed to look into this together. It goes without saying I will report back but when exactly this will be is hard to say:-).
 
Sorry to drag this topic up, for the MFT-resident files, the solution would be to initialize the unused MFT-records; MFT-records cannot be deleted (the MFT cannot shrink in size) but the unused records can be overwritten this would delete the tiny files AND also the references to bigger files that have not been securely deleted. Winhex should be able to do this.

Nice feature request :wink:
 
You must log in or register to reply here.
Back
Top