Eraser eats up all available RAM on Win7 x64

P

pcworld

New Member
First of all thanks for this great open-source effort!

(I'm a Linux user and just used Windows to try Eraser)

System: Windows 7 Home Premium 64-bit, 4 GB RAM
I told Eraser to wipe the free space of a FAT32-partition on an (external/USB) HDD. The partition's size is about 206GiB from which 84GiB are used.
At the end of the wiping process Eraser took about 2GB of memory, probably all it could allocate. (The OS was in a pretty unusable state)

At the end of the log I found:
Code: 
External component has thrown an exception

Just wanted to report this. If it doesn't affect anyone else, there's no need to fix it.

Another bug I've found:
When I select multiple entries in the log window, do right-click and choose "copy", all of the entries except for the last one are copied into the clipboard.
 
Assuming that you are using version 6.o.x, try clearing the log for the task; there will be loads of messages about non-erasure of cluster tips, which will take up RAM when Eraser is running. Then do a disk check on the drive (Windows will schedule this to take place when you reboot). Then see if you can run the task to completion.

The log copying bug is a known issue in .NET. Annoyingly, I don't think it has been fixed in .NET 4

David
 
If you're the reporter for ticket 445, thanks.

I think the key here is that you're using FAT32. FAT unused space erasures has got a component that will manually walk the directory tree of the drive and clear unused entries. My suspicion is that you've got a damaged directory structure, though I can't be certain.

It's also possible that the drive you've used was used with Linux which does implement FAT a little differently from Windows, which could throw off the Eraser code. This would require a bit of analysis to trace the bug, but let's see if I can do so.
 
DavidHB said:
Assuming that you are using version 6.o.x, try clearing the log for the task; there will be loads of messages about non-erasure of cluster tips, which will take up RAM when Eraser is running.
Click to expand...
In the log there are about 10 messages, so definitely not something taking up 2GB of RAM.

DavidHB said:
Then do a disk check on the drive (Windows will schedule this to take place when you reboot). Then see if you can run the task to completion.
Click to expand...
I already ran dosfsck on Linux before using Eraser.

Joel said:
If you're the reporter for ticket 445, thanks.
Click to expand...
No, I didn't even find that ticket.

Joel said:
I think the key here is that you're using FAT32. FAT unused space erasures has got a component that will manually walk the directory tree of the drive and clear unused entries.
Click to expand...
That's the reason why I tried Eraser. I filled up all the free space with zeros, but that didn't stop file recovery programs to still find some folder names.
By the way, is there an option to just clear/overwrite unused entry names? That way I could still overwrite the files using my Linux method and just use Eraser to clear the inodes (or whatever you call them in FAT).

Joel said:
It's also possible that the drive you've used was used with Linux which does implement FAT a little differently from Windows, which could throw off the Eraser code.
Click to expand...
You're right with this, the partition was mainly used in Linux.

Let me know if or how I could help you fixing this.
 
With this new information, I withdraw my suggestions until Joel is clear on the implications of the 'used with Linux' issue.

There isn't currently an option just to clear the MFT, but it is on the 'to do' list. I think that it's potentially quite useful, because it is quick and makes deleted file recovery significantly harder, though not impossible.

David
 
Oh, FAT doesn't have an MFT, so cleaning it is VERY easy, and already implemented in Eraser's code. It also will compact your directory indices for you.

Let me just confirm that we're both on the same page before I give suggestions: the memory usage shoots up ONLY when the progress of the erase is past the 90% mark? OR does the memory usage gradually increase until the 90% mark?

If you could just take some data points while doing an erase, that would be great. There's no need to get past 90% (and I'd advise against it -- crashing and modifying file system structures probably shouldn't exist in the same sentence) so hit cancel when that's at that stage. I just want to make sure it's the FAT code that's the problem.

As for debugging technique... do you have a spare drive of equal capacity? This could be asking for a lot here, but since you're familiar with Linux, I can do a few things with you that other users can't :P
 
Okay, I just did a static code analysis (i.e. the brain as the compiler, lol) and found out why memory's leaking like that. I've got a fix on my local machine that's untested, would you like to try it out? It should be in r2728 (so builds 2728 and later will have the fix)

I'll test it on a flash drive later.
 
Joel said:
I just did a static code analysis ... and found out why memory's leaking like that. I've got a fix on my local machine ...
Click to expand...
This really intrigued me. I was under the impression that .NET did all the memory management, and so leaks should not arise as a result of program code, because .NET would clean up all the garbage.

I know that there are complex algorithms to determine what .NET regards as garbage, so I suppose that if program code does not behave in the way the algorithm assumes, memory requirements would grow. Is that how code can create leaks, or is there some other mechanism?

David

PS I subsequently saw in the change log that Eraser in fact uses unmanaged memory, and that is where the problem is. Which begs the question: why do you need to use unmanaged memory?
 
Because if I were to write the FAT code in C#, I'd probably foam at the mouth :P

FAT is so low-level that trying to express its intent in a high-level language like C# (where pointers and such are second-class citizens) is kinda committing Seppuku.
 
Ah. Well, we all have your health and well being at heart ... :)

David
 
I'm sorry that I didn't reply sooner, somehow I didn't receive any e-mail notifications about the new posts.

Joel said:
Oh, FAT doesn't have an MFT, so cleaning it is VERY easy, and already implemented in Eraser's code. It also will compact your directory indices for you.
Click to expand...
That's why I asked if there's an option to just clean the "inodes" instead of overwriting the whole partition.

Joel said:
Let me just confirm that we're both on the same page before I give suggestions: the memory usage shoots up ONLY when the progress of the erase is past the 90% mark? OR does the memory usage gradually increase until the 90% mark?
Click to expand...
I'm not sure anymore. I only realized the high memory usage when the progress bar was near the end.

Joel said:
If you could just take some data points while doing an erase, that would be great.
Click to expand...
I guess I should first test if r2728+ fixes this?

Joel said:
As for debugging technique... do you have a spare drive of equal capacity?
Click to expand...
I fear I don't have one...

Joel said:
Okay, I just did a static code analysis (i.e. the brain as the compiler, lol) and found out why memory's leaking like that. I've got a fix on my local machine that's untested, would you like to try it out? It should be in r2728 (so builds 2728 and later will have the fix)
Click to expand...
I hope I'll find some time to test it this weekend.
 
Have a go and let me know how it goes.
 
Finally found time.
Tested with 6.1.0.2781 on the same hardware/OS/HDD as before
Here the results:

t: time the task started
t till t+90min: between 40 and 70 MiB (or whatever unit Windows' task manager uses)
t+92min till near the end of completion (t+105min): between 280 and 360 MiB

Definitely a lot less memory usage, I guess you fixed the bug! :)

Btw, I didn't receive an e-mail notification for your last post either (although I'm subscribed to this topic), is e-mail functionality broken in this board?
 
Great!

Email seems to be working for me, but I also had some email problems (on my own server) lately so I'm not sure...
 
DavidHB said:
This really intrigued me. I was under the impression that .NET did all the memory management, and so leaks should not arise as a result of program code, because .NET would clean up all the garbage.
Click to expand...

viewtopic.php?f=2&t=7540

I reported a similar problem a year ago, I seem to recall the response was along the lines "oh it's not Eraser it's some random third party bit of software" :?

Soooo......is it Eraser or not?

Incidentally, if you follow that thread you'll see that installing a new version of Eraser (same .NET installation) fixed the problem.......so I guessed deep down that Eraser was the culprit :)
 
Sorry for the misdiagnosis last year. At the time of your post there wasn't anything concrete that we could pin point to.

In this instance, the OP was clear that he was doing an unused space erase on a FAT32 partition, which was handled by native code and does allocate memory.
 
No need for apologies although you might want to have a private word with DavidHB.

He's very quick to jump on the "I love Eraser and it can do no wrong" bandwagon.........he's probably got enough egg on his face right now to conjure up an omelette for an entire nation.

Facts will always win through :roll:
 
Doesn't look like the stooge has been here since just after my previous post.

Joel? Was he a plant? If so, why? The product is, by and large, excellent. You don't need to artificially inflate things by using a plant.

What's going on?
 
Chuckles: We're all volunteers here. We contribute what time we have. I welcome any sort of help I can get, if you're offering :D
 
You must log in or register to reply here.
Back
Top