Eraser ridiculously slow, and getting slower

S

silicondeath

New Member
Hi,

I haven't found much useful information to explain this behaviour, so hopefully someone here has an idea. I am using 6.0.8 on Windows XP, I am shredding a 1TB HD, which contains about 700gb of data on it. I started the job in explorer by right-clicking the drive letter, and chosing "Eraser" on the context menu. I then went into eraser and customised the job entry because I wanted it to start while I was asleep, foolishly thinking it would be done in the morning. The job had been created containing all folders that exist on the root of this drive, as expected, and I manually set the erase method to British HMG IS5 Enhanced, which is a 3-pass shred. Now, I would presume that essentially the job that Eraser has to do from this point forward would essentially be to write over that 700gb 3 times in accordance with the algorithm. This is a SATA HD and it can be filled with 1TB of data within 2-3 hours easily. So using my advanced math skills, I would estimate (generously) that this shred should take no longer than about 8-12 hours to complete, assuming that it is more disk-I/O intense than CPU intense. I started the shred at midnight on Wednesday, and it is now 10:30pm on Friday, and it looks like it's about 90% complete. I have been monitoring the job and noticed that the first 20% of the shred was done within a couple of hours, and since then the work rate has dropped off exponentially. What it did in the first 8 hours on Wednesday took 20 hours to do the next day, and it's been getting worse and worse the closer it gets to completion. The eraser process averages about 2-4% CPU load at any one time, and apart from that my CPU is basically idle. In fact, Task Manager currently tells me that Eraser has used a total of 2hrs and 37 minutes of CPU time, despite running for more than 2 days, so there's nothing to indicate that the bottleneck is the CPU. So the next thing I would expect is that it's the data rate of the HD that's bottlenecking, but not so. I would expect that my HD activity LED would pretty much be locked glowing if this was the case until the shred is complete, and indeed it started off that way, but by the next day it was (and still is) just an occasional flicker of activity. It's more idle than active. I still have > 5gig of RAM free and plenty of pagefile, all my other applications are running like a dream, it just seems like Eraser has decided to smoke some weed and will get around to finishing shredding when it can be bothered. I don't get it.

Also FYI, if I look at the drive in Explorer I see all the old files/folders still there, and there is no sign of randomly-named files/folders I have heard talked about in other threads here (although I would assume those files relates to wiping free disk space, and I'm not doing that - I'm shredding existing data). However, even though I see all the data there in explorer, if I check the disk properties it reports 110gb of used space, far less than the 700gb that the data takes up. I see this as a good sign, because it implies that it is actually shredding the data, even if it is at snails pace.


Eraser
:x
Y U SO SLOW?!?!
 
Some more info:

Since I checked at about 9:00 this morning it seems to have completed about another 15% in > 12 hours. That's what I mean about it getting slower and slower. I guess it's possible that the progress indicator is lying somewhat, but for this kind of operation it should be easy to estimate the time to completion since it's just raw data throughput. My computer does not hibernate or go on standby (except the monitor). The eraser.exe process is currently using 126mb of memory (peak @246mb) I've also just realised that the HD is in an external caddy connected by USB 2.0 via a multiport USB hub, so perhaps it's the USB bandwidth that's bottlenecking? But this doesn't explain the obvious decrease in performance compared to the initial several hours. Even if the other devices connected to the hub can chew up the bandwidth available to the external HD, there's nothing connected to the hub that has been busy I/Oing enough to impact the bandwidth... The most likely candidates are my other backup HD, which I currently don't use, and a 32gb USB flash drive which I do use but certainly not for moving large amounts of data around.

When alls said and done, I think that this issue is either a bug in Eraser, or has something to do with the task scheduling of Windows. Maybe it soft-throttles processes if they've been running for a long time? But then again, Eraser doesn't seem to need much CPU, it's more about data throughput, so what could/would throttle a HD or USB bus more and more over time?

I say it's a bug!!

I am so confuse :cry:
 
We have had reports of erasing becoming very slow, but it is quite difficult to pin down the cause. Given the complexities of the NTFS file system (which, on a 1TB drive, I assume is what you are using), it is possible that this is a function of particular installations and circumstances. I tend not to use the often emotive term 'bug' unless it is clear that a particular problem is (1) not by design and (2) at least partly fixable by a code change in Eraser. While (1) clearly does not apply here, we don't (I believe) yet know whether (2) applies or not.

I seem to recall that Joel has on occasion suggested that users with this kind of problem should try disabling the option in Preferences to force files to be unlocked for erasing. That might be worth a try. Also, if it is possible to do a large erase in smaller chunks, that might help with a file system blockage. And I have seen Eraser have problems with a very fragmented drive or a drive that has some sort of file system issue, and needs a disk check to be run.

Another approach, if you are erasing everything or nearly everything on the drive, is to backup what you are saving then format the drive and do a free space erase. When you are erasing a whole drive, a single pass free space erase is IMO as effective as using IS5 on individual files, because what little (and it really is not very much) you lose by not having the extra passes, you more than regain by completely overwriting the previous drive 'geography', making it much more difficult for an opponent to work out where any sensitive information might have been.

David
 
Ya well I'll give you some more info so hopefully one day someone can figure this out!

* Drives recently passed a basic chkdsk and had no bad blocks last I checked.
* One drive is on external USB, another one I am currently doing is internal SATA
* Both 1TB, NTFS, basic (not dynamic) single partition, roughly 700gb used space
* Regularly defragged, one drive only has about 900 large files on it
* Fairly dramatic slowdown seems to start from about 50% through the shred, first half of shred is done within 8 hours, the rest takes > 1 or 2 days
* No apparent CPU load just decreased disk activity
* I don't have the force unlock option enabled
* Write chaching enabled on internal SATA disk, disabled on USB disk
* Transfer rate in HDTune is ~100mb/sec

I have also wiped all the data on these drives and re-done with a free-space erase using the same algorithm, the findings were interesting:
- The free space erase was MUCH faster (despite theoretically having to write much more data to the HD)
- The USB drive is in fact MUCH slower than the SATA one, which I guess is to be expected, but I still don't understand why it starts fast and then gets slow.

So my recommendations:
- If you have to shred an entire HD, quick format it first, then do a free space erase. If your HD is in an external caddy, avoid using the USB interface if at all possible - try eSATA or move it inside your PC to use SATA. I'm not sure if going from USB -> IDE will be faster, but it's worth a go too.
 
Thanks for providing enough data for Joel to work on, when he's able to respond. Here are some thoughts to be going on with.

Free space erase is normally slower than routine erasing, because much more data is written; the limiting factor tends to be hard drive speed. Things get a bit more complicated when Eraser clears the MFT at the end of the run, or if space is fragmented and Eraser has to write a lot of small files, or Windows blocks writes to a nearly full hard drive, but in general free space erasing ought to be quicker than file erasing per MB written.

File erasing involves (obviously) getting access to the file, including forcing unlocks where required (and that option is selected), and dealing with the MFT entry and any shadow copies. This can bring Eraser into competition for access with other running programs (especially security software). The actual locations of the data have to be found and overwritten, which is likely to be slower than just writing random files to free space. Getting at shadow copies, particularly if there are a lot of restore points, can also be an issue. Whether any or all of these points are relevant in your case I do not know (see previous posts), but your latest description indicates to me that you have a file access issue. For your latest tests, did you switch off the 'force unlock' setting?

Also, I seem to recall that, in the 6.1/6.2 beta builds (not currently downloadable, unfortunately), Joel did tweak the erasing algorithm a bit. What I don't know is whether these changes were back-ported to 6.0.8. So it is just possible that your problem is already fixed for future releases, when they eventually happen.

David
 
Yeah I always had the force unlock option disabled, instead I did a chkdsk from the command prompt with the /f paramater because it will then give you the option of dismounting the volume which automatically drops all open file handles. There's also nothing but data on these disks (no applications) and nothing using the data so I don't think there would have been any open file handles to begin with. I should also mention that both these HD's are the exact same model (WD Green 64mb SATA 1TB), and it's now apparent to me that the EXTREME slowdown only applies to the disk sitting in the USB-connected caddy. While the SATA drive did seem to slow a bit, it's nothing compared to the slowdown I'm seeing on the USB caddy drive. It was a bit tricky to figure that out because the way that I scheduled the erase meant that it would run again the next night at the same time, so this lead me to believe the SATA drive was also going super-slow, but it had infact already completed and started again because of the schedule. I'm now fairly certain its only the USB connected one that takes multiple days on end - and it's happening with the free-space erase too. I thought it wasn't going slow at first because it was progressing so fast, but again, at about the 50-60% mark it slows down exponentially. This also might be a result of not having write-caching enabled on the USB drive? (If you have the "optimise for quick removal" option set in windows, it disables caching)

Also, maybe the wrong forum for this, but a feature I would really love to see is a tweak to the scheduler so that you can setup a once-off wipe to run at a specific time. At the moment you seem to have to setup a recurring schedule in order to specify a start time (like 1:00am).

All in all a great app but seems like you need to armed with a bit of knowledge to make it do your bidding. If Joels going to look into this, I would strongly suggest he starts with doing some tests with USB-connected drives (also via USB hub) and seeing what happens when you flick caching on and off. If I do this myself I'll let you know what I find out... :wink:
 
--- Content removed, as as it was in breach of forum rules. The same thoughts can be expressed in a form which meets forum requirements, if the poster cares to do so ---

[post moderated by DavidHB]
 
Hi silicondeath, thanks for your detailed explanations, I apologise for my long absence on the forum.

Your evidence has been very carefully collected, and I'm coming to the same conclusions that you are. The explanation for the decrease in drive performance as the erase progresses is that disks generally write data from the outermost rim to the innermost rim, at a constant rotational speed. As such, within a single unit of time, data can be written to the outermost parts of the disk faster than that in the inner tracks. An example would be an old SATA drive I had, 250gb in capacity, which would start out at 70mb/s at the start, and get to around 45-50 mb/s midway through the process.

With regards to USB erasures, USB2 limits erasures in terms of the maximum bandwidth between the computer and the drive. After which, it depends on the caddy to convert the disk instructions to SATA instructions. As you've tested the same model of drive on two different interfaces, I'm confident that the WD drives are great (and they generally are) and that the problem lies on either the caddy or the USB interface.

Next comes write caching. Eraser does pass the "write through" flag when opening files for writing. This should mean that writes, although going to a buffer to memory-align them, should be immediately written to disk. I've had some observations that Windows (or the disk drive... it's hard to tell) does not obey this (on some occasions, but never able to decisively prove it) but as of now there's no evidence that the flag is always ignored. Write caching should therefore have no effect on the drive.

So there is probably two things that can be done: a new caddy or a new USB controller :) I've tested erasures on USB drives before, and it seems to work decently... within the bounds of the disk performance. We can pursue this further if you'd like, drop me an email or contact me on MSN/WLM and we can discuss this.

Thanks again!
 
You must log in or register to reply here.
Back
Top