NewsLeecher Pauses while Downloading

[Spiril]

NewsLeecher Pauses while Downloading

"Pausing NewsLeecher Temporarily Message
Waiting for download queue catch-up, as articles are being downloaded faster than they can be written to disk."

If you frequently see the message above, while downloading with NewsLeecher, and don't know why, please try to do the following and see if that helps:

1.
If you download faster than what you harddisks are technically able to keep up with, there is nothing to do except for lowering your download speed in NewsLeecher.

2.
Download the latest NewsLeecher beta (or forum test version) release and see if that solves the problem. If it does, please report it in this forum thread.

3.
Try to change the download destination folder to another disk and see if that helps.

4.
Try to, *temporarily*, turn off Repair'n'Extract and any anti virus / trojan scanner application you might have installed. If that helps, you will have to tweak that application to work with NewsLeecher. That could, for example, be by turning off real-time file scanning in your anti virus app, etc..

5.
If you still see the 'pausing' message while downloading, please reply to this forum thread with the following info:
----------
a.
Q - What version of NewsLeecher have you tested with?

b.
Q - How fast is your internet connection?

c.
Q - Which disks have you tested this problem with?

d.
Q - Do you see the message all the time whe downloading with NewsLeecher, or only during specific downloads?

e.
If there is any relevant debug information in the NewsLeecher log (remember to use the latest NewsLeecher beta or forum test release while testing for this issue), make a copy / paste of it, and send to 'Spiril' in a PM forum message.
----------

EDIT 1:
I've just uploaded a new unofficial forum test which puts some relevant info in the log while downloading. Please do the testing using this version and copy/paste and attach the "*SaveThread*" log lines to a reply post to this forum thread. ( download link http://www.newsleecher.com/nl_downloadpausing.01.exe )
_________________

Bug fixed. No idea how. Hate it when that happens. Trying to break it again now. Will. not. be. defeated.

[p0W3Rh0u5e]

Here a few results of pausingbeta01, i mostly get fast writes, but sometimes they're very slow.

[Lundis2]

For me I never see it on larger files unless I have a very big
repair / extract queue.
On smaller files however it popup the pause message very often.
I got 100Mbit down.

Ill try to play around a bit using diffrent hds this weekend.

Lundis

[heyman421]

I'm getting this problem non-stop with 4.0 beta 11.

I tried restricting my bandwidth, restarting the computer, and deleting existing temporary files to no avail.

Download directory is on it's own 10k rpm Raptor hard drive, speed should not be an issue.

Full download speed is only 8mb, restricted is close to 3mb.

Is there a way to restore the ability to use a specified directory for my temp files? I had this working with previous betas by creating the config file in the newsleecher directory, but this newest beta seems to have disabled my ability to use this trick.

Used to have my temporary directory on a virtual disk that was partitioned into my memory, and this was never ever an issue, so perhaps the ability to change my temp directory would correct this issue.

I don't want to have to go back to 3.9 again! Everytime i switch back and forth between a 4.0 beta and 3.9 i lose all my settings and have to start over (takes at least 30 minutes or so each time to get everything the way i like,) so i'm dead in the water until further notice.

Computer's a 2.4ghz q6600 running windows 7 professional x64 with 8gb of memory. Primary disk and download disk are both 10k rpm raptors as mentioned before, and repair and extract extracts completed files to a 3rd raid array, on a completely different controller, which is easily capable of 150MB/s all day and all night.

Newsleecher has disconnected because a temporary download file could not be created.

Error message:
The operation completed successfully

Path:
D:\newsleecher\xxxxxxxxx\

File:
!!newsleecher download - do not delete - 94056833 !!

[sdfdsdf]

Hello everyone,

I had a similar issue to you all. I think I may have found something common to all of the queue items which are causing the issue for me.

If i look at the dl queue, the only posts which seem to error out (with the error message reporting "The operation has completed sucessfully") are the items with a " " (space) before the final \ in the Dl.Folder field. Try moving down any that have a space to the bottom. I did this on mine and the downloads worked for the non-space appended Dl.Folder downloads.

Note, at the moment, we are unable to change the dl folder to see if it makes any difference (turned off in b11).

Spirl, do you think you could test this out and confirm if it is the issue. Thanks,

[Willem_h]

Hi Guys,

Problem I'm having: Downloading with 60MB/s, on a 7200 RPM 1.5 TB drive (only used as 'download disk'), newsleecher hangs every time with certain downloads, eg, downloading 50MB files:
- 84 of 84 parts - no problem
- 131 of 131 parts - "Waiting for download queue to catch up"

It doesn't make a big difference enabling or disabling RnE.

Hope this helps! I'm still at v3.9.

[Meatsafe]

I have been experiencing the same problem. I have been seeing a lot of corrupted downloads and many Newsleecher do not delete files, and News leecher has paused on me.
I cracked open perfrmon and resource manager in Windows and added counters for disk throughput on my download drive. Whenever downloads were corrupting the failed write events (in Newsleecher’s log) corresponded with spikes in disk throughput.
I have nice fast SATA 2 disks running in a hardware RAID 0 stripe set for faster performance, but the underlying problem is that I’m just downloading way too fast on too many threads for my disk controller or spindles to write the data to the disk.
I can’t see this being an issue with the Newsleecher code, as the read and writes are handled by my OS and it’s Hardware Extraction Layer and drivers like any other read / write operation. (Unless newsleecher is somehow bypassing the Operating System and talking directly to the Disk controllers). The problem seemed to start for me when I upgraded my news server subscription to use more connections.
There are some options you can take to mitigate the risk of this happening. Option 3 is my recommendation:
1) INVEST - Use dedicated physical D: drive (internal or ESATA as an external drive using USB or USB2 probably won’t be fast enough) then set the download queue to use that drive. You can configure this in Settings \ Download in the NL user interface. This option will allow you to write faster to disk as your C drive typically has your operating system, applications and a page file running on it. You do need a dedicated disk through, as creating an additional partition on the same disk this won’t help improve disk throughput performance. Remember that you will need to stop the connection and reconnect to allow the change of download location to take effect.

2) SLOW DOWN - Reduce your download speed. Ok this isn’t ideal but you can set your download speed to match the throughput capacity your disk has. This isn’t removing the bottleneck your just preventing corruption with this option. I can’t imagine anyone being too happy doing this, but it will mean fewer corrupted downloads which will save you time in the long run.

3) OPTIMIZE - Optimize the number of concurrent connection you configure in manager \ server. By reducing the number of concurrent connections you make you won’t reduce the download speed but you will reduce the number of threads competing to write to the disk and take some of the strain of the disk queue. I dropped my number of connection down from 50 to 10 and I find that I’m still utilizing 7.2 Mega Bytes (yes bytes not bits) per second download (which is pretty darned fast). This has eliminated the write failures.

4) Check out your malware protection configuration. If your disk is being scanned this will reduce disk performance. I would be very wary of disabling anti malware protection but consider tuning it to ignore you download target folder, or schedule scans for off peak hours. Do be careful that you have real time protection enabled when you are extracting any files though.
Having implemented option 3 Downloads seem to be performing as fast as they did before and corruptions have stopped. I’d recommend reducing the number of connections to 10 (or maybe less if you have a slower drive or an overworked system) and then increasing the number of connections until you start seeing write failures in the newsleecher log.

[roburtvigr]

Wow very nice information. I really like it and it is really helpful to me. but can you give me tips for fast downloading because currently I am downloading any data to my hard disk at just 12 to 20 mbps. so give me some tips for improve it.

[addictedz]

OK I am not a happy camper. i have tried 3.9 and 4.0 beta. It will download at less speed than what my 35mbit connection is. I have a 75gb wd raptor 10,000 rpm brand new in this for my downloads area. I have tried many of my other hard drives as well. I did all of the stuff that is on the things to do list to fix this. I have even reformatted twice and did not install anything but newsleecher and my motherboard drivers, nvidia video drivers. I still have this problem. Ever since I reformatted my PC this has been every other 3-4 seconds it pauses then goes. This is not an ISP issue at all. There is something wrong with this software that needs to be fixed. If anyone has any other ideas what the problem could be please contact me. I have used NL for about 2 yrs now and never had this problem. If i am going to pay for a service i expect it to work correctly.

Again I have not had any problems with this until i re-downloaded this put in the key. I have been a computer tech for over 20 years. This is definitely a problem with NL and how it connects to the servers.

Contact me ASAP !


thanks,

[addictedz]

still no reply back ... is there a BUG OR WHAT

ANYONE?

I cannot figure this out....

[capone]

I'm either having this problem, or a variation of it. I just updated from 3 to 4 (no betas), and 3 did not have the issue. It threw a lot of errors, but I could close the error window and continue normally.

What happens to me is when getting headers. It seems to finish in the background, but the gui is forzen. When I force it closed and reopen, it has all the new headers waiting.

[Disco]

I can confirm that this is a problem to newsleecher, and specifically in the 4.x versions of newsleecher.

3.95 works fine no stalling
SabNZB works fine no stalling
In fact any software works fine with no stalling with the exception of Newsleecher 4.0 which stalls an is unusable. This problem is, and has always been there in the 4.x versions.

Please stop telling us it is hardware related or to slow down our downloads / swap hard drives etc... the problem is in your 4.x engine.

I love newsleecher, great product, and have supported it for many years now. but am wondering why im paying for the latest version, when it simply does not perform like the versions you made a few years back!



All the best.

[remco66]

Hi

well i had the same prob

but i solved mine within seconds



go to.....Tools....Settings.....Download....at the bottom you see ..pause if disk gets below....set it to 0...apply....close newsleecher...restart...problem gone!

i use
4.0 final and win 7

lots of luck!

[remco66]

and?

did anyone try ?




Happy new year everybody!

[Weaver]