NewsLeecher Pauses while Downloading

Post your questions here if you need help to use NewsLeecher or if you have a question about a feature.

User avatar
Spiril
Site Admin
Posts: 4278
Joined: Fri Nov 07, 2003 3:11 am

NewsLeecher Pauses while Downloading

Post by Spiril »

NewsLeecher Pauses while Downloading

"Pausing NewsLeecher Temporarily Message
Waiting for download queue catch-up, as articles are being downloaded faster than they can be written to disk.
"

If you frequently see the message above, while downloading with NewsLeecher, and don't know why, please try to do the following and see if that helps:

1.
If you download faster than what you harddisks are technically able to keep up with, there is nothing to do except for lowering your download speed in NewsLeecher.

2.
Download the latest NewsLeecher beta (or forum test version) release and see if that solves the problem. If it does, please report it in this forum thread.

3.
Try to change the download destination folder to another disk and see if that helps.

4.
Try to, *temporarily*, turn off Repair'n'Extract and any anti virus / trojan scanner application you might have installed. If that helps, you will have to tweak that application to work with NewsLeecher. That could, for example, be by turning off real-time file scanning in your anti virus app, etc..

5.
If you still see the 'pausing' message while downloading, please reply to this forum thread with the following info:
----------
a.
Q - What version of NewsLeecher have you tested with?

b.
Q - How fast is your internet connection?

c.
Q - Which disks have you tested this problem with?

d.
Q - Do you see the message all the time whe downloading with NewsLeecher, or only during specific downloads?

e.
If there is any relevant debug information in the NewsLeecher log (remember to use the latest NewsLeecher beta or forum test release while testing for this issue), make a copy / paste of it, and send to 'Spiril' in a PM forum message.
----------

EDIT 1:
I've just uploaded a new unofficial forum test which puts some relevant info in the log while downloading. Please do the testing using this version and copy/paste and attach the "*SaveThread*" log lines to a reply post to this forum thread. ( download link http://www.newsleecher.com/nl_downloadpausing.01.exe )
bug fixed. no idea how. hate it when that happens. trying to break it again now. will. not. be. defeated.

p0W3Rh0u5e
Posts: 176
Joined: Thu Apr 21, 2005 6:46 pm

Post by p0W3Rh0u5e »

Here a few results of pausingbeta01, i mostly get fast writes, but sometimes they're very slow.

Code: Select all

19:40:16  * SaveThread  ::  Command    :: X:\Incoming\<coolstuff>\*13753511*77625601*77875200*312>
19:40:16  * SaveThread  ::  Seek Time  ::   0,0ms
19:40:16  * SaveThread  ::  Write Time ::   0,2ms  ::  Bytes Written: 249.600  ::  Per Sec: ~ 1112,323 MB  ::  SaveThreadListCnt: 21  ::  Code Executed: Simple OS " fileWrite(fileHandle, buffer[1], length(buffer)) ".
19:40:16  ----
19:40:16  * SaveThread  ::  Command    :: <savePart*X:\Incoming\<coolstuff>\*13753511*70636801*70886400*284>
19:40:16  * SaveThread  ::  Seek Time  ::   0,0ms
19:40:16  * SaveThread  ::  Write Time ::  15,7ms  ::  Bytes Written: 249.600  ::  Per Sec: ~ 15,126 MB  ::  SaveThreadListCnt: 20  ::  Code Executed: Simple OS " fileWrite(fileHandle, buffer[1], length(buffer)) ".
19:40:16  ----
19:40:16  * SaveThread  ::  Command    :: <savePart*X:\Incoming\<coolstuff>\*13753511*79123201*79372800*318>
19:40:16  * SaveThread  ::  Seek Time  ::   0,0ms
19:40:16  * SaveThread  ::  Write Time ::   0,4ms  ::  Bytes Written: 249.600  ::  Per Sec: ~ 670,527 MB  ::  SaveThreadListCnt: 19  ::  Code Executed: Simple OS " fileWrite(fileHandle, buffer[1], length(buffer)) ".
19:40:16  ----
19:40:16  * SaveThread  ::  Command    :: <savePart*X:\Incoming\<coolstuff>\*13753511*78624001*78873600*316>
19:40:16  * SaveThread  ::  Seek Time  ::   0,0ms
19:40:16  * SaveThread  ::  Write Time ::   0,4ms  ::  Bytes Written: 249.600  ::  Per Sec: ~ 655,750 MB  ::  SaveThreadListCnt: 18  ::  Code Executed: Simple OS " fileWrite(fileHandle, buffer[1], length(buffer)) ".
19:40:16  ----
19:40:16  * SaveThread  ::  Command    :: <savePart*X:\Incoming\<coolstuff>\*13753511*72883201*73132800*293>
19:40:16  * SaveThread  ::  Seek Time  ::   0,0ms
19:40:16  * SaveThread  ::  Write Time :: 362,9ms  ::  Bytes Written: 249.600  ::  Per Sec: ~ 0,656 MB  ::  SaveThreadListCnt: 17  ::  Code Executed: Simple OS " fileWrite(fileHandle, buffer[1], length(buffer)) ".
19:40:16  ----
19:40:16  * SaveThread  ::  Command    :: <savePart*X:\Incoming\<coolstuff>\*13753511*80121601*80371200*322>
19:40:16  * SaveThread  ::  Seek Time  ::   0,0ms
19:40:16  * SaveThread  ::  Write Time ::   0,1ms  ::  Bytes Written: 249.600  ::  Per Sec: ~ 1935,261 MB  ::  SaveThreadListCnt: 16  ::  Code Executed: Simple OS " fileWrite(fileHandle, buffer[1], length(buffer)) ".
19:40:16  ----
19:40:16  * SaveThread  ::  Command    :: <savePart*X:\Incoming\<coolstuff>\*13753511*78374401*78624000*315>
19:40:16  * SaveThread  ::  Seek Time  ::   0,0ms
19:40:16  * SaveThread  ::  Write Time ::   0,1ms  ::  Bytes Written: 249.600  ::  Per Sec: ~ 2000,312 MB  ::  SaveThreadListCnt: 15  ::  Code Executed: Simple OS " fileWrite(fileHandle, buffer[1], length(buffer)) ".
19:40:16  ----
19:40:16  * SaveThread  ::  Command    :: <savePart*X:\Incoming\<coolstuff>\*13753511*80620801*80870400*324>
19:40:16  * SaveThread  ::  Seek Time  ::   0,0ms
19:40:17  * SaveThread  ::  Write Time :: 278,7ms  ::  Bytes Written: 249.600  ::  Per Sec: ~ 0,854 MB  ::  SaveThreadListCnt: 14  ::  Code Executed: Simple OS " fileWrite(fileHandle, buffer[1], length(buffer)) ".
19:40:17  ----
19:40:17  * SaveThread  ::  Command    :: <savePart*X:\Incoming\<coolstuff>\*13753511*92601601*92851200*372>
19:40:17  * SaveThread  ::  Seek Time  ::   0,0ms
19:40:34  * SaveThread  ::  Write Time :: 17460,5ms  ::  Bytes Written: 249.600  ::  Per Sec: ~ 0,014 MB  ::  SaveThreadListCnt: 13  ::  Code Executed: Simple OS " fileWrite(fileHandle, buffer[1], length(buffer)) ".
19:40:34  ----
19:40:34  * SaveThread  ::  Command    :: <savePart*X:\Incoming\<coolstuff>\*13753511*84864001*85113600*341>
19:40:34  * SaveThread  ::  Seek Time  ::   0,0ms
19:40:35  * SaveThread  ::  Write Time :: 1113,2ms  ::  Bytes Written: 249.600  ::  Per Sec: ~ 0,214 MB  ::  SaveThreadListCnt: 12  ::  Code Executed: Simple OS " fileWrite(fileHandle, buffer[1], length(buffer)) ".
19:40:35  ----
19:40:35  * SaveThread  ::  Command    :: <savePart*X:\Incoming\<coolstuff>\*13753511*92851201*93100800*373>
19:40:35  * SaveThread  ::  Seek Time  ::   0,0ms
19:40:35  * SaveThread  ::  Write Time ::   0,2ms  ::  Bytes Written: 249.600  ::  Per Sec: ~ 1360,212 MB  ::  SaveThreadListCnt: 11  ::  Code Executed: Simple OS " fileWrite(fileHandle, buffer[1], length(buffer)) ".
19:40:35  ----
19:40:35  * SaveThread  ::  Command    :: <savePart*X:\Incoming\<coolstuff>\*13753511*91104001*91353600*366>
19:40:35  * SaveThread  ::  Seek Time  ::   0,0ms
19:40:35  * SaveThread  ::  Write Time ::   3,8ms  ::  Bytes Written: 249.600  ::  Per Sec: ~ 61,973 MB  ::  SaveThreadListCnt: 10  ::  Code Executed: Simple OS " fileWrite(fileHandle, buffer[1], length(buffer)) ".
19:40:35  ----
19:40:35  * SaveThread  ::  Command    :: <savePart*X:\Incoming\<coolstuff>\*13753511*84115201*84364800*338>
19:40:35  * SaveThread  ::  Seek Time  ::   0,0ms
19:40:35  * SaveThread  ::  Write Time ::  84,8ms  ::  Bytes Written: 249.600  ::  Per Sec: ~ 2,806 MB  ::  SaveThreadListCnt: 9  ::  Code Executed: Simple OS " fileWrite(fileHandle, buffer[1], length(buffer)) ".
19:40:35  ----
19:40:35  * SaveThread  ::  Command    :: <savePart*X:\Incoming\<coolstuff>\*13753511*89356801*89606400*359>
19:40:35  * SaveThread  ::  Seek Time  ::   0,0ms
19:40:36  * SaveThread  ::  Write Time ::  98,7ms  ::  Bytes Written: 249.600  ::  Per Sec: ~ 2,413 MB  ::  SaveThreadListCnt: 8  ::  Code Executed: Simple OS " fileWrite(fileHandle, buffer[1], length(buffer)) ".
19:40:36  ----
19:40:36  * SaveThread  ::  Command    :: <savePart*X:\Incoming\<coolstuff>\*13753511*80870401*81120000*325>
19:40:36  * SaveThread  ::  Seek Time  ::   0,0ms
19:40:36  * SaveThread  ::  Write Time ::  78,3ms  ::  Bytes Written: 249.600  ::  Per Sec: ~ 3,040 MB  ::  SaveThreadListCnt: 7  ::  Code Executed: Simple OS " fileWrite(fileHandle, buffer[1], length(buffer)) ".
Internetconnection: 50MBit/s
Disk: 3WARE-Raid6 with 8*1.5TB HDDs and 512MB Cache / Intel RAID1 wth 2 older 300GB-HDDs.
I see the message more often, when article sizes are smaller.

Lundis2
Posts: 11
Joined: Thu May 14, 2009 10:15 pm

smaller files

Post by Lundis2 »

For me I never see it on larger files unless I have a very big
repair / extract queue.
On smaller files however it popup the pause message very often.
I got 100Mbit down.

Ill try to play around a bit using diffrent hds this weekend.

Lundis

heyman421
Posts: 16
Joined: Tue May 01, 2007 6:20 am
Contact:

Post by heyman421 »

~~Deleted~~
Last edited by heyman421 on Tue Jul 25, 2017 5:21 am, edited 1 time in total.

sdfdsdf
Posts: 25
Joined: Fri Dec 05, 2003 4:53 am

Post by sdfdsdf »

Hello everyone,

I had a similar issue to you all. I think I may have found something common to all of the queue items which are causing the issue for me.

If i look at the dl queue, the only posts which seem to error out (with the error message reporting "The operation has completed sucessfully") are the items with a " " (space) before the final \ in the Dl.Folder field. Try moving down any that have a space to the bottom. I did this on mine and the downloads worked for the non-space appended Dl.Folder downloads.

Note, at the moment, we are unable to change the dl folder to see if it makes any difference (turned off in b11).

Spirl, do you think you could test this out and confirm if it is the issue. Thanks,

Willem_h
Posts: 1
Joined: Tue Mar 09, 2010 5:36 pm

Post by Willem_h »

Hi Guys,

Problem I'm having: Downloading with 60MB/s, on a 7200 RPM 1.5 TB drive (only used as 'download disk'), newsleecher hangs every time with certain downloads, eg, downloading 50MB files:
- 84 of 84 parts - no problem
- 131 of 131 parts - "Waiting for download queue to catch up"

It doesn't make a big difference enabling or disabling RnE.

Hope this helps! I'm still at v3.9.

Meatsafe
Posts: 1
Joined: Sun Apr 18, 2010 3:27 pm

Fix for Paused Download and Save Thread Error issues

Post by Meatsafe »

I have been experiencing the same problem. I have been seeing a lot of corrupted downloads and many Newsleecher do not delete files, and News leecher has paused on me.
I cracked open perfrmon and resource manager in Windows and added counters for disk throughput on my download drive. Whenever downloads were corrupting the failed write events (in Newsleecher’s log) corresponded with spikes in disk throughput.
I have nice fast SATA 2 disks running in a hardware RAID 0 stripe set for faster performance, but the underlying problem is that I’m just downloading way too fast on too many threads for my disk controller or spindles to write the data to the disk.
I can’t see this being an issue with the Newsleecher code, as the read and writes are handled by my OS and it’s Hardware Extraction Layer and drivers like any other read / write operation. (Unless newsleecher is somehow bypassing the Operating System and talking directly to the Disk controllers). The problem seemed to start for me when I upgraded my news server subscription to use more connections.
There are some options you can take to mitigate the risk of this happening. Option 3 is my recommendation:
1) INVEST - Use dedicated physical D: drive (internal or ESATA as an external drive using USB or USB2 probably won’t be fast enough) then set the download queue to use that drive. You can configure this in Settings \ Download in the NL user interface. This option will allow you to write faster to disk as your C drive typically has your operating system, applications and a page file running on it. You do need a dedicated disk through, as creating an additional partition on the same disk this won’t help improve disk throughput performance. Remember that you will need to stop the connection and reconnect to allow the change of download location to take effect.

2) SLOW DOWN - Reduce your download speed. Ok this isn’t ideal but you can set your download speed to match the throughput capacity your disk has. This isn’t removing the bottleneck your just preventing corruption with this option. I can’t imagine anyone being too happy doing this, but it will mean fewer corrupted downloads which will save you time in the long run.

3) OPTIMIZE - Optimize the number of concurrent connection you configure in manager \ server. By reducing the number of concurrent connections you make you won’t reduce the download speed but you will reduce the number of threads competing to write to the disk and take some of the strain of the disk queue. I dropped my number of connection down from 50 to 10 and I find that I’m still utilizing 7.2 Mega Bytes (yes bytes not bits) per second download (which is pretty darned fast). This has eliminated the write failures.

4) Check out your malware protection configuration. If your disk is being scanned this will reduce disk performance. I would be very wary of disabling anti malware protection but consider tuning it to ignore you download target folder, or schedule scans for off peak hours. Do be careful that you have real time protection enabled when you are extracting any files though.
Having implemented option 3 Downloads seem to be performing as fast as they did before and corruptions have stopped. I’d recommend reducing the number of connections to 10 (or maybe less if you have a slower drive or an overworked system) and then increasing the number of connections until you start seeing write failures in the newsleecher log.

roburtvigr
Posts: 4
Joined: Thu Aug 12, 2010 6:13 am

Post by roburtvigr »

Wow very nice information. I really like it and it is really helpful to me. but can you give me tips for fast downloading because currently I am downloading any data to my hard disk at just 12 to 20 mbps. so give me some tips for improve it.

addictedz
Posts: 3
Joined: Fri Jan 23, 2009 7:33 am

Ever since i reformatted my pc my NL pauses

Post by addictedz »

OK I am not a happy camper. i have tried 3.9 and 4.0 beta. It will download at less speed than what my 35mbit connection is. I have a 75gb wd raptor 10,000 rpm brand new in this for my downloads area. I have tried many of my other hard drives as well. I did all of the stuff that is on the things to do list to fix this. I have even reformatted twice and did not install anything but newsleecher and my motherboard drivers, nvidia video drivers. I still have this problem. Ever since I reformatted my PC this has been every other 3-4 seconds it pauses then goes. This is not an ISP issue at all. There is something wrong with this software that needs to be fixed. If anyone has any other ideas what the problem could be please contact me. I have used NL for about 2 yrs now and never had this problem. If i am going to pay for a service i expect it to work correctly.

Again I have not had any problems with this until i re-downloaded this put in the key. I have been a computer tech for over 20 years. This is definitely a problem with NL and how it connects to the servers.

Contact me ASAP !


thanks,

addictedz
Posts: 3
Joined: Fri Jan 23, 2009 7:33 am

Post by addictedz »

still no reply back ... is there a BUG OR WHAT

ANYONE?

I cannot figure this out....

capone
Posts: 6
Joined: Fri May 12, 2006 8:29 pm
Location: Maryland, US

Post by capone »

I'm either having this problem, or a variation of it. I just updated from 3 to 4 (no betas), and 3 did not have the issue. It threw a lot of errors, but I could close the error window and continue normally.

What happens to me is when getting headers. It seems to finish in the background, but the gui is forzen. When I force it closed and reopen, it has all the new headers waiting.

Disco
Posts: 2
Joined: Fri Dec 10, 2010 9:59 pm

Cofirmation that this is a problem in Version 4.0 Final

Post by Disco »

I can confirm that this is a problem to newsleecher, and specifically in the 4.x versions of newsleecher.

3.95 works fine no stalling
SabNZB works fine no stalling
In fact any software works fine with no stalling with the exception of Newsleecher 4.0 which stalls an is unusable. This problem is, and has always been there in the 4.x versions.

Please stop telling us it is hardware related or to slow down our downloads / swap hard drives etc... the problem is in your 4.x engine.

I love newsleecher, great product, and have supported it for many years now. but am wondering why im paying for the latest version, when it simply does not perform like the versions you made a few years back!



All the best.

remco66
Posts: 2
Joined: Tue Dec 21, 2010 2:30 pm

Post by remco66 »

Hi

well i had the same prob

but i solved mine within seconds



go to.....Tools....Settings.....Download....at the bottom you see ..pause if disk gets below....set it to 0...apply....close newsleecher...restart...problem gone!

i use
4.0 final and win 7

lots of luck!

remco66
Posts: 2
Joined: Tue Dec 21, 2010 2:30 pm

Post by remco66 »

and?

did anyone try ?




Happy new year everybody!

Weaver
Posts: 4
Joined: Sat Aug 20, 2005 3:45 pm

Post by Weaver »

remco66 wrote:and?

did anyone try ?




Happy new year everybody!
Nope, but reduced the number of threats. That helped for me. Had five going and went to three. Same speed and no notice/pausing

Post Reply