BUG: Articles "Hang" in Transfer Queue

[Herd]

Ive found that while some posts hang at roughly 99% if I disconnect and switch to a new server the post will finish. If you dont have a secondary server to use then I reccommend using a free server just to finish the files. The one I use doesnt actually contain any binaries so NL sees it as incomplete and finishes the file at the position where it had hung. This typically results in needing 1 block from a par but it beats the alternative of having to download a much larger par or not getting the file at all.

A quick google search will turn up various lists of free servers but to save you some time the one Im using is freenews.netfront.net . I hope this workaround helps you guys out.

[Lebowski]

I have had this problem alot, and was wondering about it.
Say I have multiple servers (+10) utilizing 2-4 connections each, all from different usenet providers.
I am downloading an nzb or even just a batch of files from SS
Is there a way for the servers to check with each other to see if the files are incomplete.

Eg: Download a batch of .rar files (a.b.mm new tv show) that are all 20,523kb. But after, I run quickpar and find that many of the files are 15 or 16 mb and not complete, but they moved out of the transfer queue without a problem.

It would be great if newsleecher would check the file sizes of each rar against all the servers and SS to see if any are incomplete, like if giganews had it complete and easynews didn't, it would grab the incomplete portion from giganews and complete the .rar.

The way I get it to complete is to shut down all the servers except one, normally giganews, and redownload the file from newsleecher.

Any ideas?

[Smudge]

If all the servers are the same priority then NL will download segments of the files from all servers at once. If one fails, it will flag that file as incomplete and continue on.

Only if there are servers with multiple priorities then will NL try other servers if the first one fails.

I'd recommend adding a server entry for GN with a priority lower than all the others. This way if any server fails to get a segment, it will try GN one more time.
_________________
Please be aware of and use the following pages...
Services Status Page : SuperSearch and Usenet Access server status, retention, server load, indexing time, etc.
Support Request Form : Use this if you have a problem with billing or account status. The forum is only for NewsLeecher application issues.
_________________
It isn't God I have a problem with. It is his fan club I cannot stand.

[Lebowski]

ahhh so thats what the priority is for.
Ya, i think i'll add NewsgroupDirect.com as the 2nd, because it's almost as good as GN, and GN as 3rd as the last resort

Thanks for the tip!

[eriji]

Here is a consistent problem as far as I can tell.

In this report I'm working with the alt.binaries.sounds.mp3.sound-effects group.

I use Giganews. They archive for 90+ days. I'm now on 3.8b6 but the problem has existed for some time, and since at least 3.6beta.

If I try to enque and download ANY article from this group that is older than about 86.4 days (it does vary a little) then the article Hangs:

The article status shows Initializing and will never do anything, but will eventually "time out"

There is no error in the log, (nor any "starting" message) but after 20 or so seconds (the timeout I have specified in Options) it will show a "resetting hanging connection." (see example in the next post)

Articles that are newer that 86.4 days will download OK.

IF I use Agent to access the same articles that Newsleecher hangs on, then Agent WILL RETRIEVE the articles with NO PROBLEM.

I contacted Giganews support, and they suggested that I clear my group cache, so I did purge it, but with no change in symptom.

I does not seem to matter if I select the article from the group article listing, from SuperSearch or from a NZB file. They all fail.

I have had this problem consistently since the Giganews upgrade to 70 days earlier this year. Before that, I never noticed it.

One thing that I just thought of... aren't there 86,400 seconds in a day?
Coincidence?

Hopefully this will be some help.

[eriji]

Here's another older article in a different group where the retrieval of an article hung and never completed.

Agent was able to retrieve the same article a couple of minutes later.

The article was listed in Newsleecher as 83 days old.

The only message in the log was a time-out after 21 seconds
12-17-06 22:23:43 Resetting hanging connection on the "Giga News" server. (21 seconds of inactivity)
12-17-06 22:23:43 Bot(4) disconnected from server: "Giga News". (0)

I have my time out set at 20 seconds.

Here is the article information:
<!-- Generator: NewsLeecher v3.8 Beta 6 (http://www.newsleecher.com/) <!DOCTYPE nzb PUBLIC "-//newzBin//DTD NZB 1.0//EN" "http://www.newzbin.com/DTD/nzb/nzb-1.0.dtd">
<nzb xmlns="http://www.newzbin.com/DTD/2003/nzb">
<file subject="cinderella (julie andrews) - read nfo-&gt;(21/22) - yenc &quot;cinderella.nfo&quot; (1/1)" date="1159225850" poster="nospam@noot.com (Nôôt)">
<groups><group>alt.binaries.sounds.mp3.soundtracks</group></groups>
<segments><segment number="1" bytes="3072">451845a6$0$14751$c3e8da3@news.astraweb.com</segment></segments>
</file>

[dwazegek]

The only explanation I can think of is that for some reason the articles won't download if they're requested by message-ID, but they do if they're requested by article number.
Do the files still download if you use an NZB to queue them up in agent?
_________________
Forum's Grammar Nazi:
I will hunt you down if you don't lose the extra o.

[eriji]

I though of doing that, but I couldn't figure out how to get Agent to do that without changing my global file associations. I'll look again and try to figure it out with an example.

From what the support guy at Giganews said (Jeremy) he could retrieve the message any which way without a problem.

I'll see what I can figure out with Agent, and try it.

[eriji]

After poking around it appears that I cannot retrieve any "old" messages in any group. Old being something over around 80 days, and that number varies by group. Most groups, its around 84.6 days others its as low as 80 days. I've not found any more than 86 days.

This is a problem unique to Newsleecher. Agent gets the same messages from the same groups just fine.

There doesn't seem to be a way to open a NZB file in Agent, so I can't test that there, but it seems to me that just retrieving the message VS hanging is enough indication of a problem with Newsleecher.

Reminder, I did not see this problem until Giganews went over 60 days. Maybe newsleecher is counting something in a small variable which can't count that high. Stranger things have happened.

If any of the developers want to correspond with me to try and get more details, send me a private message.

--eric

[shinypate1]

Have been d/l a post from premium server, using two different premium servers. Imported the nzb, post started up and now more than half of the posts are hung up, between 99 and 100% complete, just won't post to the d/l. I supersearched and got the original headers without nzb, then reloaded... still hung up. Looks like we have a bug that's repeatable, since Agent d/l no problem.

Suggestion is to find a way to force NL to post an incomplete post.

Shiny

"It's not people who bother me, it's the smell that irritates me."

[eriji]

NL-3.8b8.
Ok, again, I have articles that are 83 days old. I'm on GigaNews. They archive for 90 days. I try to retrieve them with Newsleecher. Each of 29 articles with the same age from a.b.e-book.palm hang in the queue and newsleecher logs:

01-05-07 22:02:40 Resetting hanging connection on the "Giga News" server. (22 seconds of inactivity)
01-05-07 22:02:40 Bot(2) disconnected from server: "Giga News". (0)
01-05-07 22:02:40 Resetting hanging connection on the "Giga News" server. (22 seconds of inactivity)
01-05-07 22:02:40 Bot(4) disconnected from server: "Giga News". (0)
01-05-07 22:02:40 Resetting hanging connection on the "Giga News" server. (22 seconds of inactivity)
01-05-07 22:02:40 Bot(5) disconnected from server: "Giga News". (0)
01-05-07 22:02:40 Resetting hanging connection on the "Giga News" server. (22 seconds of inactivity)
01-05-07 22:02:40 Bot( disconnected from server: "Giga News". (0)

for each message as it retries them over and over and over and over and over and over.

It never retrieves them.

In the meantime, on the same computer, same GigaNews account, same articles, same newsgroup and in less time,

AGENT retrieves the articles.

quick and simple.

Newsleecher is STILL trying to retrieve them (they are still in the queue, queued and bots are trying them one by one:
01-05-07 22:25:06 Resetting hanging connection on the "Giga News" server. (22 seconds of inactivity)
01-05-07 22:25:06 Bot(1) disconnected from server: "Giga News". (0)
01-05-07 22:25:06 Resetting hanging connection on the "Giga News" server. (22 seconds of inactivity)
01-05-07 22:25:06 Bot(5) disconnected from server: "Giga News". (0)
01-05-07 22:25:06 Resetting hanging connection on the "Giga News" server. (22 seconds of inactivity)
01-05-07 22:25:06 Bot(6) disconnected from server: "Giga News". (0)
01-05-07 22:25:06 Resetting hanging connection on the "Giga News" server. (22 seconds of inactivity)
01-05-07 22:25:06 Bot(7) disconnected from server: "Giga News". (0)


It has to be something that Newsleecher is doing wrong in talking to the server.... everyone keeps on trying to find some excuse in something else... nzb, cache, etc. but this seems to be simple, and I can sure reproduce it any day of the week.

IF AN ARTICLE IS OLDER THAN 80 DAYS then Newsleecher will probably not retrieve it. Use Agent.

Problems with Vista are getting fixed, but not problems leeching news articles. Whats wrong here?

[Smudge]

if you still have it, can you send support@newsleecher.com a copy of the NZB of these exact articles that are hanging so i can test it?
_________________
Please be aware of and use the following pages...
Services Status Page : SuperSearch and Usenet Access server status, retention, server load, indexing time, etc.
Support Request Form : Use this if you have a problem with billing or account status. The forum is only for NewsLeecher application issues.
_________________
It isn't God I have a problem with. It is his fan club I cannot stand.

[eriji]

I just sent the email a minute or so ago.

[Smudge]

I received the NZB and tested with both the standard port 119 and the SSL port to the US servers. Also downloaded as a primary server and as a secondary (Newshosting primary)

All 4 times, I downloaded without any problems.

Any suggestions of your setup that I can try in order to reproduce this problem?
_________________
Please be aware of and use the following pages...
Services Status Page : SuperSearch and Usenet Access server status, retention, server load, indexing time, etc.
Support Request Form : Use this if you have a problem with billing or account status. The forum is only for NewsLeecher application issues.
_________________
It isn't God I have a problem with. It is his fan club I cannot stand.

[eriji]

Were you using Giganews?

I have two active servers set up. Giganews at 01 priority (8 conn) and Charter.net at 02 priority (02 conn)

All settings are the same between the two. In advanced, I have send group command checked (was default)

I have never seen NL download an article from charter, although it does update headers from there for each group.

<shrugs> Otherwise, I haven't set anything up special, everything is default.