Continuous updating of Queue status so crashes don't screw up download history.

[jculp]

When crashes (spinning ball and Unresponsive App) occur, the Queue history is not accurately saved.
When app is restarted, the Queue is rebuilt to last saved state even if lots of files have been downloaded already, necessitating manual review and deletion of duplicate Queue entries to reflect updated download status. Then quit of app to force the Queue to save normally. Then restart to continue downloading

A fail-safe to preserve Queue status and Download history in Megasearch results would be a great Database feature.

[Support]

Good idea, except that a large queued object can have ten thousands or more articles. Constantly updating the database for each downloaded articles will have a huge impact on the resources and app. A better choice would be to update the database for each finished or newly added queue object. I'll put this on the todo list.

BTW. At which point do the crashes occur? Or is the resident app you mentioned before still interfering?

When crashes (spinning ball and Unresponsive App) occur, the Queue history is not accurately saved.
When app is restarted, the Queue is rebuilt to last saved state even if lots of files have been downloaded already, necessitating manual review and deletion of duplicate Queue entries to reflect updated download status. Then quit of app to force the Queue to save normally. Then restart to continue downloading

A fail-safe to preserve Queue status and Download history in Megasearch results would be a great Database feature.

[jculp]

A better choice would be to update the database for each finished or newly added queue object. I'll put this on the todo list.

I agree and that was what I was intending to suggest. Just didn't know how to word it.

BTW. At which point do the crashes occur? Or is the resident app you mentioned before still interfering?

Usenet hangs when various changes occur to Queue.
So far I've noted it does it sporadically when
1. moving queue entries up or down multiple time. (Perhaps it dislikes the abrupt interruption to the file parts downloads changing)
2. When Entries are no longer available from the time it was added to the queue till the time it actually comes up for download (this occurs with large download Queue that may take 15-24 hours to get through)
3. deleting queue entries repeatedly after checking content of rars and terminating/deleting the downloads of redundant or incomplete file sets.
(have noted a lot of posted files are missing enough rar parts that even the par recovery set cannot revive the download - perhaps a pre-check for complete rar sequential sets can be built in to MegaSearch to eliminate these results as they are just wasted download bandwidth and time. Similar to the option to eliminate password protected files)

To work around this I have manually checked the original search results and noted that certain posters are guilty of this practice and I try to avoid them.
I also check the Queue files to verify the contents of the file sets are all sequentially present and complete.
I routinely quit and restart the App to keep the database updated in case of a crash.


Overall, the app is getting more stable and useful. I have been very impressed with your updates and responsiveness.

Thanks for the great work.

[Support]

Thank you for the follow up.
Your detailed information is very useful.

However, I have one more question.
What settings are you using to cleanup the queue.
Is it "Auto cleanup", "Manual cleanup" or "Auto reorder"?
And did you ever tried a different setting with better results?

Thanks.

[jculp]

Clean up has always been set to Auto CleanUp.

The Queue is auto cleaning the list as it completes downloads and completes extraction into the designated extraction folder.
When the spinning ball occurs, a force quit is required and no db saved state is updated..
When restarting the app, it reloads the last properly saved state, regardless of what was previously downloaded and Auto Cleaned since that proper saved state db.

Is their a functional difference among the cleanup methods?

[jculp]

I may be using the term "crash" incorrectly.

The Usenetic app becomes unresponsive and I get the spinning ball, so technically not a crash of the app but it hangs and it requires a force quit and then restart.

I'll forward the logs the next time it happens.

Hope this helps.

[Support]

I may be using the term "crash" incorrectly.

The Usenetic app becomes unresponsive and I get the spinning ball, so technically not a crash of the app but it hangs and it requires a force quit and then restart.

Ah, that is something totally different and more explicable

When a large queue object is finished, Usenetic safely cleans the used memory. This is done in several steps to avoid crashing. This can take a while for very large objects and will freeze the interface because this is done on the main thread. That gives you the idea that the app hangs, but if you wait long enough it should become responsively again and will continue with the remaining tasks.

To fix this I'll change the cleaning routines and let them work in a different thread than the main thread.

Thanks.

[jculp]

Thanks for the clarification.

It happens often enough that I get impatient after 5-10 seconds.

It happened again (hung for over a hour), so I took a screen shot of Queue and copied the force quit report and have enclosed for your review.

Hope it helps.

[Support]

According to the log you attached, I was wrong with my previous theory.
This problem is difficult to track and unfortunately I can't reproduce it here.

Can you please answer the following questions that might help to tackle the issue?

1) I see you are using the bandwidth limiter. Does the same issue happens when the bandwidth limiter is off?
2) Did this issue started since a few releases back or has it always been there?
3) Does the hang also occurs by lets say 10 or less queued items or just with 100+?
4) Does the hang just occurs while adding more items to the queue?

Thanks so far.

[jculp]

1) I see you are using the bandwidth limiter. Does the same issue happens when the bandwidth limiter is off?

I have always had the limiter check as on. I will try with it off to see if it recurs.

2) Did this issue started since a few releases back or has it always been there?

Now that you mention it it does seem that this particular hang is occurring more frequently since the last update.

3) Does the hang also occurs by lets say 10 or less queued items or just with 100+?

The hang usually occurs with larger queues with larger file sets.

4) Does the hang just occurs while adding more items to the queue?

Have not noted that occurring. Adds to Queue have been fine.
Removing too quickly multiple file sets (especially ones that are in downloading process) however can hang the Queue and app.
If I pause the downloads and wait for the progress bar to stop moving, deleting multiple file sets are more stable.




Perhaps related to #2 the following has occurred several times since the last update:
It also seems that it "loses server connection" when more than two file sets are no longer available for download, no dialogue box pops up stating this, it does not reclassify it with the red badge, just no download and everything in Queue is paused with the explanation of "lost server connection". I have included a screenshot.
I test the internet and no problems show up.
When I recycle a file set farther down the Queue list it starts downloading it fine.
I then quit and restart the app to refresh the Queue db.
When I try to download the two or three file sets that were not downloading nothing happens, so I pause them.
I go farther down the list repeating it till I get to one that starts downloading. Usually the third or fourth.

I'll try to do a screen recording and edit it to the pertinent section where it occurs.