Thread 'CPDN: Tasks using a lot of memory.'

Message boards : Projects : CPDN: Tasks using a lot of memory.
Message board moderation

To post messages, you must log in.

AuthorMessage
ProfileDave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2691
United Kingdom
Message 91281 - Posted: 30 Apr 2019, 13:53:35 UTC

Looking at tasks just completed the following is typical.

Peak working set size 4,218.61 MB
Peak swap size 4,818.18 MB
Peak disk usage 1,638.14 MB


Looking at, "top" shows one task peaking at about 53% of available memory 8GB.

I haven't yet tried running two or more of these tasks at once. After the peak they drop back down to about 25% or less. Am I likely to crash one or more if I try running two or more at once or will BOINC stop me? This would be the ideal if BOINC would just refuse to run two tasks at once but I don't know enough (anything) about how it will behave in this situation. - Maybe the project people know about this anyway. Will ask Andy if he knows.

It won't let me run the tasks at all on the desktop which is an older machine which is maxed out with 4GB. Which may well mean it won't let me run more than I have memory for at once.
ID: 91281 · Report as offensive
Dr Who Fan
Avatar

Send message
Joined: 10 May 07
Posts: 1442
United States
Message 91286 - Posted: 30 Apr 2019, 14:20:24 UTC - in response to Message 91281.  

You don't specify what project. That would help.
ID: 91286 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5128
United Kingdom
Message 91287 - Posted: 30 Apr 2019, 14:25:00 UTC - in response to Message 91286.  

Since it's Dave, I'm assuming CPDN.

[Mod hat on]
I'll leave it here for a day or two to catch the attention of general readers - see if they can help.

Then I'll move it into the Projects area with a CPDN tag in the title, for future reference.
[Mod hat off]
ID: 91287 · Report as offensive
ProfileDave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2691
United Kingdom
Message 91320 - Posted: 1 May 2019, 12:42:56 UTC

Am I likely to crash one or more if I try running two or more at once or will BOINC stop me? This would be the ideal if BOINC would just refuse to run two tasks at once but I don't know enough (anything) about how it will behave in this situation. - Maybe the project people know about this anyway. Will ask Andy if he knows.


Maybe I just need to read up on how Linux and BOINC deal with memory though I suspect I will have a lot of background reading to do before I stand a chance of understanding the bits with the answers.
ID: 91320 · Report as offensive
ProfileDave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2691
United Kingdom
Message 91352 - Posted: 2 May 2019, 19:23:02 UTC - in response to Message 91320.  

OK, starting to get answers through running tasks. Two tasks of another type running that use a lot less memory. I downloaded a bunch more and because of differing deadlines that I hadn't noticed, BOINC decided it knew best and ran three of the memory heavy tasks to completion without problems. Based on this I enabled the fourth core and I currently have three tasks at over 90% complete so I would guess that however it does it Linux and BOINC together can manage the memory requirements for these tasks. They are however very non linear in progress. Slowing down a lot in the last 5% with the last two minutes shown under remaining (estimated) taking about half an hour on my relatively slow laptop.

No great problem, so long as one knows about it.
ID: 91352 · Report as offensive
ProfileDave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2691
United Kingdom
Message 91357 - Posted: 3 May 2019, 8:08:26 UTC - in response to Message 91352.  

And looking at the figures for the time taken against number of tasks with 8GB of memory I get this.
With 4 on the go Run time 2 hours 23 min 24 sec
With 3 on the go Run time 1 hours 49 min 11 sec
With 2 on the go Run time 1 hours 17 min 8 sec

I would guess this implies heavy use of the swap partition.
I will soon also have the figures for running a single task.
It looks like for these to make the best use of CPU time the recommendation will be to have at least 6GB ram/core. With all four running I also noticed the computer becomes very sluggish but I do have BOINC settings that slow BOINC down minimally when computer is in use.
ID: 91357 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5128
United Kingdom
Message 91360 - Posted: 3 May 2019, 10:04:29 UTC

I'll add David Wallom's comment from this morning:

We are definitely in two camps, one with average CPU times of ~ 900s and one between 4000s - 9000s. With the extremely high memory load I would envisage the difference is caused by memory and bus type not just processors.
and move this thread to the Projects area for future reference - CPDN users may wish to set max_concurrent via app_config.xml for this task type.
ID: 91360 · Report as offensive
ProfileDave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2691
United Kingdom
Message 91361 - Posted: 3 May 2019, 12:34:00 UTC - in response to Message 91360.  

Thanks Richard.
Having tried running 1,2,3 &4 tasks concurrently I found with only 8GB of memory, the tasks took twice as long to finish when running four tasks as compared with running just the one My 20GB swap is on the SSD so those with mechanical swap may well find the slow down even worse. Worth running all four so long as I am only using the desktop for work as running four the laptop becomes very sluggish to respond.
ID: 91361 · Report as offensive

Message boards : Projects : CPDN: Tasks using a lot of memory.

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.