Thread 'CPDN MULTI-CORE TASKS'

Message boards : Projects : CPDN MULTI-CORE TASKS
Message board moderation

To post messages, you must log in.

AuthorMessage
ProfileDave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2991
United Kingdom
Message 116736 - Posted: 28 Aug 2025, 4:40:45 UTC

For those interested in how their systems work with a high memory demand, there are a bit over a thousand task going on CPDN now. The cruncher needs to opt in to running these and they require 26GB of RAM to run. The most recent clients will not attempt to run them unless that amount of RAM is free.
ID: 116736 · Report as offensive     Reply Quote
Grant (SSSF)

Send message
Joined: 7 Dec 24
Posts: 117
Message 116737 - Posted: 28 Aug 2025, 5:20:13 UTC

That's 26GB of RAM per Task?
Grant
Darwin NT.
ID: 116737 · Report as offensive     Reply Quote
robsmith
Volunteer tester
Help desk expert

Send message
Joined: 25 May 09
Posts: 1365
United Kingdom
Message 116738 - Posted: 28 Aug 2025, 5:36:22 UTC - in response to Message 116737.  

Yes, per task!!!!!!!!!
ID: 116738 · Report as offensive     Reply Quote
Grant (SSSF)

Send message
Joined: 7 Dec 24
Posts: 117
Message 116739 - Posted: 28 Aug 2025, 7:00:11 UTC

Went to the site for a looksee, that huge RAM requirement isn't all-
LINUX only.
4 cores per Task.
4.5GB for a checkpoint file!
Looks like it's a 22MB result file?
4-6 days runtime.

So your disk & Internet connection are going to get a bit of a workout as well.
Grant
Darwin NT.
ID: 116739 · Report as offensive     Reply Quote
ProfileDave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2991
United Kingdom
Message 116741 - Posted: 28 Aug 2025, 8:37:10 UTC - in response to Message 116739.  

Linux only. Though some are running the tasks using VB or WSL/
90x400MB uploads means on my bored band there will be quite a delay between the task finishing and reporting once all the uploads have gone through.
4-6 days runtime.
I have one just over 12 hours in and it is running a tad over 28%/day. That one is in VB. Once the other tasks finish on the host machine, I am hopeful tasks there will finish in just under 3 days.

One other thing I should have mentioned, no one will get any of these tasks unless they opt in under project settings on the website. Glenn did look at what percentage of machines are capable of running these tasks but I can't remember what he found without looking for the post.
ID: 116741 · Report as offensive     Reply Quote
mmonnin

Send message
Joined: 1 Jul 16
Posts: 159
United States
Message 116742 - Posted: 28 Aug 2025, 22:48:45 UTC

Server status shows they as all gone.

I haven't seen them use 25GB yet, usually somewhere in the teens of GB. It does vary a lot. Each BOINCTask interval update it can vary a couple GB up and down so the memory transfer systems are getting a workout too. I don't recall any kind of BOINC app vary so much in memory usage across the run time. 7950x 1d3hr clock time, 3d5 CPU time 18% completed.
ID: 116742 · Report as offensive     Reply Quote
ProfileDave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2991
United Kingdom
Message 116743 - Posted: 29 Aug 2025, 4:26:50 UTC

Server status shows they as all gone.

Not akll gone but paused while Glenn discusses with the project whether to reduce file upload size.
ID: 116743 · Report as offensive     Reply Quote
ProfileDave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2991
United Kingdom
Message 116744 - Posted: 29 Aug 2025, 9:00:13 UTC - in response to Message 116742.  

I don't recall any kind of BOINC app vary so much in memory usage across the run time.
The single core OIFS tasks did this as well, with memory useage varying between about 1GB and 8 GB. It means that is was possible to run more at once than in theory was possible so long as they didn't peak at the same time. These tasks use code from ECMWF that originally ran on super computers so compared with that the peak memory requirements are very low! These ones are at a higher resolution than previous batches which is why they produce such massive amounts of data and require so much memory.
ID: 116744 · Report as offensive     Reply Quote
ProfileDave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2991
United Kingdom
Message 116801 - Posted: 5 Sep 2025, 11:37:09 UTC

New batch of these has gone out. 2,000 tasks. About 15% less total output but it still produces data a lot faster than my bored band connection can send it back to Oxford. There will be in the region of 39 uploads of 600-700MB each. Another consideration if you are short on disk space should Oxford's interweb go down and the uploads back up.

Hopefully, most machines with enough RAM to run these will also have the disk space!
ID: 116801 · Report as offensive     Reply Quote

Message boards : Projects : CPDN MULTI-CORE TASKS

Copyright © 2025 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.