Message boards : Questions and problems : Effect of settings on multi-core tasks.
Message board moderation
Author | Message |
---|---|
Send message Joined: 28 Jun 10 Posts: 2706 |
I have BOINC set to download 0.1days worth of work and up to 0 days worth of additional work. On LHC I am mostly getting multi-core tasks that run on 8 cores. (I have number of cores set to 50% max, i.e. half the 16 cores BOINC sees. It would seem that BOINC can not see that the tasks it is downloading are going to use 8 cores each as the total work downloaded amounts to 0.5 of a day. |
Send message Joined: 5 Oct 06 Posts: 5129 |
Already reported at https://github.com/BOINC/boinc/issues/4151 (Jan 2021). This one should be trivial to fix, if anyone can be bothered. |
Send message Joined: 28 Jun 10 Posts: 2706 |
Already reported at https://github.com/BOINC/boinc/issues/4151 (Jan 2021).Thanks Richard, guess I should have looked there before posting. If more projects start doing multi-core work, then there may be some motivation to fix it. Main reason for raising it was the forthcoming multi-core stuff on CPDN as once it has been through testing and moves to the main site, there are sure to be people noticing it. |
Send message Joined: 5 Oct 06 Posts: 5129 |
Looks like we have a taker: https://github.com/BOINC/boinc/pull/4992. Now, how can we persuade MilkyWay or PrimeGrid to load it for testing? |
Send message Joined: 28 Jun 10 Posts: 2706 |
Looks like we have a taker: https://github.com/BOINC/boinc/pull/4992. Or LHC? |
Send message Joined: 5 Oct 06 Posts: 5129 |
Or LHC?I was going to say I hadn't seen any MT apps there, but I see they've now got a long_native_mt plan task. No work currently, but I'll ask Laurence. |
Send message Joined: 28 Jun 10 Posts: 2706 |
I seem to get nearly all mt tasks from them. Interestingly I did an update with one task still running and unlike when I am out of work which leads to 8 being downloaded, I only got one.Or LHC?I was going to say I hadn't seen any MT apps there, but I see they've now got a long_native_mt plan task. No work currently, but I'll ask Laurence. |
Send message Joined: 5 Oct 06 Posts: 5129 |
OK, I've written to Laurence, and attached my Linux boxes. I'm trying to stay native - don't want to add VBox as well - and they may prefer to use their dev site. We'll see what happens. |
Send message Joined: 5 Oct 06 Posts: 5129 |
And I've snagged three. 01/11/2022 11:41:36 | LHC@home | Requesting new tasks for CPU 01/11/2022 11:41:36 | LHC@home | [sched_op] CPU work request: 16034.67 seconds; 0.26 devices 01/11/2022 11:41:37 | LHC@home | [sched_op] estimated total CPU task duration: 23975 secondsThey are designated to run on all six cores, and estimated to run for 2 hours 13 minutes - I make that a total of 143,784 core-seconds. |
Send message Joined: 28 Jun 10 Posts: 2706 |
They are designated to run on all six cores, and estimated to run for 2 hours 13 minutes - I make that a total of 143,784 core-seconds.The last Atlas task I ran was 34920core seconds running on 8 cores 1 hour 12 minutes 45 seconds. |
Send message Joined: 5 Oct 06 Posts: 5129 |
I've been trying - and failing - to run these tasks, but I need cvmfs and it's not installing on Mint 21. I'll take a break, and then try installing it on the older and more stable Mint 20.3 |
Send message Joined: 5 Oct 06 Posts: 5129 |
Mint 20.3 seems to take the cvmfs installation. Part 2 tomorrow. |
Send message Joined: 28 Jun 10 Posts: 2706 |
I've been trying - and failing - to run these tasks, but I need cvmfs and it's not installing on Mint 21. I'll take a break, and then try installing it on the older and more stable Mint 20.3I have only been running Atlas using VB rather than natively. That took a long time till I tracked down the issue(s) causing problems. |
Send message Joined: 2 Feb 22 Posts: 84 |
I've been trying - and failing - to run these tasks, but I need cvmfs and it's not installing on Mint 21. ... Did you allow user namespaces? Required settings may vary a bit depending on the kernel version and the policy of your Linux vendor. Did you install and enable the automounter service? Did you install the recent cvmfs-config package beside the main package? Use the format you prefer: http://ecsft.cern.ch/dist/cvmfs/cvmfs-config/cvmfs-config-default-latest.noarch.rpm http://ecsft.cern.ch/dist/cvmfs/cvmfs-config/cvmfs-config-default_latest_all.deb Did you run (once) "cvmfs_config setup"? Beside other steps it copies "cvmfs.auto" to the right location and configures the automounter to use it. This needs to be checked. Some systems now require a simple "sudo systemctl restart autofs" to work fine. Others require a reboot. Few systems require 2 reboots. If a repository list is given in /etc/cvmfs/default.local the probes should now work. e.g. CVMFS_REPOSITORIES=cvmfs-config,atlas Output of "cvmfs_config probe" should look like this: Probing /cvmfs/cvmfs-config.cern.ch... OK Probing /cvmfs/atlas.cern.ch... OK |
Send message Joined: 5 Oct 06 Posts: 5129 |
Thanks for looking into this. I can post the error messages, if it would help? Where would you like them? My second machine (Linux Mint 20.3, very similar hardware) accepted the cvmfs quickstart settings without complaint, and is currently an hour into its first ATLAS native task. It's recorded 4 hours 39 minutes of CPU time in that hour (6 cores, nothing else running, but slow to start the MT phase), so I think this one's OK. |
Send message Joined: 5 Oct 06 Posts: 5129 |
And having said that, it bailed out after a further 20 minutes. Uploaded and validated - task 368415294. Looks OK to me. |
Send message Joined: 2 Feb 22 Posts: 84 |
ATLAS tasks from LHC@home run through an initialization/configuration phase. That phase is always singlecore even if BOINC (or vbox) treats it as n-core. Depending on the "power", the total load and the network/internet load of the client computer it can take 3..25 min until ATLAS has set up everything that is required to start n (posix-)threads (called worker threads) that concurrently do the event processing. Each task processes 200 events and each event can take between a few seconds and >20 min to be processed. Once a worker has finished an event it immediately picks the next one from the event queue until that queue is empty. At the end there's another singlecore phase that combines the results from all workers and reports a (small) logfile plus a (large) result file back to BOINC. If you have trouble with CVMFS, it might be better to post questions and error messages at the LHC@home forum. Your example task succeeded which can be seen close to the end of stderr.txt: [2022-11-02 09:33:35] HITS file was successfully produced: If this line is shown and the HITS file has a reasonable size (here: 160 MB) errors/warnings mentioned before can be ignored. They are not BOINC related. |
Send message Joined: 5 Oct 06 Posts: 5129 |
Thanks again. So, we can take it that cvmfs runs OK on Mint 20.3 But it doesn't on Mint 21. I get: richard@Rosa:~$ sudo apt-get install cvmfs [sudo] password for richard: Reading package lists... Done Building dependency tree... Done Reading state information... Done cvmfs is already the newest version (2.9.4~1+ubuntu20.04). The following packages were automatically installed and are no longer required: libflashrom1 libftdi1-2 Use 'sudo apt autoremove' to remove them. 0 to upgrade, 0 to newly install, 0 to remove and 5 not to upgrade. richard@Rosa:~$ sudo cvmfs_config setup Error: failed to load cvmfs library, tried: './libcvmfs_fuse3_stub.so' '/usr/lib/libcvmfs_fuse3_stub.so' '/usr/lib64/libcvmfs_fuse3_stub.so' './libcvmfs_fuse_stub.so' '/usr/lib/libcvmfs_fuse_stub.so' '/usr/lib64/libcvmfs_fuse_stub.so' ./libcvmfs_fuse3_stub.so: cannot open shared object file: No such file or directory libcrypto.so.1.1: cannot open shared object file: No such file or directory /usr/lib64/libcvmfs_fuse3_stub.so: cannot open shared object file: No such file or directory ./libcvmfs_fuse_stub.so: cannot open shared object file: No such file or directory libcrypto.so.1.1: cannot open shared object file: No such file or directory /usr/lib64/libcvmfs_fuse_stub.so: cannot open shared object file: No such file or directory Failed to read CernVM-FS configurationCould you pass that back to the LHC team, please? All those fuse files seem to be present in /usr/lib: |
Send message Joined: 2 Feb 22 Posts: 84 |
Looks like Mint 21 installs newer libraries than those the ubuntu CVMFS package is compiled against. I suspect nobody at CERN or LHC@home will solve those library issues and compile a package for a distribution not used there. If a precompiled packet doesn't work the recommended way would be to get the CVMFS source from github and compile the client on the target computer. See: https://github.com/cvmfs/cvmfs Since it's just for a test it may be too much work to do so, hence the test may be done only on Mint 20.3. |
Send message Joined: 5 Oct 06 Posts: 5129 |
Agreed. But since Mint is derived from Ubuntu, it might be worth keeping an eye on the next Ubuntu release, in case action is needed there, |
Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License,
Version 1.2 or any later version published by the Free Software Foundation.