Message boards : Questions and problems : CRC checks?
Message board moderation
Author | Message |
---|---|
Send message Joined: 28 Jun 10 Posts: 2706 |
I know there is the option to turn off cyclic redundancy checks on image files. Are these checks done on other files that get downloaded for different task types before the actual tasks are downloaded? |
Send message Joined: 5 Oct 06 Posts: 5129 |
There are two sets of files: "project" files and "task/workunit" files.The project files are typically downloaded when you you attach to a project, and often repeated at subsequent contacts. Image files are usually sent in the first group, and are often in a lossy compressible format like JPEG, for eye candy in simple view - intermediate hosts on the internet download path can re-compress them to save bandwidth, which is what the CRC option is designed to allow. In general, task data will be sent in some format like zip, which doesn't introduce lossy changes - the unzipped file should be an exact match to the original. I saw the original posts which possibly triggered this question: I have a nagging fear that BOINC sometimes miscounts when a large download is interrupted part way through and restarted from an intermediate point, but that tends to show up as 'xxxxx expected, yyyyy received' errors - possibly only with debug logging. I'll keep my eye open. |
Send message Joined: 28 Jun 10 Posts: 2706 |
There are two sets of files: "project" files and "task/workunit" files.The project files are typically downloaded when you you attach to a project, and often repeated at subsequent contacts. Image files are usually sent in the first group, and are often in a lossy compressible format like JPEG, for eye candy in simple view - intermediate hosts on the internet download path can re-compress them to save bandwidth, which is what the CRC option is designed to allow.My brain immediately went to disk image files for VB tasks! I was clearly overthinking things. It seems to me that BOINC should ideally do a CRC check on all downloaded files and retry if the check doesn't match. I wonder if there would be any mileage in putting in a feature request to git-hub? |
Send message Joined: 5 Oct 06 Posts: 5129 |
It seems to me that BOINC should ideally do a CRC check on all downloaded files and retry if the check doesn't match.I think it already does, but I'll check the sources. |
Send message Joined: 28 Jun 10 Posts: 2706 |
Thanks, no point in asking for something already there! |
Send message Joined: 5 Oct 06 Posts: 5129 |
All my CPDN data files (where this query first arose) have MD5 checksums - either calculated by the server when the file was first loaded to form part of a workunit, or (for an upload file) by the client, when that stage of the computation has completed. Application files are checked for a more sophisticated digital signature. https://github.com/BOINC/boinc/blob/master/client/cs_files.cpp#L127 has this comment: // verify_contents // if true, validate the contents of the file based either on = // the digital signature of the file or its MD5 checksum. // Otherwise just check its existence and size.(and more) That looks good enough for now - we can dig deeper if the errors continue. |
Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License,
Version 1.2 or any later version published by the Free Software Foundation.