Thread 'Download lots of files or zip into one?'

Message boards : Questions and problems : Download lots of files or zip into one?
Message board moderation

To post messages, you must log in.

AuthorMessage
ProfileDave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2706
United Kingdom
Message 109692 - Posted: 26 Aug 2022, 13:29:06 UTC

In discussions on the WCG board there are posts to do with issues with bandwidth and files needing several attempts to download them. Seeing that there are a lot of files for each task, what are the pros and cons of zipping them into a much lower number of larger files as opposed to having lots of smaller files.

Not looking for solutions, just a bit more understanding.
ID: 109692 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5129
United Kingdom
Message 109693 - Posted: 26 Aug 2022, 14:35:22 UTC - in response to Message 109692.  

We had that type of problem at GPUGrid. Project administrators tend to be more comfortable with Linux than Windows, so they sent out an app with a .gz archive, and tried to unpack it with tar. Apparently Windows 10 has a native tar app, but Windows 7 doesn't. I suggested 7-zip, and they got that working - not as versatile as tar, but it worked.

My first copy of WinZip came on a paid-for floppy disk, but Windows added native zip, somewhere down the line. And then took it out again from Windows 11.

Standards are tricky things - there are too many of them, and you have to support them all.
ID: 109693 · Report as offensive
ProfileDave
Help desk expert

Send message
Joined: 28 Jun 10
Posts: 2706
United Kingdom
Message 109694 - Posted: 26 Aug 2022, 16:28:55 UTC - in response to Message 109693.  

Standards are tricky things - there are too many of them, and you have to support them all.
So, maybe less of a problem for projects that send different task types to different operating systems. (Is CPDN the only project that does this?)
ID: 109694 · Report as offensive
Richard Haselgrove
Volunteer tester
Help desk expert

Send message
Joined: 5 Oct 06
Posts: 5129
United Kingdom
Message 109697 - Posted: 26 Aug 2022, 17:58:00 UTC - in response to Message 109694.  

Most projects try to send all their apps to as many platforms as possible, but come can't manage them all. I think a specialist policy for a complicated project like CPDN would be fair enough.
ID: 109697 · Report as offensive

Message boards : Questions and problems : Download lots of files or zip into one?

Copyright © 2024 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.