The word "priority" is used in many different places, and with many different meanings, according to context - in both computing in general, and BOINC in particular.

Please give us an example or two of the context for this particular usage.

So now we have:
Extreme 22 hours to go on 17 days deadline
Extreme "50" days to go on 21 days deadline (* will miss deadline)
Four of WCG 15 minutes to go each on 3 days deadline

* is the problem.

It's the only one of the tasks you mention that - on the basis of the information that BOINC has been given by PrimeGrid - is at risk of missing its deadline. The most important policy directive that BOINC has in this situation is "avoid missing deadlines, at all costs".

So the 50 day estimate task * gets first stab at the machine's resources, and only other tasks which can fit around it will be run.

Before a task has started running, BOINC is only given two pieces of information about it: the number of floating point operations that will be needed to complete the task (estimated by the project staff in advance), and the speed of the device it will be running on (estimated by BOINC from the observed speed of previous tasks run by the same application on the same device). Nothing else.

PrimeGrid could possibly make a better attempt to get those estimates right ...

Someone above my pay grade will have to explain "why?": I just try to explain the "what?"

In the world of the BOINC client scheduler, “earliest deadline” doesn’t mean “the deadline occurring first in the calendar”; it means “the task most likely to miss its deadline”.

In your case, then (as Richard has already explained), “earliest deadline first” policy (which BoincTasks labels “high priority”) means the “50-⁠day” tasks get to run, because they are projected to miss their deadline. Everything else is lower priority.

The prioritisation algorithm is:

1. favor jobs in danger of deadline miss
   
2. favor coproc jobs, so that e.g. if we're RAM-limited we'll use the GPU instead of the CPU
   
3. favor jobs in the middle of time slice, or that haven't checkpointed since start of time slice
   
4. for CPU jobs, favor jobs that use more CPUs
   
5. favor jobs selected first by schedule_cpus() (e.g., because their project has high sched priority)
   

If you speak C++, you can see the implementation here.

Especially when you then don't do things you could have

But it will do them. (Or at least it should…) Under the existing policy, and all else remaining equal: at some point during the next 3 days, the WCG tasks will become the ones most likely to miss their deadline (even with the others still in progress), and they will be given priority such that they complete in time.

until some point where WCG is also behind

With a 1-⁠day work buffer, that shouldn’t happen. The client will realise WCG needs prioritising before it gets behind, not once it gets behind – so (assuming the remaining-time estimates aren’t wildly wrong) the tasks will finish by their deadline.

You said earlier "the WCG tasks will become the ones most likely to miss their deadline"

“Most likely” is perhaps inaccurate. At every scheduling point, BOINC predicts whether each task will miss its deadline. There is no relative grading of probability; it’s 0 or 100%. Right now, your PrimeGrid tasks get priority because under all conceivable scheduling choices, they will miss their deadline. In that situation, the client’s policy is to give them the chance to run – it allows for the remaining-time and achieved-performance estimates to be wrong (and perhaps to improve over time), and it is not permitted to abort the tasks simply because it believes it has no chance of finishing them. The WCG tasks do not get priority yet, because there is still plenty of time before the deadline to do the remaining work; the client does not need to favour them over the PrimeGrid tasks now, because it can reschedule later.

WCG with 15 minutes to go on a 2 minute deadline

It should never get that close to the wire. With a work buffer of 1 day, the scheduler applies 1 day of padding to task deadlines (because it could run for that long before next contacting a server). So 1 day before the WCG tasks are due, the scheduler will predict that all tasks will miss their deadline, at which point they get prioritised in deadline order (the intuitive interpretation of “earliest deadline first”).

I've seen it fail many times.

If you can get a concrete example of that, please capture the client state, feed it in to the Client Emulator, and raise a bug.

I'm sure WCG would appreciate their tasks getting done sooner

If a project wants its tasks returned sooner, it needs to set earlier deadlines. BOINC is not a race; it does not care whether tasks finish 1 second or 1 week before their deadline.

It would be better to do stuff you know you can get done in time first.

Doing that each time, BOINC will never do work for the other projects, when you always have a project with earlier deadlines than others. Just leave things well alone and you'll notice that the next time BOINC asks for work from Primegrid, is going to be quite a distance time, so as to give the other project(s) time to play catch up.

And besides, BOINC projects use redundancy. Work is not just sent out to you, but to another computer as well. When one doesn't return the work in time, it's sent out to a 3rd computer. Until a canonical result comes back.

You asked earlier as to why BOINC only has one scheduler for all project's applications. First off, the scheduler comes from the time that computers had just a CPU and most projects had 1 application. Only later have new hardware options been added, like GPUs and multiple applications per project. Seeing how the development of BOINC was then and still is done by volunteers and rewriting the scheduler from scratch to include all the new things people want is quite a job, it's been put on the back burner. Maybe one day.

The way I see it is that it is Prime Grid's responsibility to either get better estimates or to increase the deadlines. Pretty sure there are other projects out there that mess things up as well, like CPDN, the main one I am involved in having deadlines of over a year for work that PhD students need back a lot quicker for their theses. Much tighter deadlines on some recent batches have worked well.

The other thing I would do if it were such a big issue for me is use my account over at git-hub to put in a request for what you want. That would still need a developer to pick it up and run with it but if they don't get the requests proving a demand, there isn't a chance of it being prioritised.

add

Incidentally, I think Amicable numbers is guilty of overestimating the time tasks will take, certainly on their multi-CPU tasks where it looks to me like the estimate is for if the task is running on just on processor rather than the six I am using.