Collatz was running solid this time for a bit longer than it has over the past several months -- almost 10 days before a breakdown.

In the past 3 months it has 'blown up' 11 times. Worst case was over the new year when it was offline for 12 days. Average down time is around 3 days -- though that varies from one day to 12 days.

Collatz came back up several hours ago.

And Collatz went back down overnight again <sigh>

Should I rename this thread to "News on when Collatz is down"? ;-)

Perhaps we need a one key macro.

I've taken to considering an easier copy and past post.

"Collatz is down again"


"Collatz is up"


From the traffic that's over on Collatz when it is up, the underlying cause remains a mystery. So the project will continue to ping pong.

As for me, it really has resulted in my running collatz on a short leash with available alternatives increasingly becoming my primaries.

CPDN news:

Jonathan Miller, CPDN wrote:

Hi Chaps,

Following a brief outage yesterday, we are having issues with the database server, and other VMs.

The project is effectively down, and I can't yet say what is running or not.

More info as I get it.

Jonathan Miller, CPDN, 2 hours later wrote:

We appear to be back up and running.

Yesterday we had a problem with the VMs data store, which seems to have cascaded into today's problem with the database server's drives.
Both are now fixed.

We also had a problem with the networking of the database server, and we have worked around that.

This was posted Monday on the Constellation Questions and Answers forum:

"We had a server crash resulting in a DB crash. In the moment, the server is not hosted on a emergency hardware with a slower internet connection. We are working to get a new server and locate them back to the data center."

---

WuProp was down last night but is up now.

Collatz is back up.

Collatz down again.

Seti@Home is out of work, due to database trouble.

Matt Lebofsky, project scientists wrote news here:

So! All the recent headaches are due to continuing issues with the master science database. While all the data seems to be intact, there's something fundamentally wrong causing informix to keep hanging up (usually when we are continuing work on reconnecting the fragmented result tables).

During the previous recent crashes I would clean up what informix was complaining about in various error messages (always the result table indexes) but this time I'm in the process of doing a comprehensive check of everything in the database just to be sure. And, in fact, I'm seeing minor problems that I've been able to clean up thus far (once again, no loss in data - just internal bookkeeping and broken index issues).

I thought this full check would be done by now (ha ha) but it's not even close. Meanwhile we *should* be able to do Astropulse work, but the software blanking engine requires the master science database to do some integrity checks, so that is all offline as well.

There are ways to speed up such events in the future. We're working on enacting several improvements. Yes, we here are all beyond tired of our project grinding to a halt and things will change for the better.

- Matt

He also wrote earlier on the 8th of April:

Hey - just so y'all know it's the science database again. This time enough is enough and I'm doing a comprehensive set of integrity checks on everything in that database before starting it up again. So no MB splitting or assimilating. Eventually some work will show up for AP splitting in the meantime...

Might be back up by the end of the work day, if not shortly after that. I did find one problem which was obscured while checking things out after previous crashes, so there's hope.

- Matt

More from Matt at Seti:
Due to recent problems we are doing a deep cleaning of one of our larger databases. This is unfortunately progressing slower than expected, and keeping us from generating new work. We might be in this state until next week.

Whoa what happened here at the BOINC BOARDS...????? NOT Reachable for past 15-20 minutes.

Whoa what happened here at the BOINC BOARDS...????? NOT Reachable for past 15-20 minutes.

Known issue. Change from http;// to https://

---

http vs https was not the problem. Site was inaccessible.

My Chrome web browser automatically selects https when it's available via an extension called HTTPS Everywhere

Link to Chrome Store extension Encrypt the Web! Automatically use HTTPS security on many sites.

HTTPS Everywhere is a Firefox, Chrome, and Opera extension that encrypts your communications with many major websites, making your browsing more secure.

Collatz server(s) have choked & gone into "down mode" again

CPDN (Climate Prediction)

Database downtime 30 April 2015 07:00 - 12:00 GMT

What is up with the Collatz project? It has been like a Yo-Yo the last couple of weeks, up-down-up-down-up-down!!!

---

Brent, Collatz has been 'yo-yo mode' for the past few *months*.

What is going on there is that it is a one person project -- with limited resources and it has also seen quite a lot of user population growth over the past month - over 3000 new users there.

Frankly, I've been surprised that is has been as stable as it has been over the past several weeks. I expected it to revert to the serious ping pong it was going through November through March.

GPUGrid went unreachable about 3 hours ago. (8AM PDT)