Stickied Incidents

27th November 2019

Dwellers Dwellers need to be reprovisioned. Affects Mastodon.

Still down: Mastodon, RabbitMQ, Notifications center, tools.nasqueron.org

2020-02-15 Development facilities like Docker build are ready to use

2020-02-14 A fresh CentOS 8.1 has been reinstalled through Salt.

2020-02-13 Upgrade RAM, new 1 To disk attached to avoid previous Mastodon issues.

2019-11-27 According to the servers log: 07:13 dereckson: Disk drives look corrupted. Attach a new 1 To drive to the machine. Install a clean CentOS 7 on this drive to recover data.

Past Incidents

11th February 2016

Ysul Ysul doesn't respond

During T714 resolution, the network isn't responsive anymore.

Task to track this issue: T716

Update 22:01 UTC: Ysul is now responsive and in working state. 23:00 -!- Wearg [surfboard@ysul.nasqueron.org] has joined #wolfplex

Dwellers Some Dwellers services have been restarted

Following an operational issue, Docker and LXC services on Dwellers have been restarted.

We're watching the situation to see everything is OK.

Etherpad and mail server are still restarting.

Updates

Update 3:00 UTC: mail server looks good

Update 3:05 UTC: Etherpad looks good

Update 3:08 UTC: The ops test suite prod-environment-behaves-correctly pass

10th February 2016

No incidents reported

9th February 2016

No incidents reported

8th February 2016

No incidents reported

7th February 2016

No incidents reported

6th February 2016

No incidents reported

5th February 2016

No incidents reported