Stickied Incidents

27th November 2019

Dwellers Dwellers need to be reprovisioned. Affects Mastodon.

Still down: Mastodon, RabbitMQ, Notifications center, tools.nasqueron.org

2020-02-15 Development facilities like Docker build are ready to use

2020-02-14 A fresh CentOS 8.1 has been reinstalled through Salt.

2020-02-13 Upgrade RAM, new 1 To disk attached to avoid previous Mastodon issues.

2019-11-27 According to the servers log: 07:13 dereckson: Disk drives look corrupted. Attach a new 1 To drive to the machine. Install a clean CentOS 7 on this drive to recover data.

Past Incidents

15th July 2017

No incidents reported

14th July 2017

No incidents reported

13th July 2017

No incidents reported

12th July 2017

No incidents reported

11th July 2017

No incidents reported

10th July 2017

No incidents reported

9th July 2017

No incidents reported

8th July 2017

Mastodon Space disk issues on Mastodon redis cache

Space disk issue in the partition used by the Mastodon redis cache. We're watching this.

2017-07-11

We're again in too low space disk condition. Filled https://devcentral.nasqueron.org/T1226 and adding a new 200G disk.

18:53 UTC — New disk has been added, formatted and mounted. Next step is to move data there.
19:03 UTC — Mastodon instance has been shutted down to move data
19:07 UTC — Data is moving to new partition
19:49 UTC — Data is still moving (21 Gb transferred)
22:30 UTC — Mastodon instance has been restarted

2017-07-08

Solved cleaning up not used files on the partition