Some systems are experiencing issues

Stickied Incidents

27th November 2019

Dwellers Dwellers need to be reprovisioned. Affects Mastodon.

Still down: Mastodon, RabbitMQ, Notifications center, tools.nasqueron.org

2020-02-15 Development facilities like Docker build are ready to use

2020-02-14 A fresh CentOS 8.1 has been reinstalled through Salt.

2020-02-13 Upgrade RAM, new 1 To disk attached to avoid previous Mastodon issues.

2019-11-27 According to the servers log: 07:13 dereckson: Disk drives look corrupted. Attach a new 1 To drive to the machine. Install a clean CentOS 7 on this drive to recover data.

Past Incidents

8th July 2017

Mastodon Space disk issues on Mastodon redis cache

Space disk issue in the partition used by the Mastodon redis cache. We're watching this.

2017-07-11

We're again in too low space disk condition. Filled https://devcentral.nasqueron.org/T1226 and adding a new 200G disk.

  • 18:53 UTC ­— New disk has been added, formatted and mounted. Next step is to move data there.
  • 19:03 UTC ­— Mastodon instance has been shutted down to move data
  • 19:07 UTC ­— Data is moving to new partition
  • 19:49 UTC ­— Data is still moving (21 Gb transferred)
  • 22:30 UTC — Mastodon instance has been restarted

2017-07-08

Solved cleaning up not used files on the partition

7th July 2017

No incidents reported

6th July 2017

No incidents reported

5th July 2017

No incidents reported

4th July 2017

No incidents reported

3rd July 2017

Mastodon Restart Mastodon instance

Fralambert and deadsuperhero reported performance issues with Mastodon web UI to load the feeds. There is nothing in the logs to document a known problem. We restart the Mastodon components to see if the behavior improves.

2nd July 2017

No incidents reported