You are here : Accueil Zaclys Gazette

La gazette de la mère Zaclys

Server Maintenance (June 21, 2017)

Since Wednesday June 21 morning we have serious disruptions on most of our services

Everything came in order on Friday 23 at 12:30!

All our apologies, this is the first big break since 2008 ...


Wednesday morning after a "classic" upgrade of a server and the migration of a dozen of our dedicated cloud servers, we had to reboot our main server ... which never left again ...

This is rare but it happens (the updates on linux can alter the boot loader Grub, and we also replace a disk SSD recently on this server, what we know now is the combination of 2 that Caused the error when restarting ...) ... in these cases there is no loss of data but the server does not start anymore.

We have several intervention procedures in these cases, which are prepared when we do the stress tests at the installation of the servers and well on this time none has solved the problem. Ya of days like that ...

Our sysadmin was on the move ... and we lost 1 day to request a helping hand from our provider OVH (no answer) to finally subscribe to the VIP support to accelerate ... they intervened this Thursday to make a Auditing the server without bringing real solutions (and we are still waiting for some technical answers to confirm scenarios of fast resumption for another time), but they have confirmed our tracks and made a hardware check (ram, cpu) which is a good thing. And now with vip support we have direct access by mail and such.

Thursday evening our sysadmin with the support of a 2nd sysadmin (Mat who recently joined us and who had taken advantage of the day to learn about zfs and grub), were able to take the hand remotely on the server (thanks ovh who Gives us very handy tools for that). We were able to fix the "boot loader" (grub zfs) and restart the server and the disks with all the data. But we still have to fix a configuration bp on partition numbers that blocks a clean start of the machine.

The 2 sysadmin have to work together tomorrow morning until 12h30 to finish this and take advantage to present to Mat the infrastructure ... Thanks to Mat for the help, we were not too much to think about the problem!

In order to answer the questions we receive, we will invest in a service with our sysadmin (which gives us volunteer help on its Free time but has a job in real life) so that he can take on his "professional" time in case of need. And we're going to expand the team with Mat who recently joined us.

Our apologies for this breakdown is the first one that has been so long since the beginning of the adventure, but we have accumulated bad luck on this one ... we also learned a lot

Thank you for your confidence and see you soon

 

PS: The incident tracking table is available at the bottom of the page
https://www.zaclys.com/travaux-et-incidents,travaux

 

 



3 569 clicks - Created the 22/06/2017 by Tito - Modified 17/07/2017



React, comment, discuss ...

Info34 reactions.

  • by Bill (Zaclynaute) - 26/06/2017 21:11

    @XavierV
    Ca fait dans les 30Mb environ pour l'album. Ca dépend déjà de votre vitesse d'upload, vous savez elle est de combien ?

  • by XavierV (Zaclynaute) - 26/06/2017 19:22

    C'est bon maintenant pour l'album, mais c'est bizarre que ça prenne plusieurs heures, non ?

  • by XavierV (Zaclynaute) - 26/06/2017 15:09

    J'ai créé un album ce matin. J'y ai mis 200 photos de 100 à 200 Ko. Et quand je vais voir l'album, ça m'affiche toujours "en cours" ? Que puis je faire?

   [...] pour voir les réactions précédentes cliquer ici »



(sert uniquement à vous signaler les réactions)






Sharing?

diaspora  G+  facebook  twitter  Digg  Yahoo  Delicious  Technorati  myspace




A lire également :



retour


+ récents




Tout voir …



Sitemap | Help | Legal Notice | RGPD | Works and incidents | Budget | Make a donation | Chouchen | Z pub | Logo et charte

© Association Mother Zaclys 1998-2018  - Zaclys v2.5

follow us sur Diaspora, Mastodon, Google+, FaceBook ou Twitter

Translations Google Translate