On 27 February, between 2 a.m. and 8 a.m. CET, the RDAP, EPP and Registrar Panel services of .vlaanderen were unavailable. The cause was an automatic database failover, after which our authentication service was unable to connect to this database. This had an impact on the aforementioned services. Such failovers occur occasionally, but this time it had a greater impact than usual. Our monitoring systems detected the problem immediately, but it was not until around 7:40 a.m. CET. that we actively noticed the problem. After restarting the affected authentication service, all services were operational again.
We have since improved an alert notification issue, which was the result of an incorrect configuration in one of our monitoring tools.
We have also attempted to reproduce this behaviour in our test environment, but have been unsuccessful. We simulated various database failovers, but our systems handled them well and we were unable to reproduce a similar interruption. However, we are still looking into this.
We also plan to make our setup more robust, where we will try to detect similar situations more quickly and intervene automatically.