Springe direkt zu Inhalt

GitLab Outage

There was a small out of our GitLab instance from 15:08 to 15:40. Read on for post mortem.

News from Jul 29, 2021

Yesterday, we migrated our GitLab instance from version 13.12 on a server running Debian buster to 14.1 on a new server running Debian bullseye. Major migrations like this one take usually take a bit of time because of everything that has bindings against the C-library needs to be recompiled. 

GitLab itself consists of multiple components that are mostly written in Ruby and Go, but the Ruby parts use a lot of Gems that use C bindings. While rebuilding everything yesterday we did not fully rebuild Gitaly - the component handling the repositories -, since it's partly written in Go. Some parts, though, are written in Ruby, using some Gems with C-bindings, which we missed, since errors only triggered when showing the project overview page of a project with a license file, which we did not test in our test environment prior to upgrading.

We have now rebuilt Gitaly and everything should be working again. User data was not affected. 

20 / 60