Gitlab Droplet on Digital Ocean – Add a robots.txt

If you’re using Digital Ocean’s Gitlab droplet, it’ll give you a fully configured instance for you to start using straight away. The most often case for people using Gitlab is for private repositories, if that’s the case for you then it’s worth creating a file at:

/opt/gitlab/embedded/service/gitlab-rails/public/robots.txt

and then putting a disallow rule in such as:

User-Agent: *
Disallow: /

Although not a massive security feature it should stop your public facing stuff from showing up on search engines.