1 2 3 4 5 6 7 8 9 10
User-agent: * Allow: /*/blob/master/ Allow: /*/tree/master/ Allow: /*/raw/master/ Disallow: /*/tree/ Disallow: /*/blob/ Disallow: /*/raw/ Disallow: /*/blame/ Disallow: /*/commit/ Disallow: /*/tarball/