Rate limiting for some git-related requests

[fox]

I've noticed an unusual amount of git-related HTTP requests in nginx logs which cause a noticable increase in overall traffic sent per month (each request is only ~16K but with the rate of 1 per 3-4 seconds per IP it stacks up quite quickly - going by webalizer this specific URL endpoint caused 88% of monthly hits and 36% overall bytes sent in April, however this could be legit git traffic, at least partially):

Code: Select all

(ip redacted) - - [09/Apr/2017:08:46:00 +0300] "GET /gitlab/fox/tt-rss.git/info/refs?service=git-upload-pack HTTP/1.1" 200 13638 "-" "git/1.9.1"

Only three IP addresses originate the *vast* majority of the above traffic, one of those registered to Digital Ocean. Maybe they NAT all outbound traffic for their hosted sites under one address? Maybe some special person out there decided to git pull every second? Who knows, really.

Anyway, for the time being I'm implementing a rate limit if the following two conditions match: request URI contains "service=git-upload-pack" and user agent contains "git".

If you're screwed by this and can't git pull anymore because Digital Ocean or whoever *is* putting everyone behind one colossal NAT please post here so I can rework or remove this limit.

[cpforbes]

I've got a server running at Digital Ocean and far as I know it there is no NAT involved.

My server better not be attempting to pull every second, but if (ip redacted) ends in 228 let me know.

[fox]

nah. nevertheless, since April 9:

Code: Select all

  41521 x.x.30.57
  49819 x.x.82.78
  90014 x.x.178.187
 104441 x.x.214.130
 121943 x.x.84.0
 122200 x.x.24.105


bottom to top first two IPs are Digital Ocean, the 3rd one is "LeaseWeb Netherlands", the 4th one is some hosting provider in France (online.net).

gg guys

[JustAMacUser]

Sounds like an automated script gone bad. You're a lot nicer than I would be; I'd just block the IPs at the firewall and be done with it... Of course then you'd get a new forum member complaining that they can access Git locally but not from their VPS.

As mentioned, I don't think Digital Ocean does anything but VPSs with dedicated IPs.

[sleeper_service]

Sounds like an automated script gone bad. You're a lot nicer than I would be; I'd just block the IPs at the firewall and be done with it... Of course then you'd get a new forum member complaining that they can access Git locally but not from their VPS.

and then you'd know who the troublemaker is

[JustAMacUser]

and then you'd know who the troublemaker is

Win-win for everyone!

[fox]

hehe

[joeyteel]

Digital Ocean definitely doesn't do outbound NAT or half my sites wouldn't function correctly. I can also confirm that those aren't either of my instances.

If they're hitting you that often I'd report it personally. Digital Ocean takes abuse reports pretty seriously.

[fox]

whatever it was it stopped now so it's all cool *shrug emoji*

maybe the mysterious owner reads this forum, lol

[fox]