Cloudflare DDoS protection

Support requests, bug reports, etc. go here. Dedicated servers / VDS hosting only
ibreakcellphones
Bear Rating Trainee
Bear Rating Trainee
Posts: 31
Joined: 28 Mar 2013, 09:49

Cloudflare DDoS protection

Postby ibreakcellphones » 09 Aug 2016, 20:05

Hey, all.

I'm trying to work with http://www.theamericanconservative.com/dreher/feed/. They had a post explode on them, and so started using Cloudflare's DDoS protection to attempt to throttle what they were responding to.

I've emailed them to ask them to turn off the DDoS protection for feeds, but I was wondering if there was a way to attack this problem from both ends. I've attached a transcription of retrieving the feed, taken through a browser. The first two steps are identical to what I've traced from the updater.

1. Request the feed.
2. Receive Cloudflare's DDoS protection JavaScript stuff with a 503 Service Temporarily Unavailable, setting cookie __cfduid.
(Since, as far as I can tell, TTRSS doesn't run JavaScript, TTRSS stops here)
3. Browser runs JavaScript, and comes up with the answer. Sends it to the server with __cfduid cookie.
4. Server responds with a 302 moved temporarily, pointing to the original feed URL, setting a 90 minute cookie called cf_clearance.
5. Browser requests original feed URL, sending the two cookies.
6. Server responds with a 200 OK and the feed contents.

Since it involves running JavaScript and so on, I have my doubts about whether it's fixable, so I'll save you some trouble.

Image
Attachments
ttrssUpdaterTranscript.txt
ttrss updater sessions with same server
(4.82 KiB) Downloaded 190 times
dreherFeedExchange.txt
HTTP sessions of retrieving the feed from Chrome/Fiddler
(146.69 KiB) Downloaded 182 times

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Cloudflare DDoS protection

Postby fox » 09 Aug 2016, 20:58

you can make a plugin which hooks on HOOK_FEED_FETCHED and does something with this although i'm not sure how would you plug the browser there (maybe there's a javascript library? idk)

all things considered though in this situation i think i would just unsubscribe.

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Cloudflare DDoS protection

Postby fox » 09 Aug 2016, 21:53

what if you run this feed through feedburner and subscribe to that?

ibreakcellphones
Bear Rating Trainee
Bear Rating Trainee
Posts: 31
Joined: 28 Mar 2013, 09:49

Re: Cloudflare DDoS protection

Postby ibreakcellphones » 10 Aug 2016, 23:07

Feedburner didn't work, but it appears that my letter to the publisher did! I don't know if they're hooking off the User-Agent or just removing the DDOS from feeds period, but it works.

Thanks for your time, fox. Great product, &c.

arjuan
Bear Rating Trainee
Bear Rating Trainee
Posts: 2
Joined: 11 Aug 2016, 00:29

Re: Cloudflare DDoS protection

Postby arjuan » 11 Aug 2016, 00:32

FYI - looks like they're still using cloudflare to cache the feed :

$ curl -I http://www.theamericanconservative.com/dreher/feed/
HTTP/1.1 200 OK
Date: Wed, 10 Aug 2016 21:30:29 GMT
Content-Type: text/html
Connection: keep-alive
Set-Cookie: __cfduid=d56dfe47acaeacf24e63cc2db9742f68f1470864628; expires=Thu, 10-Aug-17 21:30:28 GMT; path=/; domain=.theamericanconservative.com; HttpOnly
X-Powered-By: PHP/5.5.9-1ubuntu4.6
Set-Cookie: modal-interstitial=1; expires=Thu, 11-Aug-2016 21:30:29 GMT; Max-Age=86400
X-Pingback: http://www.theamericanconservative.com/xmlrpc.php
Last-Modified: Wed, 10 Aug 2016 20:09:01 GMT
Server: cloudflare-nginx
CF-RAY: 2d068c99ca715122-SJC

ibreakcellphones
Bear Rating Trainee
Bear Rating Trainee
Posts: 31
Joined: 28 Mar 2013, 09:49

Re: Cloudflare DDoS protection

Postby ibreakcellphones » 11 Aug 2016, 17:42

Caching I can live with. It's when they require whatever's getting the feed to run some JavaScript and then post a result to "prove you're not a robot" that breaks retrieval that I want to break out a LART.


Return to “Support”

Who is online

Users browsing this forum: No registered users and 13 guests