Page 1 of 1

Cloudflare DDoS protection

Posted: 09 Aug 2016, 20:05
by ibreakcellphones
Hey, all.

I'm trying to work with http://www.theamericanconservative.com/dreher/feed/. They had a post explode on them, and so started using Cloudflare's DDoS protection to attempt to throttle what they were responding to.

I've emailed them to ask them to turn off the DDoS protection for feeds, but I was wondering if there was a way to attack this problem from both ends. I've attached a transcription of retrieving the feed, taken through a browser. The first two steps are identical to what I've traced from the updater.

1. Request the feed.
2. Receive Cloudflare's DDoS protection JavaScript stuff with a 503 Service Temporarily Unavailable, setting cookie __cfduid.
(Since, as far as I can tell, TTRSS doesn't run JavaScript, TTRSS stops here)
3. Browser runs JavaScript, and comes up with the answer. Sends it to the server with __cfduid cookie.
4. Server responds with a 302 moved temporarily, pointing to the original feed URL, setting a 90 minute cookie called cf_clearance.
5. Browser requests original feed URL, sending the two cookies.
6. Server responds with a 200 OK and the feed contents.

Since it involves running JavaScript and so on, I have my doubts about whether it's fixable, so I'll save you some trouble.

Image

Re: Cloudflare DDoS protection

Posted: 09 Aug 2016, 20:58
by fox
you can make a plugin which hooks on HOOK_FEED_FETCHED and does something with this although i'm not sure how would you plug the browser there (maybe there's a javascript library? idk)

all things considered though in this situation i think i would just unsubscribe.

Re: Cloudflare DDoS protection

Posted: 09 Aug 2016, 21:53
by fox
what if you run this feed through feedburner and subscribe to that?

Re: Cloudflare DDoS protection

Posted: 10 Aug 2016, 23:07
by ibreakcellphones
Feedburner didn't work, but it appears that my letter to the publisher did! I don't know if they're hooking off the User-Agent or just removing the DDOS from feeds period, but it works.

Thanks for your time, fox. Great product, &c.

Re: Cloudflare DDoS protection

Posted: 11 Aug 2016, 00:32
by arjuan
FYI - looks like they're still using cloudflare to cache the feed :

$ curl -I http://www.theamericanconservative.com/dreher/feed/
HTTP/1.1 200 OK
Date: Wed, 10 Aug 2016 21:30:29 GMT
Content-Type: text/html
Connection: keep-alive
Set-Cookie: __cfduid=d56dfe47acaeacf24e63cc2db9742f68f1470864628; expires=Thu, 10-Aug-17 21:30:28 GMT; path=/; domain=.theamericanconservative.com; HttpOnly
X-Powered-By: PHP/5.5.9-1ubuntu4.6
Set-Cookie: modal-interstitial=1; expires=Thu, 11-Aug-2016 21:30:29 GMT; Max-Age=86400
X-Pingback: http://www.theamericanconservative.com/xmlrpc.php
Last-Modified: Wed, 10 Aug 2016 20:09:01 GMT
Server: cloudflare-nginx
CF-RAY: 2d068c99ca715122-SJC

Re: Cloudflare DDoS protection

Posted: 11 Aug 2016, 17:42
by ibreakcellphones
Caching I can live with. It's when they require whatever's getting the feed to run some JavaScript and then post a result to "prove you're not a robot" that breaks retrieval that I want to break out a LART.