Page 1 of 1

Feed parsing error since update

Posted: 27 Jan 2017, 12:28
by ivanwin
Hi,

I've had this feed working for months and after an update it doesn't work anymore.
Here's the myfeedsucks' report:

Processing feed: http://mangapedia.fr/rss.xml

Fetching URL (UA: Tiny Tiny RSS/17.1 (http://tt-rss.org/))

Operation failed. Diagnostic output below.
Error message: file_get_contents(http://mangapedia.fr/rss.xml): failed to open stream: HTTP request failed! HTTP/1.1 503 Service Temporarily Unavailable
Code: 503
Used curl:
Content type: text/html; charset=UTF-8

Of course using a browser directly to the url http://mangapedia.fr/rss.xml is working and Firefox sees it as a valid feed file...
Does someone have an idea?

Re: Feed parsing error since update

Posted: 27 Jan 2017, 12:44
by fox
one idea would be reading both subforum rules and maybe even the error message

7 days probation

>Of course using a browser directly to the url http://mangapedia.fr/rss.xml is working and Firefox sees it as a valid feed file...

imagine if you will a world where computers with different network addresses, CDNs, and all sorts of other wacky shit exists

Re: Feed parsing error since update

Posted: 27 Jan 2017, 21:11
by sleeper_service

Code: Select all

$ curl -i http://mangapedia.fr/rss.xml
HTTP/1.1 503 Service Temporarily Unavailable
Date: Fri, 27 Jan 2017 18:10:16 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: close
Set-Cookie: __cfduid=d8522a51bb6db4b761e8e1112255335db1485540616; expires=Sat, 27-Jan-18 18:10:16 GMT; path=/; domain=.mangapedia.fr; HttpOnly
X-Frame-Options: SAMEORIGIN
Refresh: 8;URL=/cdn-cgi/l/chk_jschl?pass=1485540620.513-ZI/Mg9NZfu
Cache-Control: no-cache
Server: cloudflare-nginx
CF-RAY: 327e291533061177-DFW

<!DOCTYPE HTML>
<html lang="en-US">
<head>
  <meta charset="UTF-8" />
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
  <meta http-equiv="X-UA-Compatible" content="IE=Edge,chrome=1" />
  <meta name="robots" content="noindex, nofollow" />
  <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1" />
  <title>Just a moment...</title>
  <style type="text/css">
    html, body {width: 100%; height: 100%; margin: 0; padding: 0;}
    body {background-color: #ffffff; font-family: Helvetica, Arial, sans-serif; font-size: 100%;}
    h1 {font-size: 1.5em; color: #404040; text-align: center;}
    p {font-size: 1em; color: #404040; text-align: center; margin: 10px 0 0 0;}
    #spinner {margin: 0 auto 30px auto; display: block;}
    .attribution {margin-top: 20px;}
    @-webkit-keyframes bubbles { 33%: { -webkit-transform: translateY(10px); transform: translateY(10px); } 66% { -webkit-transform: translateY(-10px); transform: translateY(-10px); } 100% { -webkit-transform: translateY(0); transform: translateY(0); } }
    @keyframes bubbles { 33%: { -webkit-transform: translateY(10px); transform: translateY(10px); } 66% { -webkit-transform: translateY(-10px); transform: translateY(-10px); } 100% { -webkit-transform: translateY(0); transform: translateY(0); } }
    .bubbles { background-color: #404040; width:15px; height: 15px; margin:2px; border-radius:100%; -webkit-animation:bubbles 0.6s 0.07s infinite ease-in-out; animation:bubbles 0.6s 0.07s infinite ease-in-out; -webkit-animation-fill-mode:both; animation-fill-mode:both; display:inline-block; }
  </style>

    <script type="text/javascript">
  //<![CDATA[
  (function(){
    var a = function() {try{return !!window.addEventListener} catch(e) {return !1} },
    b = function(b, c) {a() ? document.addEventListener("DOMContentLoaded", b, c) : document.attachEvent("onreadystatechange", b)};
    b(function(){
      var a = document.getElementById('cf-content');a.style.display = 'block';
      setTimeout(function(){
        var s,t,o,p,b,r,e,a,k,i,n,g,f, CqEkpcH={"MooAMIwr":+((!+[]+!![]+!![]+!![]+[])+(!+[]+!![]+!![]+!![]+!![]+!![]))};
        t = document.createElement('div');
        t.innerHTML="<a href='/'>x</a>";
        t = t.firstChild.href;r = t.match(/https?:\/\//)[0];
        t = t.substr(r.length); t = t.substr(0,t.length-1);
        a = document.getElementById('jschl-answer');
        f = document.getElementById('challenge-form');
        ;CqEkpcH.MooAMIwr-=+((!+[]+!![]+[])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+!![]));CqEkpcH.MooAMIwr-=+((!+[]+!![]+[])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]));CqEkpcH.MooAMIwr-=+((!+[]+!![]+!![]+!![]+[])+(+[]));CqEkpcH.MooAMIwr*=!+[]+!![]+!![]+!![];CqEkpcH.MooAMIwr+=!+[]+!![]+!![]+!![]+!![]+!![];CqEkpcH.MooAMIwr+=+((!+[]+!![]+[])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]));CqEkpcH.MooAMIwr*=+((!+[]+!![]+[])+(+[]));CqEkpcH.MooAMIwr+=+((+!![]+[])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]));a.value = parseInt(CqEkpcH.MooAMIwr, 10) + t.length; '; 121'
        f.submit();
      }, 4000);
    }, false);
  })();
  //]]>
</script>


</head>
<body>
  <table width="100%" height="100%" cellpadding="20">
    <tr>
      <td align="center" valign="middle">
          <div class="cf-browser-verification cf-im-under-attack">
  <noscript><h1 data-translate="turn_on_js" style="color:#bd2426;">Please turn JavaScript on and reload the page.</h1></noscript>
  <div id="cf-content" style="display:none">
    <div>
      <div class="bubbles"></div>
      <div class="bubbles"></div>
      <div class="bubbles"></div>
    </div>
    <h1><span data-translate="checking_browser">Checking your browser before accessing</span> mangapedia.fr.</h1>
    <p data-translate="process_is_automatic">This process is automatic. Your browser will redirect to your requested content shortly.</p>
    <p data-translate="allow_5_secs">Please allow up to 5 seconds&hellip;</p>
  </div>
  <form id="challenge-form" action="/cdn-cgi/l/chk_jschl" method="get">
    <input type="hidden" name="jschl_vc" value="a3a86a21ff527df947a003a6466136f4"/>
    <input type="hidden" name="pass" value="1485540620.513-ZI/Mg9NZfu"/>
    <input type="hidden" id="jschl-answer" name="jschl_answer"/>
  </form>
</div>


          <div class="attribution">
            <a href="https://www.cloudflare.com/5xx-error-landing?utm_source=iuam" target="_blank" style="font-size: 12px;">DDoS protection by Cloudflare</a>
            <br>
            Ray ID: 327e291533061177
          </div>
      </td>
    </tr>
  </table>
</body>
</html>


yeah, I'd say that feed sucks.

Re: Feed parsing error since update

Posted: 27 Jan 2017, 21:51
by fox
putting rss feed behind cloudflare, smart

Re: Feed parsing error since update

Posted: 28 Jan 2017, 00:35
by ibreakcellphones
I had this same sort of trouble with another feed. Only thing I could do was to ask the site owner to move the RSS feeds outside of cloudflare.

Re: Feed parsing error since update

Posted: 28 Jan 2017, 12:12
by fox
i kind of lowkey want to implement some kind of detection for this retarded shit just to shame idiots who use cloudflare for rss feeds

also the way they abuse http 503 status code makes me sad

Image

should probably read something like "cloudlflare, contact the idiot who runs this site"

Re: Feed parsing error since update

Posted: 29 Jan 2017, 15:53
by Skibbi
Few month ago I had similar issue with RSS behind cloudflare. The solution was very simple - change user agent to something popular. But maybe now they have more sophisticated protection.

Re: Feed parsing error since update

Posted: 04 Feb 2017, 18:40
by Nicola
Skibbi wrote:Few month ago I had similar issue with RSS behind cloudflare. The solution was very simple - change user agent to something popular. But maybe now they have more sophisticated protection.



As you change the user agent? Would you be kind enough to help me? Thank you.

Re: Feed parsing error since update

Posted: 04 Feb 2017, 20:21
by Skibbi
Unfortunately the issue I had was related to perl script, and I just set the user agent inside the script.
But what you can do is to use curl to download problematic RSS:

Code: Select all

curl -A "UserAgentString" http://url.com > /var/www/your_domain/rss

Then point tt-rss to your local domain as a source for RSS.

Re: Feed parsing error since update

Posted: 04 Feb 2017, 21:20
by JustAMacUser
Skibbi wrote:But what you can do is to use curl to download problematic RSS [...] Then point tt-rss to your local domain as a source for RSS.


... or just define SELF_USER_AGENT in config.php and TT-RSS will use that.

Re: Feed parsing error since update

Posted: 04 Feb 2017, 21:46
by fox
please don't ask for support after doing that tho

Re: Feed parsing error since update

Posted: 04 Feb 2017, 22:39
by Nicola
fox wrote:please don't ask for support after doing that tho


Thanks Skibbi JustAMacUser and fox for your answers. Ask fox if there is another way, preferably supported?

Re: Feed parsing error since update

Posted: 04 Feb 2017, 23:01
by JustAMacUser
fox has a good point. Changing the user agent for every request (which is what will happen if you set that constant I suggested) could be a bad thing. If you wanted it per feed, you'd have to write a plugin that hooks HOOK_FETCH_FEED. Check if it's the feed URL in question; if so, call fetch_file_contents() just as TT-RSS would normally in rssfuncs.php but with a specific useragent key. Then return the results.

Re: Feed parsing error since update

Posted: 04 Feb 2017, 23:24
by fox
i think its best to contact the site owner and tell them that putting an rss feed - a thing normally accessed by automated agents - behind a literal captcha is unimaginably retarded

Re: Feed parsing error since update

Posted: 06 Feb 2017, 18:04
by ivanwin
Hello,
Thanks for all your responses, thanks to you i understand what is wrong.
I've ask the website admin to do something about it. Hope it will work again.

Sorry @fox if I didn't respected the rules.

see you and thank again