[solved] Massive problem - some feeds aren't updated

Support requests, bug reports, etc. go here. Dedicated servers / VDS hosting only
feader
Bear Rating Master
Bear Rating Master
Posts: 160
Joined: 26 Dec 2012, 20:03

[solved] Massive problem - some feeds aren't updated

Postby feader » 16 Apr 2013, 18:45

I'm currently living on the trunk version. After yesterday's pull, some feeds are not updated any more, but when I check them with curl on the CLI, they have new entries. I tried update.php --force-update and update.php --indexes, that doesn't help (I did this some time ago that is longer that the default update interval).
All feeds use the global update interval, which is finite, and fgrep UPDATE config.php gives me define('SIMPLE_UPDATE_MODE', false); as it should. I run the update daemon with --quiet, and output to a log file, but it indicates no trouble. When running without the --quiet, its output looked normal, and it mentioned some of the feeds that aren't updated as scheduled.
Feeds that are updated include this forum ones, and a feed I had manipulated with af_feedmod. Feeds that aren't updated include this, which wasn't in af_feedmods scope. I misconfigured af_feedmod yesterday (gave it a ends-with to some xpath), and it was mentioned in the daemon's log, but that shouldn't cause such a large scale failure?

I had some plugins installed, like af_feedmod, and videoframes, but deactivated them. I've no idea what to do. :?

P.S. Clicking twice on some feed that isn't updated produces this icon
alert.png
alert.png (340 Bytes) Viewed 7414 times
in the upper right corner.
Last edited by feader on 27 Apr 2013, 17:31, edited 1 time in total.

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Massive problem - some feeds aren't updated

Postby fox » 16 Apr 2013, 18:50

Post apache error logs when the triangle happens.

Edit: also, try disabling all plugins.

feader
Bear Rating Master
Bear Rating Master
Posts: 160
Joined: 26 Dec 2012, 20:03

Re: Massive problem - some feeds aren't updated

Postby feader » 16 Apr 2013, 20:15

Yeah, I need to fix the #@*$ apache logs on my machine. However, I got one thing that is related to tt RSS
[Tue Apr 16 18:07:57 2013] [error] [client myIP] File does not exist: /my/path/to/tt-rss/null

myIP and /my/path/to/tt-rss/ are the right ones, replaced them here. Got this message while logging into ttRSS and selecting a folder wherein a "faulty" feed is stored, but when the triangle is showing up, nothing gets logged. Sorry if it indicates something that is absolutely obvious, but again I have no idea…

P.S. disabled all plug-ins save notes (couldn't in the gui, may have to edit config.php).
Last edited by feader on 16 Apr 2013, 20:21, edited 1 time in total.

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Massive problem - some feeds aren't updated

Postby fox » 16 Apr 2013, 20:20

Try latest trunk with built-in error logging.

feader
Bear Rating Master
Bear Rating Master
Posts: 160
Joined: 26 Dec 2012, 20:03

Re: Massive problem - some feeds aren't updated

Postby feader » 16 Apr 2013, 22:29

Pulled from git, and restarted the daemon with

Code: Select all

/usr/bin/php /var/services/web/tt-rss/update.php --daemon --log /var/log/ttrss.log >> /var/log/ttrss.log 2>&1

(I guess this is the error logging you mentioned?) Got the standard log thing, with the exception of

Code: Select all

[16:35:17/21791] Logging to /var/log/ttrss.log

which is from the new log system, I guess. Only odd lines in the log are these

Code: Select all

[16:44:27/22352] Base feed: http://feeds.arstechnica.com/arstechnica/apple?format=xml
[16:44:27/22352]  => 1970-01-01 00:00:00, 28

[17:09:17/23331] Base feed: http://feeds.arstechnica.com/arstechnica/apple?format=xml
[17:09:17/23331]  => 1970-01-01 00:00:00, 28

[18:08:57/25687] Base feed: http://feeds.arstechnica.com/arstechnica/apple?format=xml
[18:08:57/25687]  => 1970-01-01 00:00:00, 28

and their kind. All the feeds that still work have a date in the 2013 range, but the others have the one set by --force-update. :(

Edit: OK, actually looked in the prefs: Have error messages like
E_ERROR (1) lib/floIcon.php:791 Allowed memory size of 134217728 bytes exhausted (tried to allocate 1745223641 bytes) admin 18:36

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Massive problem - some feeds aren't updated

Postby fox » 16 Apr 2013, 23:05

floicon is apparently really fucking great piece of coding.

Edit: can you post the feed url it chokes on favicon of?

feader
Bear Rating Master
Bear Rating Master
Posts: 160
Joined: 26 Dec 2012, 20:03

Re: Massive problem - some feeds aren't updated

Postby feader » 16 Apr 2013, 23:30

fox wrote:Edit: can you post the feed url it chokes on favicon of?

I'm not exactly sure what you meant, but I tried to attach 28.ico, but am told by the board here that
Image file type 17 for mimetype image/vnd.microsoft.icon not supported.


Possible short term solution for my troubles: Delete the icons? Oh, and feed-icons is only 364K large, so no need for 1.5GB surely?

Additionally, I found some Apache logs that may be helpful, or not:
[Tue Apr 16 18:23:19 2013] [error] [client IP] File does not exist: /var/services/web/favicon.ico, referer: https://tt-rss-URL/public.php?op=dbupdate

[Tue Apr 16 18:31:26 2013] [error] [client IP] File does not exist: /var/services/web/rules.abe
[Tue Apr 16 18:31:41 2013] [error] [client IP] File does not exist: /var/services/web/tt-rss/null

(The first is from after the latest git pull, when I had to update the db)

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Massive problem - some feeds aren't updated

Postby fox » 16 Apr 2013, 23:50

Post the problematic feed URL, I'll get the icon using that.

Sent from my Transformer Prime TF201 using Tapatalk HD

feader
Bear Rating Master
Bear Rating Master
Posts: 160
Joined: 26 Dec 2012, 20:03

Re: Massive problem - some feeds aren't updated

Postby feader » 17 Apr 2013, 00:06


User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Massive problem - some feeds aren't updated

Postby fox » 17 Apr 2013, 00:20

does latest trunk still try to allocate too much ram?

feader
Bear Rating Master
Bear Rating Master
Posts: 160
Joined: 26 Dec 2012, 20:03

Re: Massive problem - some feeds aren't updated

Postby feader » 17 Apr 2013, 00:43

fresh pull, one feed that wasn't updated now is, but now the error log gets overflowed:

Code: Select all

E_ERROR (1)   lib/floIcon.php:798   Allowed memory size of 134217728 bytes exhausted (tried to allocate 632553472 bytes)   admin   22:37
E_WARNING (2)   lib/floIcon.php:795   fread(): Length parameter must be greater than 0   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Empty string or invalid image   admin   22:37
E_WARNING (2)   lib/floIcon.php:795   fread(): Length parameter must be greater than 0   admin   22:37
E_WARNING (2)   lib/floIcon.php:792   unpack(): Type L: not enough input, need 4, have 0   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Data is not in a recognized format   admin   22:37
E_WARNING (2)   lib/floIcon.php:795   fread(): Length parameter must be greater than 0   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Empty string or invalid image   admin   22:37
E_WARNING (2)   lib/floIcon.php:795   fread(): Length parameter must be greater than 0   admin   22:37
[…]
E_WARNING (2)   lib/floIcon.php:792   unpack(): Type L: not enough input, need 4, have 0   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Empty string or invalid image   admin   22:37
E_WARNING (2)   lib/floIcon.php:795   fread(): Length parameter must be greater than 0   admin   22:37
E_WARNING (2)   lib/floIcon.php:792   unpack(): Type L: not enough input, need 4, have 0   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Couldn't create GD Image Stream out of Data   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Passed data is not in 'WBMP' format   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): gd warning: one parameter to a memory allocation multiplication is negative or zero, failing operation gracefully   admin   22:37
E_WARNING (2)   lib/floIcon.php:795   fread(): Length parameter must be greater than 0   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Empty string or invalid image   admin   22:37
E_WARNING (2)   lib/floIcon.php:795   fread(): Length parameter must be greater than 0   admin   22:37
E_WARNING (2)   lib/floIcon.php:792   unpack(): Type L: not enough input, need 4, have 0   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Couldn't create GD Image Stream out of Data   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Passed data is not in 'WBMP' format   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): gd warning: one parameter to a memory allocation multiplication is negative or zero, failing operation gracefully   admin   22:37
[…]
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Empty string or invalid image   admin   22:37
E_WARNING (2)   lib/floIcon.php:792   unpack(): Type L: not enough input, need 4, have 0   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Couldn't create GD Image Stream out of Data   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Passed data is not in 'WBMP' format   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): gd warning: one parameter to a memory allocation multiplication is negative or zero, failing operation gracefully   admin   22:37
E_WARNING (2)   lib/floIcon.php:795   fread(): Length parameter must be greater than 0   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Empty string or invalid image   admin   22:37
E_WARNING (2)   lib/floIcon.php:795   fread(): Length parameter must be greater than 0   admin   22:37
E_WARNING (2)   lib/floIcon.php:792   unpack(): Type L: not enough input, need 4, have 0   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Couldn't create GD Image Stream out of Data   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Passed data is not in 'WBMP' format   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): gd warning: one parameter to a memory allocation multiplication is negative or zero, failing operation gracefully   admin   22:37
E_WARNING (2)   lib/floIcon.php:795   fread(): Length parameter must be greater than 0   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Empty string or invalid image   admin   22:37
E_WARNING (2)   lib/floIcon.php:795   fread(): Length parameter must be greater than 0   admin   22:37
E_WARNING (2)   lib/floIcon.php:792   unpack(): Type L: not enough input, need 4, have 0   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Couldn't create GD Image Stream out of Data   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Passed data is not in 'WBMP' format   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): gd warning: one parameter to a memory allocation multiplication is negative or zero, failing operation gracefully   admin   22:37
E_WARNING (2)   lib/floIcon.php:795   fread(): Length parameter must be greater than 0   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Empty string or invalid image   admin   22:37
E_WARNING (2)   lib/floIcon.php:792   unpack(): Type L: not enough input, need 4, have 0   admin   22:37
E_WARNING (2)   lib/floIcon.php:798   imagecreatefromstring(): Couldn't create GD Image Stream out of Data   admin   22:37

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Massive problem - some feeds aren't updated

Postby fox » 17 Apr 2013, 01:03

same feed url? post the problematic feed icon somewhere too, i dunno dropbox or something

in general try to provide context and data

feader
Bear Rating Master
Bear Rating Master
Posts: 160
Joined: 26 Dec 2012, 20:03

Re: Massive problem - some feeds aren't updated

Postby feader » 17 Apr 2013, 01:25

Sadly, I don't know at which URLs it exactly barks, I posted what was in the error log. I looked at the daemon output, and this is a bit strange:

Code: Select all

[20:59:42/11175] Base feed: http://www.heise.de/newsticker/heise-atom.xml
[20:59:42/11175]  => 1970-01-01 00:00:00, 6
[20:59:43/9834] Sleeping for 60 seconds...
[…]
[21:01:43/11192] Removed 0 files.
[21:01:43/11192] Removing old error log entries...
[21:01:43/11192] Base feed: http://feeds.arstechnica.com/arstechnica/apple?format=xml
[21:01:43/11192]  => 2013-04-16 20:40:03, 28

The heise feed (http://www.heise.de/newsticker/heise-atom.xml) still doesn't show up (with red triangle), and doesn't have a fresh timestamp (this is one instance, there are others earlier and later with the 1970 timestamp). The ars technica feed (http://feeds.arstechnica.com/arstechnic ... format=xml) gets timestamped, but also doesn't show new entries (I can see them with curl, update: Red triangle showing now). edit: nc just showed up again.

Some other feeds showed up in the meantime. I think I mail you the icons if needs be. Thank you for your support.

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Massive problem - some feeds aren't updated

Postby fox » 17 Apr 2013, 08:28

If you are subscribed to other arstechnica feeds, the items won't show up because they'll get caught in the duplicate filter, unless you have enabled duplicate articles.

I have subscribed to heise.de and the icon processed just fine, items showing up and everything. Here's the favicon I got for heise: https://madoka.volgo-balt.ru/tt-rss/feed-icons/7253.ico what if you compare it to the one your tt-rss downloaded?

In order to do anything about it I need to replicate the issue first, and I can't do that so far.

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Massive problem - some feeds aren't updated

Postby fox » 17 Apr 2013, 09:20

I just ran the following in two tt-rss directories and it never crashed on any icon. Strange.

Code: Select all

<html>
<style type="text/css">
img { max-width : 120px; height : auto; }
</style>
<body>
<?php
   error_reporting(E_ALL & ~E_NOTICE & ~E_DEPRECATED);

   set_include_path(dirname(__FILE__) ."/include" . PATH_SEPARATOR .
      get_include_path());

   $icons = glob("feed-icons/*.ico");

   include "colors.php";

   print "<table>";
   foreach ($icons as $icon) {
      print "<tr><td><img src='$icon'></td>";

      $imgs = @getimagesize($icon);

      print "<td>" . $imgs['mime'] . "</td>";

      $color = calculate_avg_color($icon);

      print "<td style='background-color : $color'>" . $color . "</td>";

      print "</tr>";
   }
   print "</table>";
?>
</body>
</html>


Please try it. Post the icon it chokes on.


Return to “Support”

Who is online

Users browsing this forum: No registered users and 25 guests