Page 1 of 1

Magpie overcaching problem

Posted: 04 Apr 2012, 22:21
by gmargo
While testing the recent "Manual Update" problems, I set one of my feeds to a "weekly" update. It was updated afterwards as expected.
But now I've switched the update interval back to "default", which is 30 minutes.
And I've noticed that I'm not getting any updates on that feed, even if I click on the Update button.

A manual "wget" fetch of the feed URL yields what I expect, but that is not reflected in the database.

I think it's a Magpie cache problem.
Magpie was told to cache this feed for one week.
And so it refuses to fetch a new copy.

Re: Magpie overcaching problem

Posted: 04 Apr 2012, 22:53
by fox
Whatever is set up in tt-rss has nothing to do with magpie caching. It has some internal logic to determine updated stuff of which I have no idea. You can disable the cache if you have problems with it.

Re: Magpie overcaching problem

Posted: 04 Apr 2012, 23:22
by gmargo
There is tt-rss code dealing with magpie caching. See include/rssfuncs.php approx line 351:

Code: Select all

            } else if ($update_method == 1) {

                define('MAGPIE_CACHE_AGE', get_feed_update_interval($link, $feed) * 60);
                define('MAGPIE_CACHE_ON', !$no_cache);
                define('MAGPIE_FETCH_TIME_OUT', 60);
                define('MAGPIE_CACHE_DIR', CACHE_DIR . "/magpie");

                $rss = @fetch_rss($fetch_url);

Re: Magpie overcaching problem

Posted: 04 Apr 2012, 23:35
by fox
Ah, hoisted on my own petard. Maybe add a check there if last_updated is null, don't cache? Also, probably, same goes for Simplepie (?).

Re: Magpie overcaching problem

Posted: 06 Apr 2012, 22:03
by gmargo
I think I was wrong about the source of the problem I was observing.
It was not an "overcaching" problem. Instead a feed was being incorrectly parsed into nothingness, and hence it seemed to be not updated. I think it was a coincidence that the particular feed I used for testing was the problematic one. I haven't a fix for the parsing problem yet.

However I do have this fix, as you suggested:
If a "manual update" occurs, then ignore any cached item.

Update: problem with patch, wait.....
Update: problem fixed. Can't set cache time to zero, since Magpie interprets that as a default 1 hour.

Re: Magpie overcaching problem

Posted: 07 Apr 2012, 09:23
by snow922
fox wrote:Whatever is set up in tt-rss has nothing to do with magpie caching. It has some internal logic to determine updated stuff of which I have no idea. You can disable the cache if you have problems with it.

yes, you can stop the cache. :roll:

Re: Magpie overcaching problem

Posted: 08 Apr 2012, 21:01
by gmargo
I discovered my magpie feed parsing problem was of my own creation. I was attempting to filter some bad characters from the feed content, and that apparently screwed up several of my feeds.

However, the above patch for manual update is still valid.

Re: Magpie overcaching problem

Posted: 09 Apr 2012, 11:58
by fox
Thanks, merged into trunk.