Magpie overcaching problem

Development-related discussion, including bundled plugins
User avatar
gmargo
Bear Rating Disaster
Bear Rating Disaster
Posts: 62
Joined: 08 Jan 2012, 22:33
Location: Silicon Valley

Magpie overcaching problem

Postby gmargo » 04 Apr 2012, 22:21

While testing the recent "Manual Update" problems, I set one of my feeds to a "weekly" update. It was updated afterwards as expected.
But now I've switched the update interval back to "default", which is 30 minutes.
And I've noticed that I'm not getting any updates on that feed, even if I click on the Update button.

A manual "wget" fetch of the feed URL yields what I expect, but that is not reflected in the database.

I think it's a Magpie cache problem.
Magpie was told to cache this feed for one week.
And so it refuses to fetch a new copy.

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Magpie overcaching problem

Postby fox » 04 Apr 2012, 22:53

Whatever is set up in tt-rss has nothing to do with magpie caching. It has some internal logic to determine updated stuff of which I have no idea. You can disable the cache if you have problems with it.

User avatar
gmargo
Bear Rating Disaster
Bear Rating Disaster
Posts: 62
Joined: 08 Jan 2012, 22:33
Location: Silicon Valley

Re: Magpie overcaching problem

Postby gmargo » 04 Apr 2012, 23:22

There is tt-rss code dealing with magpie caching. See include/rssfuncs.php approx line 351:

Code: Select all

            } else if ($update_method == 1) {

                define('MAGPIE_CACHE_AGE', get_feed_update_interval($link, $feed) * 60);
                define('MAGPIE_CACHE_ON', !$no_cache);
                define('MAGPIE_FETCH_TIME_OUT', 60);
                define('MAGPIE_CACHE_DIR', CACHE_DIR . "/magpie");

                $rss = @fetch_rss($fetch_url);

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Magpie overcaching problem

Postby fox » 04 Apr 2012, 23:35

Ah, hoisted on my own petard. Maybe add a check there if last_updated is null, don't cache? Also, probably, same goes for Simplepie (?).

User avatar
gmargo
Bear Rating Disaster
Bear Rating Disaster
Posts: 62
Joined: 08 Jan 2012, 22:33
Location: Silicon Valley

Re: Magpie overcaching problem

Postby gmargo » 06 Apr 2012, 22:03

I think I was wrong about the source of the problem I was observing.
It was not an "overcaching" problem. Instead a feed was being incorrectly parsed into nothingness, and hence it seemed to be not updated. I think it was a coincidence that the particular feed I used for testing was the problematic one. I haven't a fix for the parsing problem yet.

However I do have this fix, as you suggested:
If a "manual update" occurs, then ignore any cached item.

Update: problem with patch, wait.....
Update: problem fixed. Can't set cache time to zero, since Magpie interprets that as a default 1 hour.
Attachments
Ignore_cache_if_manual_update_v2.patch.gz
Ignore cache if manual update.
(600 Bytes) Downloaded 135 times

snow922
Bear Rating Trainee
Bear Rating Trainee
Posts: 1
Joined: 07 Apr 2012, 09:01
Contact:

Re: Magpie overcaching problem

Postby snow922 » 07 Apr 2012, 09:23

fox wrote:Whatever is set up in tt-rss has nothing to do with magpie caching. It has some internal logic to determine updated stuff of which I have no idea. You can disable the cache if you have problems with it.

yes, you can stop the cache. :roll:

User avatar
gmargo
Bear Rating Disaster
Bear Rating Disaster
Posts: 62
Joined: 08 Jan 2012, 22:33
Location: Silicon Valley

Re: Magpie overcaching problem

Postby gmargo » 08 Apr 2012, 21:01

I discovered my magpie feed parsing problem was of my own creation. I was attempting to filter some bad characters from the feed content, and that apparently screwed up several of my feeds.

However, the above patch for manual update is still valid.

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Magpie overcaching problem

Postby fox » 09 Apr 2012, 11:58

Thanks, merged into trunk.


Return to “Development”

Who is online

Users browsing this forum: No registered users and 2 guests