heads up: headline list behavior changes

Development-related discussion, including bundled plugins
User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

heads up: headline list behavior changes

Postby fox » 12 Jul 2015, 14:03

sometimes when reading through feeds using infinite scrolling things went kinda weird - already read headlines appeared, maybe some were missed, etc. this took me a few years to figure out because i'm rather dumb.

see, when frontend requests another page while scrolling down, it only has one additional number to pass to backend: we have X headlines loaded, please return another batch, skipping X.

unfortunately that doesn't really work so well in all cases. to give one example: while we were happily scrolling through stuff the feed got updated in background and Y new headlines were inserted. on next page request sql server counts from the updated information in the database which frontend is not aware of, so that magical X number now points to the wrong database row and our pagination is incorrect.

the proper way to fix this would be somehow forcing the database to start counting from a specific ID at all times but i have no idea how to do that. postgresql has window functions and shit but it sounds entirely too complicated and i have no idea if mysql even has anything like that. maybe i'm missing something obvious, wouldn't be the first time.

so, i've added a hack: when the first page is sent, frontend saves the first row ID and passes it on all subsequent page requests. backend requests the first row for current view (yeah it's an additional query but it should be fast enough to not be noticeable) and checks if it changed. if it did, we notify user to reload the feed because paginating further is impossible in any reliable fashion.

anyway, main reason for this post is that i had to screw around with the way headlines are generated. many other things in tt-rss depend on it - generated rss feeds, api, etc. so if you see something not working as it should, report here.

pcause
Bear Rating Master
Bear Rating Master
Posts: 144
Joined: 23 Aug 2013, 19:52

Re: heads up: headline list behavior changes

Postby pcause » 12 Jul 2015, 18:39

I had feeds stop updating. Apparently there was a second batch of updates. Pulled them and then did a force update. Getting some update snow but see this in the log:

E_USER_ERROR (256) classes/db/pgsql.php:46 Query SELECT DISTINCT date_entered, guid, ttrss_entries.id, ttrss_entries.title, updated, score FROM ttrss_entries,ttrss_user_entries,ttrss_feeds,ttrss_labels2,ttrss_user_labels2 WHERE ttrss_user_entries.feed_id = ttrss_feeds.id AND ttrss_user_entries.ref_id = ttrss_entries.id AND ttrss_user_entries.owner_uid = '2' AND ttrss_labels2.id = ttrss_user_labels2.label_id AND ttrss_user_labels2.article_id = ref_id ORDER BY ttrss_feeds.title, score DESC, date_entered DESC, updated DESC LIMIT 1 failed: ERROR: for SELECT DISTINCT, ORDER BY expressions must appear in select list LINE 19: ... ttrss_user_labels2.article_id = ref_id ORDER BY ttrss_feed... ^

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: heads up: headline list behavior changes

Postby fox » 12 Jul 2015, 18:48

i really need to kill the group by feed thing. try pulling again.

nameless
Bear Rating Master
Bear Rating Master
Posts: 126
Joined: 28 Aug 2013, 20:33

Re: heads up: headline list behavior changes

Postby nameless » 12 Jul 2015, 22:57


User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: heads up: headline list behavior changes

Postby fox » 12 Jul 2015, 23:07


nameless
Bear Rating Master
Bear Rating Master
Posts: 126
Joined: 28 Aug 2013, 20:33

Re: heads up: headline list behavior changes

Postby nameless » 12 Jul 2015, 23:21

Thanks

nameless
Bear Rating Master
Bear Rating Master
Posts: 126
Joined: 28 Aug 2013, 20:33

Re: heads up: headline list behavior changes

Postby nameless » 13 Jul 2015, 16:39

I this just me or is performance worse now?
My server is idling most of the time but Fresh Articles seem to load significantly slower now.

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: heads up: headline list behavior changes

Postby fox » 13 Jul 2015, 17:59

i dunno, i haven't really noticed any visible difference. could be.

e: i've changed a few things, it may be faster now, or not.

nameless
Bear Rating Master
Bear Rating Master
Posts: 126
Joined: 28 Aug 2013, 20:33

Re: heads up: headline list behavior changes

Postby nameless » 13 Jul 2015, 19:57

Can anybody second my observations?
If so I might go some heads back and do some benchmarks.
I actually doubt this is one my side of things.

JustAMacUser
Bear Rating Overlord
Bear Rating Overlord
Posts: 373
Joined: 20 Aug 2013, 23:13

Re: heads up: headline list behavior changes

Postby JustAMacUser » 13 Jul 2015, 20:02

My dev environment is sluggish (and produces errors) with last night's commits, I haven't applied the few recent ones. I'll report back if the latest updates haven't resolved it.

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: heads up: headline list behavior changes

Postby fox » 13 Jul 2015, 21:14

it might be a bit slower in firefox, i dunno

well maybe if mozilla actually optimized their garbage browser instead of bolting retarded proprietary crap on top of it it'd function better

nameless
Bear Rating Master
Bear Rating Master
Posts: 126
Joined: 28 Aug 2013, 20:33

Re: heads up: headline list behavior changes

Postby nameless » 13 Jul 2015, 21:20

I am using Chromium though.

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: heads up: headline list behavior changes

Postby fox » 13 Jul 2015, 21:39

i dunno, i'm not seeing any difference in chrome.

e: in any case, this headline format change doesn't really bring anything particularly amazing to the table, it's just a bit neater code. if it's slower i'll just revert it.

at least try pulling to 57c52 first.

Maru
Bear Rating Trainee
Bear Rating Trainee
Posts: 40
Joined: 20 Oct 2013, 14:26

Re: heads up: headline list behavior changes

Postby Maru » 13 Jul 2015, 21:58

Ok I found another issue apparently cause by the recent update. When I try to remove a feed I get the following error

Exception: TypeError: reply.headlines.content.each is not a function
Function: headlines_callback2()

The feed is removed from the list though

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: heads up: headline list behavior changes

Postby fox » 13 Jul 2015, 22:05

yeah, i've just rolled it back, so we might have new and exciting bugs now. it also seems faster, a bit.


Return to “Development”

Who is online

Users browsing this forum: No registered users and 4 guests