API discussion

[nils.braden]

Hi fox,

Happy new year btw :)

I just finished implementing the offline-cache for images included in the content of articles and attachments and now need to fetch lots of articles. The caching is simply no use if you have to open all feeds by hand so the articles get cached and it is only afterwards you can start caching images for the articles.

The image-cache is supposed to make offline-browsing possible while "beeing on the road" and you can download all you need while you are on WIFI. So where do I get the articles? The current way is to call (deprecated) getNewArticles which probably only works for mysql databases (I think I saw some mysql-specific-calls there). You mentioned an RPC-API a while ago, do you think this is the way to do it? What would be the benefit of fetching articles this way in contrast to a normal API-call?

Greetings,
nils

edit: Just tried to fetch articles the normal way (fetch Headlines, then articles for this feed). Takes 2.5 minutes for 35 feeds :/

[fox]

Happy new year to you too!

Hmm... The RPC way is gone, I think I've cleaned it out of the code while removing Gears support. getNewArticles is quite messy and shares a lot of code with getArticles. Ideally, I'd like to drop both of them and replace them with some kind of less bloated unified method.

Ideally it should use api_get_headlines (since it has support for embedding article content anyway). Maybe add another parameter to output only articles newer than specific timestamp?

[nils.braden]

That sounds good to me. As this function would basically return articles i'd suggest calling it getArticles but i fyou internally use getHeadlines that's fine too. At the moment output from getHeadlines is missing URL/CommentURL, Tags and Attachments, are these supposed to be in there or do you think that should be separated? Perhaps some parameter like "get_everything" could be added for content, urls and stuff so you would normally only get Headlines but this way you'd end up with full articles.

I would backup the idea of a timestamp for fetching articles. That would be useful to reduce the amount of articles. Currently I always fetch the latest 30 articles to make sure I don't miss any new articles but normally there are only a few so there is way too much bandwith used.

Greetings,
nils

[nils.braden]

Hi,,

I stumbled accross an issue and would like to know if there is anything wrong with what I am doing there. Problem is, when calling login in a single_user_mode tt-rss instance there should be no username and password as far as I know. But login always returns API_DISABLED when called without arguments. Calling with username/password which were created while not beeing in singe_user_mode works like expected.

Don't know how to handle this in the client, should I request the setting for single_user_mode before logging in?

Greetings,
nils

[fox]

That's interesting. What if you try default username (e.g. admin)?

[nils.braden]

With op=login&user=admin the login works, i get an SID. with empty username or empty username and empty password it returns API_DISABLED too.

[fox]

I suppose it should just use admin when in single user mode.

e.g. https://github.com/gothfox/Tiny-Tiny-RS ... 6e651b26b1

[nils.braden]

Doesn't seem to work, I think the chek for single_user_mode needs to be before line 73 where the uid of the login-name is requested.

[fox]

Oh, right. Fixed in trunk.

[nils.braden]

api/index.php lines 51 to 59:

Code: Select all

if (!$_SESSION["uid"] && $op != "login" && $op != "isLoggedIn") {
   print api_wrap_reply(API_STATUS_ERR, $seq, array("error" => 'NOT_LOGGED_IN'));
   return;
}

if ($_SESSION["uid"] && $op != "logout" && !get_pref($link, 'ENABLE_API_ACCESS')) {
   print api_wrap_reply(API_STATUS_ERR, $seq, array("error" => 'API_DISABLED'));
   return;
}

One thing that doesn't work here:
Trying to login with wrong username/password doesn't give NOT_LOGGED_IN, it always gives API_DISABLED. Tried to find a simple solution to match all cases (right credentials and api disabled, wrong cred and api disabled, wrong cred and api enabled) but that seems to be complicated.


Another problem I got:
Are you planning on introducing some way of requesting only new content since some timestamp in the past? Don't know how to handle this case:
1. I read some articles in a feed
2. Some unread articles are left
3. Fetching new articles:
3.1. I fetch a number of ~30 articles to make sure I got all new ones, if there are more unread articles I fetch the number of unread, eg. 45 articles.
(Problem: Perhaps there are read articles after the 30 I fetch by default. Raising the number need more bandwith.)
3.2. I fetch all unread articles, no matter if I got them already.
(Problem: Perhaps there are read articles in between and I definitely fetch articles twice)
3.3. I dont remember way number 3 anymore but i think it was something that didnt work too... ^^

Greetings,
nils

[fox]

First one should be simple enough to fix: https://github.com/gothfox/Tiny-Tiny-RS ... 44c809d0cb

I'm still not sure bulk loading stuff is the right thing to do things. I want to click on the feed I'm interested in and read it, not wait until it finishes hogging the wonky 3g connection loading random stuff that just happens to be unread. At this moment I don't even use tt-rss reader tbh, mobile version works way faster and it doesn't do any sort of preloading at all.

BTW there are two officially unsupported methods for the ttrss-reader in the API (getArticles/getNewArticles). Do you use them?

Maybe we need to rethink either the API or the way ttrss-reader works with it, since mobile version being faster doesn't make much sense in my opinion unless I have severely overbloated returned JSON.

[nils.braden]

mobile version works way faster and it doesn't do any sort of preloading at all.

Srsly? I can't even use it properly, or is there a mobile-version I don't know of? I just visit the main-page with my android-browser (stock) and the interface is not really usable to me. Can't resize the article-window, can't scroll most of the time and so on. Strange thing, can't imagine how you are using it.

[fox]

Open the mobile/ subfolder.

[nils.braden]

Oh.

Just a thought: What about putting this information somewhere on the main-information-site? :D
I did find just one reference to the mobile-version in Compatible Browsers and there it is only mentioned without explanation on where to find it. I fear this could make my users switch over to the mobile-version but that's not the point ;)

As for getArticles/getNewArticles, i don't use them because I wrote them and I know how bad my PHP-skills where at that time. And you told me not to use them. ^^
So you can safely remove them.

As for the app beeing slow and stuff, perhaps it doesn't really compete with a JavaScript based (it is, isn't it?) Website but it has the advantage of preloading stuff if you are going to be offline for some time or can't use mobile networks for some reason. I often start the ImageCache in the morning, then switch to offline-mode and leave for the bus...

edit: Hm, when I think about it...
Actually bulk-loading of articles is the one single difference to the mobile-website. Everything else can be easily achieved with JavaScript and a browser, only caching lots of articles for later offline-browsing cannot.

[fox]

Yeah, I suppose something about tt-rss being obvious to me doesn't mean it's obvious to everyone else. Documented here: http://tt-rss.org/redmine/wiki/tt-rss/MobileVersion and mentioned on the front page. Showing the pimp screenshots of my gingerbead Desire (lol).

As for the app beeing slow and stuff, perhaps it doesn't really compete with a JavaScript based (it is, isn't it?) Website

I think it still should be faster. Receiving JSON from server just has to be faster than processing data from the database into HTML and pushing it to the web browser. Unless I screwed the pooch on the API big time.

but it has the advantage of preloading stuff if you are going to be offline for some time or can't use mobile networks for some reason. I often start the ImageCache in the morning, then switch to offline-mode and leave for the bus...

Yeah I keep forgetting about this use-case. Probably because I have a prepaid data plan. Maybe it would be a good idea to add a synchronize-for-later button/offline mode? We could fit it in the existing API call by adding something like a timestamp parameter to getHeadlines (which can include article content).

E.g. you click on sync for offline mode button, app requests articles of "All articles" virtual-feed which are unread and are newer than specified timestamp. In online mode, it just does the usual song and dance with getHeadlines as I go around reading stuff while keeping background traffic to the minimum (maybe lazy-loading of article content or the articles near to the one I'm reading now or whatever). And definitely not fetching anything not related to what I'm actually trying to read in foreground since that's what could easily give the app the appearance of being unresponsive.

BTW I would still prefer the app if possible. I like it, it's just that at this moment mobile/ works better.