API discussion

[fox]

Let me start with one thing - I have just noticed that API doesn't seem to update ccache properly. Which means using API clients might cause counter desyncs with tt-rss proper. Which is Bad(tm).

[nils.braden]

I'll just copy the start over here so we can reference it... (i also added links to github):

I think it would be good to hear some other opinions about these API-Calls so i'll just start here and make a list of the calls I suggested and why i wanted to have them :)

1. getCounters - Returns an array with all categories and their feeds with the number of unread articles inside. This is helpful if you want to quickly access all unread-counts for the categories and feeds, without reading all the data. (Link)
2. getArticles - This fetches all articles for one feed (or category, if specified) and can be limited to fetch only unread items and/or to fetch only a given number of newest articles. (Link)
3. getNewArticles - Takes a "time" and a "displayUnread" parameter. Time specifies a point in time and all articles that are updated since then are included. DisplayUnread filters out all articles are already read. I use this to get all relevant articles with their content with one GET-Request. (Link)
4. login - Change to the login-method which allows base64-encoded passwords to avoid problems with encoded special characters. It allows "unencoded" passwords too, it just tries both ways. (Link)

Well these are the methods which we discussed a lot already via email ;)

Perhaps the first thing we should do is to cut this list down to the methods which are to be discussed. I guess the modified login-method isn't much of a problem and i guess it will be #2 and #3 which are in danger of beeing cut out. What do you think?

This ccache-thing you mention, is it short for "counter-cache"? Never seen it in the api/index.html so I didn't know about it.

Greetings,
nils

[fox]

Yeah, if anything modifies the counters in any way (which means primarily changing unread status of articles) it might invalidate counter cache for affected feeds. I think only one method in the API actually changes anything, so it's not a big deal.

Modified login is already in, I don't think there's anything to discuss there, as long as it works. I'm still not all that hot on bulk downloading of stuff (well, we talked about it a lot) but I really don't have anything better which won't involve extensive rewrites of your application.

E.g. the API client workflow I had in mind was something like that (I guess it's not surprising since that's mostly how tt-rss works)

- init, etc
- request the headlines list as selected by user
- download the headlines, receive headline IDs, start fetching loaded article data in background in case user clicks on something
- rinse, repeat, view another feed

You went there another way around which somewhat better (you might have stuff preloaded which you haven't seen in the headlines) and somewhat worse (you can have stuff cached which you won't realistically click on). Am I making sense?

[fox]

One more thing, I'd like the getCounters method to use getAllCounters() which is now refactored to return JSON-able data. The data format is somewhat different from the one you chose to use, so either you'll need to slightly update your app (the changes should be minimal) or we'll need to massage data in the API handler to maintain compatibility. Personally I'd prefer doing the former for maximum data format unification in tt-rss.

[nils.braden]

The getCounters-Method is quiet short with this new getAllCounters-function:

Code: Select all

case "getCounters":
    print json_encode(getAllCounters($link));
    break;


It should be no big deal to consume the new output and i agree that this is the way to do it. One thing about the output, is it right that it returns the JSON-Content as shown below? There are duplicate IDs but i dont know why. These are 0 and -2, once with "cat=true", does that mean that there is a feed ("Archived") and a category ("Uncategorized") with id 0 like it is with getHeadlines and getFeeds? So it would mean if "cat=true" we have a category as defined in http://tt-rss.org/redmine/wiki/tt-rss/J ... e#getFeeds and if cat ist not set it is a feed, following the definition in http://tt-rss.org/redmine/wiki/tt-rss/J ... tHeadlines ?

Code: Select all

[{"id":"global-unread","counter":"134"},{"id":"subscribed-feeds","counter":"34"},{"id":0,"counter":"0","description":"Archived articles"},{"id":-1,"counter":"0","description":"Starred articles"},{"id":-2,"counter":"0","description":"Published articles"},{"id":-3,"counter":"79","description":"Fresh articles"},{"id":-4,"counter":"79","description":"All articles"},{"id":"80","updated":"14:32","counter":15,"has_img":0},{"id":"90","updated":"14:32","counter":0,"has_img":0},{"id":"97","updated":"14:33","counter":1,"has_img":0},{"id":"88","updated":"14:34","counter":0,"has_img":0}, ... {"id":"104","updated":"14:33","counter":1,"has_img":0},{"id":"105","updated":"14:34","counter":0,"has_img":0},{"id":"109","updated":"14:33","counter":0,"has_img":0},{"id":"110","updated":"14:33","counter":0,"has_img":0},{"id":-2,"cat":true,"counter":"0"},{"id":0,"cat":true,"counter":"79"}]

About the bulk-downloading: I'll fix things step by step. Perhaps it would be appropriate to leave the methods in the API for now but somehow mark them as deprecated and don't talk about them in the API-Reference. What i want to achieve is a fast user-interface where everything is there when the user needs to see it. I think this can be done with the 2-step-downloading of headlines->content too and i would like to test bulk-downloading of headlines. I could fetch all category-ids and call getHeadlines with is_cat=true for every category-id. With these two steps there would be all data there except for the content which could be fetched for every article and while one article is read the content for the rest of the feed could be fetched.

BUT. What i had in mind earlier was some kind of pre-download for when you are at home and got a fast connection. Not sure if the kind of bulk-downloading which is implemented is the right way to do it, perhaps some kind of compressed format for all data would fit better for this need. Would be great if i could compete with NewsRob/feedR which i think both allow the pre-fetch all neccessary data for the user. Later perhaps i could parse the xml for URLs and start caching images as well...

What do you think about that? Would be nice to hear some opinions about these plans :)

Greetings,
nils

[fox]

I have actually tweaked the counter functions a bit to present more logical format (cat: true -> kind: "cat"). If kind: is not defined, assume it's a feed of some sort. Anyway, since you're okay with this, I'm changing getCounters to use getAllCounters().

Also, I'm seeing a bug there: feed icon checking (e.g. has_img:) doesn't work properly, probably because api/index.php is in a different directory and PHP is too stupid to figure this sort of thing out.

Feed IDs can be a bit confusing. Feed 0 is Archived, category 0 is Uncategorized. Category -2 is Labels, Feed -2 is, um, I dunno. Published, I think. There should be a list of IDs somewhere on the wiki, likely on the API page.

We can definitely leave the API calls undocumented (which would mean unsupported as far as I'm concerned). So when better ideas surface, we'll be able to phase them out. By the way, tt-rss already has a mostly working download/sync interface for its offline mode. Although it's not public at the moment and uses the internal RPC server but this is something that we can change.

[fox]

One more thing I thought about: maybe there is some sense in changing typical API response to include request sequence id, e.g. from

Code: Select all

{"version":"1.4.3.1"}

to

Code: Select all

{"seq":"13","payload":{"version":"1.4.3.1"}}

Where seq is a HTTP request parameter supplied by caller. That would theoretically help the client to identify what exactly did it receive. Also, this would make the whole response thing more extensible, e.g. we might be able to add status codes or method names to the response in the future without changing the contents of payload and breaking API compatibility.

[nils.braden]

Hi,

1. I totally agree with the idea of adding a squence-numbers.

2. Offline-Mode:

By the way, tt-rss already has a mostly working download/sync interface for its offline mode. Although it's not public at the moment and uses the internal RPC server but this is something that we can change.

Didn't know about this, perhaps this would be appropriate for the offline-usage-idea. Can you give me some pointers on where to look for this interface in the code?

Btw. I'll be on vacation the next two weeks and got a lot to do afterwards. Hope I find the time to implement these changes soon..
Greetings,
nils

[fox]

Check the "download" method in backend-rpc and, I guess, offline.js. The whole thing is somewhat weird but it kinda works.

Edit: API sequence thing is done in trunk.

Edit2: Have fun on your vacation!

[nils.braden]

Hi there, i'm back from vacation. Was a nice trip, Tenerife has 25°C and beach, really nice :)

Back to business. I implemented the necessary changes to read from the new API and as far as i can see it hasn't changed since then. Do you have any plans about when you will publish the new version of Tiny Tiny RSS? I figured I can't make my reader support the old AND the new API at once without further trouble so I cleaned out the old methods. But this leads me to the problem that i can't release any new version until the server is updated...

Greetings,
nils

[fox]

Hi there, i'm back from vacation. Was a nice trip, Tenerife has 25°C and beach, really nice

Damn you! It's -20C here, the whole city is up its collective ass in snow and my vacation is in like six months. Spring can't come soon enough.

Do you have any plans about when you will publish the new version of Tiny Tiny RSS? I figured I can't make my reader support the old AND the new API at once without further trouble so I cleaned out the old methods. But this leads me to the problem that i can't release any new version until the server is updated...

http://tt-rss.org/redmine/projects/tt-rss/roadmap

I'd really like to see the new version, though. Maybe you could release it outside of the market or something for people who are using trunk?

[nils.braden]

Just a short notice as i spent a while debugging: There is a typo in the API-docs on catchup feeds (http://tt-rss.org/redmine/wiki/tt-rss/J ... atchupFeed), the second parameter is called "category" (not "is_cat").

Greetings,
nils

[fox]

Yeah, it should be is_cat like the rest of the methods. Also, getArticles uses is_category. Changed both to is_cat for consistency.

[nils.braden]

Hi,

Merry Christmas btw. :)

Just had a problem yesterday, someone doesn't get a session_id (it is null when automatically assigned but calling with ...sid=asdf seems to work. Should i just create a session-id on the device or do you have any idea what the problem could be? The link to the issue is here: https://code.google.com/p/ttrss-reader- ... tail?id=35

Greetings,
nils

[fox]

No idea. Session id can't really be null unless something is seriously broken with his PHP session settings. I'm not sure setting arbitrary sid is a good idea, you are supposed to get this from the server. If the server is broke, I suggest fixing it, not hiding the problem.