Page 1 of 2

Plugin to generate .epub file

Posted: 16 May 2015, 15:16
by elenzar
Hi there,

I am new to this forum, even if I have been using this nice software for something like a year now.

I have been adapting an existing plugin (the import_export one) to generate an .epub file, in order to sync this file to my e-ink reader (Kobo) and read my feeds in a restful way for my eyes. So far the resulting plugin works to generate the file when clicking on the button in the prefs panel in TTRSS. However, I would like it to run every day automatically, so that I don't have to log in and generate the file manually. To do this I have written a small php script to run, but I find that it should first connect to my account to generate the .epub file. Hence I have a few questions:
* do you think there is a better way (more integrated to TTRSS for instance) to run this task daily than with a cronjob?
* I saw there are authentication plugins, however I don't really know how to use these in the php script. I guess that authentication is what prevent my use of the 'backend.php' file at the moment. Hence I would be grateful for hints or examples about how to solve this problem.

If more info about the plugin I wrote is needed, here is the code:

Any remark or suggestion will be gratefully received.

Best regards,

Re: Plugin to generate .epub file

Posted: 16 May 2015, 20:52
by JustAMacUser
I think this is an interesting idea. I didn't really look at your code but I would like to suggest a few things that could help.

Take a look at TT-RSS's built-in email digest. See how it parses for articles and sends them, etc. My thought is that instead of playing around with using the backend.php and dealing with authentication, you could perhaps generate the epub file and email it your ereader (does Kobo have a service like Kindle where you can receive stuff on it via email?). At the very least, that section of the core should have a good chunk of code you could re-use for this purpose. Also, instead of having a button that you click to manually generate this, create a user-defined checkbox (just like the email digest feature), so you (and optionally other users) can turn this feature on and off per account.

Instead of using cron to generate this stuff, use TT-RSS's built-in maintenance hooks (e.g. HOOK_HOUSEKEEPING or something), which is already running regularly as part of the software's schedule for fetching and cleaning up feeds.

Re: Plugin to generate .epub file

Posted: 16 May 2015, 23:52
by elenzar
Using the email digest as a base is indeed a pretty good idea, since it would both solve the problem and result in a more elegant solution. Beside it may allow to send it by email quite easily (even if I would not have any use of it, my device does not offer to receive stuff through email; I have it synced with a folder on a website). Still, this week was the first time I went through some of the TTRSS code, so I am not well aware of how difficult adapting this part of the code will be.

I shall have a closer look in the code, to see what I can do. Will keep you informed of what I could do.


Re: Plugin to generate .epub file

Posted: 31 May 2015, 20:14
by elenzar
I could take a closer look this week end, and here is what I have come about:
* run a php file built on "include/digest.php" from "include/rssfunc.php", which is where the email digest is run rom if I am correct;
* add a section in "class/pref/prefs.php" to allow the admin to enable this feature (by adding some entries to the databases) and to allow users to enable this epub generation when the admin turn the feature on.

Now I just have to work. Since I am not used to developing and coding I do not know how much time it will take me. You shall hear from me again when I am stuck or, better, when I am done :)

If you have any remarks I still take it gratefully.

Re: Plugin to generate .epub file

Posted: 31 May 2015, 21:19
by JustAMacUser
What you're looking to do can be done entirely with the available plugin hooks. Don't go modifying the core or you'll frustrate yourself down the road when it's updated.

Look at *how* TT-RSS handles the email digest and implement a version using the plugin system. The plugin hooks exist for this reason.

Re: Plugin to generate .epub file

Posted: 03 Jun 2015, 00:16
by elenzar
OK for your suggestions, and thanks. I will work with the hooks. At the moment my first tries with the HOOK_UPDATE_TASK (the name seems pretty clear to me) are not successful, I will work to understand what is wrong. I took some code after the cache_starred_images plugin for this.

As about how TT-RSS handles the digest, I have the impression that this digest is called from the "include/rssfuncs.php" file. I cannot see any plugin handling this digest, and it seems to me that the "/include/digest.php" file which is called does not offer any hints about using the hooks. Once again I may be mistaken about that, but if this is the case, could you point it to me?

And once again, thanks for your help :)

Re: Plugin to generate .epub file

Posted: 03 Jun 2015, 04:18
by JustAMacUser
Right, the HOOK_UPDATE_TASK hook runs when the feeds are updated. This is a good time to do the task you want.

So, during that task, collect the unread articles, compile them into the ePub file and then save it in the cache directory for the user to download later or email it to the user.

If you're saving it, set up some type of download button on the front end so the user can download it. You'll probably want to have PHP handle serving the download so you can restrict users to only downloading the file meant for them. But get the first half working then worry about this.

The point about the built-in digest is just for you to look over it so you can see how it runs, queries for articles, etc.

Re: Plugin to generate .epub file

Posted: 03 Jun 2015, 13:54
by fox
housekeeping hook might be a better one for this, update task is probably per-process so it could be a bit too much

Re: Plugin to generate .epub file

Posted: 04 Jun 2015, 09:01
by elenzar
Very well, thank you both for your replies. Now I have to find the time to work on it and get some probing results.

Re: Plugin to generate .epub file

Posted: 14 Jun 2015, 18:16
by elenzar

Back to you because some things do not work, and I have troubles finding what is the cause. From what I gathered, my problem is that the hooks I use (whether UPDATE_TASK or HOUSEKEEPING) are not called or registered by the PluginHost instance. Since UPDATE_TASK should be run when using the "update.php" script, I focused on this one to troubleshoot what I could. Running this update script in CLI shows that the "run_hook" function is called with UPDATE_TASK as an argument, but the hooks array for this hook type is empty at this moment. Hence nothing fo what is contained in my plugin script is used.
Moreover, I think that the PluginHost instance is somehow reseted several times during the update process : when printing the number of elements in the hooks array for different types of hooks, I can see that these numbers are reset several times during the update process.

I thought that my install might be the cause (lighttpd server on my everyday laptop), but I find that my plugin script does not show more results on an Apache server.

Hence my question:
* do these hooks (UPDATE_TASK, HOUSEKEEPING) work for you? If so, could you provide me a simple exemple of a plugin script which use the hooks and is effectively working ? (the plugin to cache images from starred articles gives me some troubles as well, and is the only such example)
* did I miss something about the way the hooks are used?

Re: Plugin to generate .epub file

Posted: 14 Jun 2015, 20:08
by JustAMacUser
No one can tell you if you missed something since we have not seen your code. Obviously the hooks work, I have several plugins that run across multiple hooks and they work well.

Have you looked at other plugins in the plugins directory to see how they do things? The cache starred images is one I use that hooks the housekeeping call.

Your plugin must be in a class. The class must be a child of the Plugin class. You must have an init method and this method must register the hooks you want to use. Registering a hook basically tells the plugin host class that you want it to call your class when running. So if you want something to run during housekeeping you'd register the HOOK_HOUSE_KEEPING in your init method. Then you'd create a method named hook_house_keeping to do what you want.

e: I missed that part where you said you were using the cached starred images plugin. If that native plugin doesn't work, then you may have a broken install. Try re-install everything.

You can also add a constant DAEMON_EXTENDED_DEBUG as true in your config.php file to see additional output from the CLI updater.

Re: Plugin to generate .epub file

Posted: 14 Jun 2015, 20:16
by elenzar
I get from your answer that:
* all hooks do work, including HOUSEKEEPING and UPDATE_TASK, so I will try to understand what is wrong on my side, till I have the cached starred images plugin working properly;
* from your last paragraph, that I have understood well enough how plugins work.

So my problem right now is really about having these hooks running properly.

Thanks for your feedback and your suggestions.

Re: Plugin to generate .epub file

Posted: 14 Jun 2015, 21:06
by JustAMacUser
It doesn't look live you've pushed any recent commits to your GitHub project. However, there are some errors in the current one that could be causing trouble. Just at a quick glance...

the creationZip and deleteDir functions use chdir(), which could create problems with TT-RSS. It's also not necessary for their operation. You can just pass the path along with the file name. Also deleteDir() references "self" which leads me to believe you pulled it out of a class. Since it's not in a class this would certain generate errors. If you're going to add extra functions to do things you plugin needs, add them as private methods inside your plugin class and reference them as (for example) $this->deleteDir($dir).

I won't go into other things, but if you want the best help, you should push the latest updates to your repo so others can look at it.

Re: Plugin to generate .epub file

Posted: 14 Jun 2015, 23:58
by elenzar
Ths repository is indeed old, I did not update it since my first posts here. This was my early code to see if I could generate a useful .epub file from what I could gather from the TTRSS database, using the import/export plugin as a base code to do so. I do agree that it is not clean and that it should require some work to have a proper plugin, especially since I am quite clumsy writing PHP.
However what I mainly did since my first post is not about this code but really about using the hooks. At first I thought that I got something wrong since I could not have any results from using these hooks, but it now seems that it is really an issue about how I did my install. As a result none of what I did since my first post is really relevant pushing to the github repo.
I admit that my question today about my understanding of the working of the hooks was confusing since you actually don't have much to tell about that point; I am sorry you lost some time reviewing some of the early code on Github. Well, your time is not really lost since your advice will still be useful when it comes to produce a better code.

In any case, thanks again for your help.

edit: I just pushed the plugin code that I used as a base to understand where the issue in using the hooks stands. It seems quite simple to me and I do not think that it explains that I do not have any results. I expect this test plugin to create a file in the cache directory with some specific text in it when its hooks are called. It does not write what I expect though, which is why I think the problem is in my install and the way it handles the hooks.

Re: Plugin to generate .epub file

Posted: 15 Jun 2015, 08:54
by fox
its possible that housekeeping/update tasks are run in global context so they are for system plugins only.