Long term code goals

Development-related discussion, including bundled plugins
ifireball
Bear Rating Trainee
Bear Rating Trainee
Posts: 6
Joined: 24 Mar 2013, 01:47

Long term code goals

Postby ifireball » 02 Apr 2013, 23:21

Hi there, been pushing small patches in lately, and beginning to think about going for bigger fish.
Stuff like adding DB query logging, and ways to do tuning like using prepared queries.
This would probably be a lot of work, so I been wondering, what does the code future look like? I wouldn't want to do all this work just to have to redo it when some new DAL gets pushed in.
So what is in TT-RSS's long-term future? Some DAL or ORM? Rebasing on some framework?

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Long term code goals

Postby fox » 02 Apr 2013, 23:28

The future mostly involves me hacking on shit when I'm bored until it stops being fun. I have no idea what DAL or ORM is, but I hope it helped.

jchristi
Bear Rating Trainee
Bear Rating Trainee
Posts: 17
Joined: 03 Apr 2013, 21:18

Re: Long term code goals

Postby jchristi » 03 Apr 2013, 21:24

He's referring to Database Abstraction Layer such as PHP Data Objects (PDO) and Object-relational Mapping such as ActiveRecord (Ruby), Django ORM (Python), CakePHP ORM, etc. ifireball, I was somewhat interested in the same question but fox's answer seemed to help. It doesn't sound like he has any major refactoring plans. However, rather than focusing on the SQL aspect, it would probably yield more performance benefits to look into better caching schemes and NoSQL.

ifireball
Bear Rating Trainee
Bear Rating Trainee
Posts: 6
Joined: 24 Mar 2013, 01:47

Re: Long term code goals

Postby ifireball » 06 Apr 2013, 13:04

jchristi, NoSQL is a buzzword. Its good for some things but not others, and its certainly not the solution for everything. Having looked through TT-RSS DB code, I can tell you that a NoSQL DB will probably not do much good there, as many queries are somewhat complex and not key-based, and also becasue TT-RSS leans heavily on RDBMS functionality such as foreign keys.
With regards to caching - it helps when you have multipile webservers running aginst one DB host. but I think a typical TT-RSS install has the DB running on the same host as the webserver. When this happens, when you cache you only compete with the DB for RAM that it could use to do its own caching (which is way more efficient then anything PHP can do).
Having said that - there are many things you can do to speed things up when you talk to a relational DB, and many DB engines allow you to dig into query plans and figure out how to optimize them, but in order to do that you need some instrumentation at the app level. At the very basic level, you need to be able to generate a log showing which queries the APP is running, at what frequency and how long each one is taking.
My plan for now is to try and generate such log, but since the functions in db.php are really deep-level app core suff, I plan to write some unit tests around them before I try anything, becasue I don't feel like breaking stuff and not knowing about it.
With regrad to PDO, I've seen pull request trying to switch from mysql_* functions to mysqli_*. It seems that fox is concerned about protability, so I think PDO is out of the question since it seems to be stable enough only in php 5.3.0 and up.

Joschasa
Bear Rating Trainee
Bear Rating Trainee
Posts: 34
Joined: 03 May 2010, 11:58

Re: AW: Long term code goals

Postby Joschasa » 06 Apr 2013, 16:09


ifireball
Bear Rating Trainee
Bear Rating Trainee
Posts: 6
Joined: 24 Mar 2013, 01:47

Re: AW: Long term code goals

Postby ifireball » 06 Apr 2013, 16:19


jchristi
Bear Rating Trainee
Bear Rating Trainee
Posts: 17
Joined: 03 Apr 2013, 21:18

Re: Long term code goals

Postby jchristi » 06 Apr 2013, 20:15


ifireball
Bear Rating Trainee
Bear Rating Trainee
Posts: 6
Joined: 24 Mar 2013, 01:47

Re: Long term code goals

Postby ifireball » 06 Apr 2013, 20:40

That kind of cache only helps if you tend to run the same query again and again, whereas the DB engine cache can optimize the case where different queries go to the same data block for example. The DB can be extremely efficient CPU-wise if you let it, techniques like using parametrized queries can do wonders to decreasing CPU usage for SQL parsing. Of corse if you only used MySQL with the mysql_* functions so far you'd think its a CPU hog because you force it to re-parse the query and rebuild the query plan every time.
Typically you cache by copying data you get from the DB to an in-memory hash table, the thing is, that by the time you simplify your query enough to map it into a hash table, you also simplify it enough for the DB that it essentially goes for the same hash table approach in its own memory and you end up with 2 copies of almost the same hash table (And you also have to implement stuff LRU paging on your own, where the DB gives you this for free, or you can use memcached which is essentially a simplified DB engine...).
In any case I suppse the right thing to do is allow implementing both approaches and let site admins work out what works best for them with optional config.php parameters, but that requires some clean-up of db.php code...


Return to “Development”

Who is online

Users browsing this forum: No registered users and 3 guests