Bayesian classifier for TTRSS

Development-related discussion, including bundled plugins
JustAMacUser
Bear Rating Overlord
Bear Rating Overlord
Posts: 373
Joined: 20 Aug 2013, 23:13

Re: Bayesian classifier for TTRSS

Postby JustAMacUser » 18 Jun 2015, 01:27

Perhaps excluding all articles and conjunctions (in English, anyway—I don't know about other languages).

Something like

pcause
Bear Rating Master
Bear Rating Master
Posts: 144
Joined: 23 Aug 2013, 19:52

Re: Bayesian classifier for TTRSS

Postby pcause » 18 Jun 2015, 03:30

Wikipedia has a list of the 100 most common English words:



all of these should likely be excluded.

e: here is one that has the 500 most common and a link to common verbs



trick is to pick the right amount to skip.

JustAMacUser
Bear Rating Overlord
Bear Rating Overlord
Posts: 373
Joined: 20 Aug 2013, 23:13

Re: Bayesian classifier for TTRSS

Postby JustAMacUser » 18 Jun 2015, 03:52

Verbs and nouns are important though. Conjunctions (and, but, ...) and articles (a, the, ...) are more related to syntax and don't really "matter" (so to speak).

e: That Wikipedia list is pretty good. The second list I think is a bit too inclusive.

rknobbe-other
Bear Rating Trainee
Bear Rating Trainee
Posts: 13
Joined: 11 Jun 2015, 22:37

Re: Bayesian classifier for TTRSS

Postby rknobbe-other » 18 Jun 2015, 07:31


User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Bayesian classifier for TTRSS

Postby fox » 18 Jun 2015, 08:43

adding more stopwords looks like a good idea

https://github.com/gothfox/Tiny-Tiny-RS ... f7aa1cee43

e: i'm running it now with trigrams which imo should match better than using whole words especially since there's no stemming and stuff, it's kinda slow but overall works

AngryChris
Bear Rating Master
Bear Rating Master
Posts: 135
Joined: 08 Apr 2013, 02:42

Re: Bayesian classifier for TTRSS

Postby AngryChris » 18 Jun 2015, 14:53


User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Bayesian classifier for TTRSS

Postby fox » 18 Jun 2015, 15:13

well if you didn't enable the plugin there should be no difference in update process

try stepping back through changesets to see where it stops working, maybe restart the daemon? it's hard to say really because update stuff has not really been changed at all when plugin went in, especially nothing wrt how locking/subprocesses work

e: also try single process daemon, see if it hangs or does something strange

pcause
Bear Rating Master
Bear Rating Master
Posts: 144
Joined: 23 Aug 2013, 19:52

Re: Bayesian classifier for TTRSS

Postby pcause » 18 Jun 2015, 17:53

fox, thanks for adding more stop words. i'd think this is an area that you might want to let a user tune with an ability to add/remove words and perhaps treat certain words as +/- indicators. for example, if I am an apple fanbois I might want to add +iphone, +iwatch, +ios, -android and have any articles in any category about apple treated as good and about android treated as bad. words in the list without a +/- are treated as stop words, words with treated as good/bad indicators.

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Bayesian classifier for TTRSS

Postby fox » 18 Jun 2015, 18:00

that is highly unlikely but i'm not stopping anyone to make whatever kitchen sink plugin they want based on mine

e: also the whole point of having a bayesian classifier is that you won't need to specify manual rules like that, otherwise just use filters

AngryChris
Bear Rating Master
Bear Rating Master
Posts: 135
Joined: 08 Apr 2013, 02:42

Re: Bayesian classifier for TTRSS

Postby AngryChris » 18 Jun 2015, 18:11


AngryChris
Bear Rating Master
Bear Rating Master
Posts: 135
Joined: 08 Apr 2013, 02:42

Re: Bayesian classifier for TTRSS

Postby AngryChris » 18 Jun 2015, 20:01

Disabling the Vice feed made the issue go away. I'm going to work out a post for a new thread. While I accept the "feed is broken" -- having the updater fail on it and stop working for all feeds sounds like something possibly worth thinking of a work around for.

User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Bayesian classifier for TTRSS

Postby fox » 18 Jun 2015, 20:11

i'm currently subbed to it and it seems to work fine so far

i'm also still not sure whether you have bayes plugin enabled or not

Athanasius
Bear Rating Trainee
Bear Rating Trainee
Posts: 38
Joined: 02 Apr 2013, 21:01

Re: Bayesian classifier for TTRSS

Postby Athanasius » 18 Jun 2015, 20:29


User avatar
fox
^ me reading your posts ^
Posts: 6318
Joined: 27 Aug 2005, 22:53
Location: Saint-Petersburg, Russia
Contact:

Re: Bayesian classifier for TTRSS

Postby fox » 18 Jun 2015, 20:44

tbf that's all minor shit which shouldn't cause daemon crashes

nameless
Bear Rating Master
Bear Rating Master
Posts: 126
Joined: 28 Aug 2013, 20:33

Re: Bayesian classifier for TTRSS

Postby nameless » 18 Jun 2015, 23:19



Return to “Development”

Who is online

Users browsing this forum: No registered users and 5 guests