Http 429/Too many requests

[erw]

Hi,

Recently I set up tt-rss and everything goes fine except one thing:
I need to fetch feeds from one website but couple different RSS and sometimes there are HTTP 429(too many requests) for some feeds.

Is there some kind of workaround?

[sleeper_service]

seriously?

how about not fetching them as often?

[erw]

So you want to answer without reading question first?

As I mentioned - it's not often it's just lots of feeds from one provider.

[linoth]

Well, since you've supplied literally zero useful details on the actual problem, it's hard to offer a given solution.

If it's something like fetching every four hours, you could try to stagger them by hand, by disabling updates on some and manually re-enabling an hour later, or what-have-you, in order to satisfy the provider's rate limit. That should work indefinitely unless you're that close to the limit, but if your server ever goes down for an extended period and can't update, you'll have to set it up all over again.

You could also potentially dig around and try to change the internal update scheduling, or set up a cron job to manually run the update script every hour, or however long you needed to wait between fetches, in order to enforce your own rate limit that way. Alternatively, you could lower the number of feeds that could be updated per daemon execution, if we're talking an X requests in Y minutes sort of thing.

I don't believe plugins would be of much use here, but I could very well be wrong.

I suppose you could get really fancy and write your own proxy script of sorts, which kept track of your own requests and enforced a rate limit before allowing the update to proceed as scheduled. That's starting to get a little excessive and convoluted, though.

But these are all pretty much common sense things.

Regardless, I'd recommend mellowing the hell out and actually providing some useful information before you get your ass banned. You're the one not being remotely helpful here. We don't know the site in question to get any answers for ourselves, we don't know the rate limit you're trying to conform to, we don't know how often you're actually updating, we don't know the actual scope of how many feeds we're talking about here, we don't know a goddamn thing and you're expecting miracles to come out of our asses to satisfy your entitlement.

[erw]

That what I thought.

It looks like there is quite enough info to answer the question:
1 provider, multiple feeds - HTTP 429 in result. What else do you need? How does it matter what provider is it or what the limit is?
Problem is clear - too many requests since tt updates them very quickly without pause between requests.

I hope simple sleep should help.

[JustAMacUser]

@erw,

You're so pleasant in your responses. It's really quite wonderful. /s

So you want to answer without reading question first?

As I mentioned - it's not often it's just lots of feeds from one provider.

[emphasis mine]

Side note: It's nice that you start your journey in the forums being snarky to a moderator.

So, you already "mentioned" that "it's not often"? Let's take a look-see:

Hi,

Recently I set up tt-rss and everything goes fine except one thing:
I need to fetch feeds from one website but couple different RSS and sometimes there are HTTP 429(too many requests) for some feeds.

Is there some kind of workaround?

At what point in your post did you actually mention that you're not fetching often?

At any rate, what is "often"? For some that might be 15-minutes and others that might be not more than once a day.

Frankly, your attitude to those helping you is less than desirable but I'm a nice guy so I'll give you some advice.

How does it matter what provider is it or what the limit is?

It matters because if we know who the provider is, we might be able to give some better advice. Larger ones like FeedBurner have limits that others have already encountered and managed; if that was the provider, we could give more targeted advice. If it's joe-some-guy-blog-with-feeds-for-each-category that's a different story.

Problem is clear - too many requests since tt updates them very quickly without pause between requests.

The problem really is clear. Abundantly so. You're fetching too frequently, as you said. The solution is to fetch less frequently.

That being said, you could write a plugin to hook the maintenance function in TT-RSS. It runs hourly by default, and then you could manually modify fetch times for the feeds based on when other feeds were updated. Or you could hook just before feeds are fetched, check the times and return erroneous data to TT-RSS so it aborts fetching, etc. Both are hacks but they could be coded in plugins to do what you want without much upkeep thereafter. Another thing you could do if you're on a VPS is add more IPs and setup some scripts to rotate which IP is used for outbound requests to port 80 and 443.

Do you want the real solution?

Talk to the owner of the web site. Explain what you're doing and why, and ask how often he or she recommends you fetch. Respect their response. Problem. Solved.

[sleeper_service]

So you want to answer without reading question first?

As I mentioned - it's not often it's just lots of feeds from one provider.

not my fault if you're too mentally insufficient to understand that if you're hammering on ONE provider with LOTS of feeds, they might very well decide you're trying to... what was it ??? "Http 429/Too many requests" make ... "too many requests".

but, hey, if you're too dumb to understand that, I'd be happy to suspend your account for a couple weeks so you can try to think on it some more.

just let me know.

e: on second thought, why muffle your shit when you're providing so much mock worthy fodder.

[erw]

Like I care.

[erw]

C'mon guys, I expected like 2-3 more.
But I should admit @JustAMacUser - that was incredible, like lots of words, really lots.

[sleeper_service]

I'm torn, ban him and delete the thread, or move it to the comedy bistro?

[JustAMacUser]

I'm torn, ban him and delete the thread, or move it to the comedy bistro?

Well, I don't think this place is a democracy. So...

e: If it were my choice, I'd vote is comedy bistro..

[sleeper_service]

I'm torn, ban him and delete the thread, or move it to the comedy bistro?

Well, I don't think this place is a democracy. So...

e: If it were my choice, I'd vote is comedy bistro..

I'll don't have the cosmic godlike powers to actually move it into the 'bistro, we'll have to wait for his nibs to surface

[fox]

this sure was a thread. idk about its bistro worthiness though.

also not sure why op got all salty re: his obviously torrent feeds *shocked gasps in the audience*.

technically i guess in this situation there's no provided solution. other than running some kind of adhoc proxy like suggested above. i'm not sure if something like that could be implemented with the way updating works and whether it would be a good idea.

[disconn3ct]

this sure was a thread. idk about its bistro worthiness though.

also not sure why op got all salty re: his obviously torrent feeds *shocked gasps in the audience*.

Could be hammering reddit porn.

[sleeper_service]

probably nothing that high class.