Adaptive feed update interval #51

New issue

Closed

opened 2017-04-14 19:01:21 -04:00 by jking · 1 comment

jking commented

2017-04-14 19:01:21 -04:00

Owner

A. If the feed provides update hints, use them

B. Otherwise if the feed was unchanged and:

If it's been less than half an hour since it has changed, check in 15 minutes
If it's been less than an hour since it changed, check in 30 minutes
If it's been less than three hours, check in an hour
If it's been more than 36 hours, check in 24 hours
Otherwise check in three hours

C. Or if the feed was changed, compute the three intervals between the last four articles, and if any two fall within time frames listed in B, use that; otherwise check in an hour.

A. If the feed provides update hints, use them B. Otherwise if the feed was unchanged and: 1. If it's been less than half an hour since it has changed, check in 15 minutes 2. If it's been less than an hour since it changed, check in 30 minutes 3. If it's been less than three hours, check in an hour 4. If it's been more than 36 hours, check in 24 hours 5. Otherwise check in three hours C. Or if the feed was changed, compute the three intervals between the last four articles, and if any two fall within time frames listed in B, use that; otherwise check in an hour.

jking commented

2017-04-24 10:34:42 -04:00

Author

Owner

Addendum:

A is unrealistic without caching the hints, otherwise they are not available when the upstream server return 304. Given that such hints are not commonly used, it's almost certainly not worth the effort.

Additionally, an algorithm for repeated errors is separately required as well. Algo B could be used, but it probably makes more sense to do the following:

If less than three erroneous checks (not including current check), check in five minutes
If less than 15 erroneous checks (not including current check), check in three hours
Thereafter check in 24 hours

This allows for a quicker initial re-check, with a quick back-off, and a daily check after roughly 36 hours.

Addendum: A is unrealistic without caching the hints, otherwise they are not available when the upstream server return 304. Given that such hints are not commonly used, it's almost certainly not worth the effort. Additionally, an algorithm for repeated errors is separately required as well. Algo B could be used, but it probably makes more sense to do the following: 1. If less than three erroneous checks (not including current check), check in five minutes 2. If less than 15 erroneous checks (not including current check), check in three hours 3. Thereafter check in 24 hours This allows for a quicker initial re-check, with a quick back-off, and a daily check after roughly 36 hours.