Adaptive feed update interval #51

Closed
opened 7 years ago by jking · 1 comments
jking commented 7 years ago
Owner

A. If the feed provides update hints, use them

B. Otherwise if the feed was unchanged and:

  1. If it's been less than half an hour since it has changed, check in 15 minutes
  2. If it's been less than an hour since it changed, check in 30 minutes
  3. If it's been less than three hours, check in an hour
  4. If it's been more than 36 hours, check in 24 hours
  5. Otherwise check in three hours

C. Or if the feed was changed, compute the three intervals between the last four articles, and if any two fall within time frames listed in B, use that; otherwise check in an hour.

A. If the feed provides update hints, use them B. Otherwise if the feed was unchanged and: 1. If it's been less than half an hour since it has changed, check in 15 minutes 2. If it's been less than an hour since it changed, check in 30 minutes 3. If it's been less than three hours, check in an hour 4. If it's been more than 36 hours, check in 24 hours 5. Otherwise check in three hours C. Or if the feed was changed, compute the three intervals between the last four articles, and if any two fall within time frames listed in B, use that; otherwise check in an hour.
Poster
Owner

Addendum:

A is unrealistic without caching the hints, otherwise they are not available when the upstream server return 304. Given that such hints are not commonly used, it's almost certainly not worth the effort.

Additionally, an algorithm for repeated errors is separately required as well. Algo B could be used, but it probably makes more sense to do the following:

  1. If less than three erroneous checks (not including current check), check in five minutes
  2. If less than 15 erroneous checks (not including current check), check in three hours
  3. Thereafter check in 24 hours

This allows for a quicker initial re-check, with a quick back-off, and a daily check after roughly 36 hours.

Addendum: A is unrealistic without caching the hints, otherwise they are not available when the upstream server return 304. Given that such hints are not commonly used, it's almost certainly not worth the effort. Additionally, an algorithm for repeated errors is separately required as well. Algo B could be used, but it probably makes more sense to do the following: 1. If less than three erroneous checks (not including current check), check in five minutes 2. If less than 15 erroneous checks (not including current check), check in three hours 3. Thereafter check in 24 hours This allows for a quicker initial re-check, with a quick back-off, and a daily check after roughly 36 hours.
jking closed this issue 7 years ago
Sign in to join this conversation.
No Milestone
No Assignees
1 Participants
Notifications
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Loading…
There is no content yet.