Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to BYSETPOS support #449

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
Open

Conversation

nehresma
Copy link

In August 2016, Nicolas Marlier added BYSETPOS support (#349). This PR updates that PR with a few small changes to run in modern Ruby (use Integer instead of Fixnum) and a more modern rspec.

@nehresma
Copy link
Author

Hat tip to @NicolasMarlier for doing this.

@davidstosik
Copy link

I can confirm this works as expected, this is fantastic! 👍

Copy link

@davidstosik davidstosik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left comments to see if we can improve the code of this PR.
I'll try implementing them myself and link a diff here a bit later.

.gitignore Outdated
@@ -8,5 +8,8 @@
/spec/reports/
/tmp/

# rubymine
.idea

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that necessary? It looks like an accidental addition that is not part of this PR.

@@ -198,6 +200,36 @@ def self.which_occurrence_in_month(time, wday)
[nth_occurrence_of_weekday, this_weekday_in_month_count]
end

# Use Activesupport CoreExt functions to manipulate time
def self.start_of_month time
time.beginning_of_month

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not reuse the same name?


# Use Activesupport CoreExt functions to manipulate time
def self.start_of_year time
time.beginning_of_year

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not reuse the same name?


# Use Activesupport CoreExt functions to manipulate time
def self.previous_month time
time - 1.month

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think time.last_month exists?


# Use Activesupport CoreExt functions to manipulate time
def self.previous_year time
time - 1.year

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think time.last_year exists?

return by_set_pos([by_set_pos]) if by_set_pos.is_a?(Integer)

unless by_set_pos.nil? || by_set_pos.is_a?(Array)
raise ArgumentError, "Expecting Array or nil value for count, got #{by_set_pos.inspect}"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This validation seems unnecessary as well. by_set_pos is always an Array.

end
by_set_pos.flatten!
by_set_pos.each do |set_pos|
unless (set_pos >= -366 && set_pos <= -1) ||

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about (-366..366).include(set_post) && set_pos != 0?

end

@by_set_pos = by_set_pos
replace_validations_for(:by_set_pos, by_set_pos && [Validation.new(by_set_pos, self)])

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, by_set_pos cannot be nil or false here (it's an Array), in which case by_set_pos && [Validation.new(by_set_pos, self)] is equivalent to its right-side operand, [Validation.new(by_set_pos, self)].

lib/ice_cube/validations/yearly_by_set_pos.rb Show resolved Hide resolved
@nehresma
Copy link
Author

nehresma commented Jan 4, 2019

Hello @davidstosik, @NicolasMarlier provided this original PR. I simply updated it to work with modern versions of Ruby.



new_schedule = IceCube::Schedule.new(TimeUtil.previous_month(step_time)) do |s|
s.add_recurrence_rule IceCube::Rule.from_hash(rule.to_hash.reject{|k, v| [:by_set_pos, :count, :until].include? k})
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With ActiveSupport, this reject construct simplifies to except(:by_set_pos, :count, :util)



new_schedule = IceCube::Schedule.new(TimeUtil.previous_year(step_time)) do |s|
s.add_recurrence_rule IceCube::Rule.from_hash(rule.to_hash.reject{|k, v| [:by_set_pos, :count, :until].include? k})
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here, use Hash#except

schedule = IceCube::Schedule.from_ical "RRULE:FREQ=MONTHLY;COUNT=4;BYDAY=WE;BYSETPOS=4"
schedule.start_time = Time.new(2015, 5, 28, 12, 0, 0)
expect(schedule.occurrences_between(Time.new(2015, 01, 01), Time.new(2017, 01, 01))).to eq([
Time.new(2015,6,24,12,0,0),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say just breaking .to onto a newline is good enough

joshbeckman added a commit to officeluv/ice_cube that referenced this pull request Feb 5, 2019
In August 2016, @NicolasMarlier added BYSETPOS support (ice-cube-ruby#349).

Then, in July 2018, @nehresma added a few small changes to run in modern
Ruby and a more modern rspec
(ice-cube-ruby#449).

Then, in January 2019, @davidstosik and @k3rni suggested changes to
reduce complexity.

This incorporates all the above into a single diff.
@dimerman
Copy link
Contributor

@nehresma what's the status of this PR? I'd very much like to use the setpos functionality.

@nehresma
Copy link
Author

@dimerman It has been merged but I do not believe a new version of the gem has been released. I've pointed my Gemfile to the latest git ref on master for the time being. Hopefully someone will have a few minutes to cut a new gem release at some point so we wont have to point to specific refs.

@nehresma
Copy link
Author

@dimerman Err, no -- I'm an idiot. I just realized it has NOT been merged. I checked my Gemfile and I'm pointing to the ref with BYSETPOS support out on my fork -- not master on seejohnrun/ice_cube.

Sorry for the confusion. I should have double checked before trusting my memory.

@dimerman
Copy link
Contributor

dimerman commented Mar 23, 2019

@nehresma no worries - I've been pointing my gemfile to your branch/repo. thanks for that!

However, I found the following case is not working right:

rule_a="FREQ=MONTHLY;COUNT=12;BYDAY=MO,TU,WE,TH,FR;BYSETPOS=-1"
rule_b="FREQ=MONTHLY;COUNT=12;BYDAY=MO,TU,WE,TH,FR;BYMONTHDAY=13,14,15;BYSETPOS=-1"

when I create a IceCube::Schedule and add each rule individually, the occurrences are accurate. But if I create a schedule with both these rules, I get 18 occurrences instead of the expected 24.
I thought you might want to know. 🙌

@mattbooks
Copy link

What's still holding this back — the PR feedback seems pretty minor? If that gets cleaned up can we get this merged?

@mattbooks
Copy link

@seejohnrun It seems like several people need this and are using forks to get around it being missing. I'd be more than happy to help clean this up to avoid using a fork myself if it is otherwise merge-able.



def build_s(builder)
builder.piece(:by_set_pos) << by_set_pos
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @nehresma

Looks like this should be improved:

> IceCube::Rule.monthly(2).day(1,2,3,4,5).by_set_pos(2).to_s
"Every 2 months on Weekdays [2]"

@pacso
Copy link
Collaborator

pacso commented Oct 21, 2021

@nehresma / @mattbooks / @ovinix - If you can get this PR cleaned up and resolve the feedback and conflicts with master, I'll give it a proper review.

@Naren1997
Copy link

Naren1997 commented Mar 6, 2022

Hello, looks like the gem IS already supporting what BYTSETPOS does but in different ical format. Example: If we want recurring for every 2 months 3rd Wednesday, ical format w.r.t the BYSETPOS is FREQ=MONTHLY;INTERVAL=2;BYDAY=WE;BYSETPOS=3 which is same as FREQ=MONTHLY;INTERVAL=2;BYDAY=3WE, the latter one is already supported by the gem, so not sure why we still wants this PR.

@davidstosik
Copy link

not sure why we still wants this PR

I don’t use the gem anymore, but you might want to consider that some people use the gem to parse strings produced by another piece of software they don’t have control on, and this would require the whole specification to be supported. 😉

@jhubert
Copy link

jhubert commented May 13, 2022

@avit Anything we can do to help get this merged?

@dimerman
Copy link
Contributor

@davidstosik what gem do you use instead?

@davidstosik
Copy link

davidstosik commented May 19, 2022

Nothing. 😬
My need to handle repeating schedules in Ruby has vanished. 🙇🏻‍♂️

Nicolas Marlier and others added 4 commits December 12, 2022 11:15
@nehresma
Copy link
Author

It's been over 4 years since I created this PR and over 7 years since Nicolas Marlier's initial commit with this support. I have rebased it against current master and I've applied changes per the feedback received above.

If someone has time to review and merge, that would be most appreciated. BYSETPOS is part of RFC 5545 and it would be great if this gem could finally get support merged in rather than just the "noop" that it currently is (https://github.com/ice-cube-ruby/ice_cube/blob/master/lib/ice_cube/parsers/ical_parser.rb#L77-L78).

@nehresma
Copy link
Author

@jkehres I've fixed the interval use for month and year frequencies with bysetpos. I ran out of time for adding support for the other frequencies yet. I'll add support for those after Christmas.

@nehresma
Copy link
Author

I've pushed bysetpos support for weekly frequencies too. I don't think bysetpos makes sense with frequencies of daily, hourly, minutely, or secondly. RFC 5545 makes no mention of them, and indeed all of it's examples are with monthly and weekly frequencies. If someone can think of an example where using bysetpos with one of these other frequencies makes sense, let me know.

@@ -78,7 +78,7 @@ module IceCube
end

it "should be able to make a round-trip to YAML with .day_of_year" do
schedule1 = Schedule.new(Time.now)
schedule1 = Schedule.new(Time.zone.now)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Time.zone is another active support extension. The stock Time class in Ruby has no zone method.

@jkehres
Copy link

jkehres commented Jan 11, 2023

I've pushed bysetpos support for weekly frequencies too. I don't think bysetpos makes sense with frequencies of daily, hourly, minutely, or secondly. RFC 5545 makes no mention of them, and indeed all of it's examples are with monthly and weekly frequencies. If someone can think of an example where using bysetpos with one of these other frequencies makes sense, let me know.

I believe BYSETPOS support is still needed for daily, hourly, and minutely frequencies to be compliant with the RFC, which I think should be the goal of this PR. As I mentioned in my comment here, you can see BYSETPOS having an effect for these other frequencies using rrule.js - another library that implements RFC 5545 (itself a port of the python-dateutil library). If ice_cube and these other libraries all properly implement the spec, they should all give the same answer for the same RRULE. We could use that fact to aid in testing this PR to ensure we have a compliant implementation of the RFC. We could us one of these other libraries to generate a more exhaustive set of test inputs and outputs for RRULEs that use BYSETPOS and, thus, increase the number of tests for this new code along with our confidence in it.

Secondly, even though the RFC gives no examples for these other frequencies with BYSETPOS, I think you can derive that they are necessary from the details in the spec.

From the description of BYSETPOS (emphasis mine):

The BYSETPOS rule part specifies a COMMA-separated list of values that corresponds to the nth occurrence within the set of recurrence instances specified by the rule. BYSETPOS operates on a set of recurrence instances in one interval of the recurrence rule.

My understanding of the spec is that "interval" in this context means one unit of the frequency regardless of the INTERVAL rule part so one year for yearly, one month for monthly, etc. If there are multiple recurrences instances within one interval (e.g. year, month, etc.), then BYSETPOS has an effect and selects the nth one. Normally, there is only a single recurrence instance within an interval. The way you get more than one recurrence instance in an interval is by adding another BYxxx rule part.

Again, from the spec (emphasis mine):

BYxxx rule parts modify the recurrence in some manner. BYxxx rule parts for a period of time that is the same or greater than the frequency generally reduce or limit the number of occurrences of the recurrence generated. For example, "FREQ=DAILY;BYMONTH=1" reduces the number of recurrence instances from all days (if BYMONTH rule part is not present) to all days in January. BYxxx rule parts for a period of time less than the frequency generally increase or expand the number of occurrences of the recurrence. For example, "FREQ=YEARLY;BYMONTH=1,2" increases the number of days within the yearly recurrence set from 1 (if BYMONTH rule part is not present) to 2.

Furthermore, this table from the spec shows how each BYxxx rule part interacts with a given frequency and whether it expands or limits the number of recurrences instances:

   +----------+--------+--------+-------+-------+------+-------+------+
   |          |SECONDLY|MINUTELY|HOURLY |DAILY  |WEEKLY|MONTHLY|YEARLY|
   +----------+--------+--------+-------+-------+------+-------+------+
   |BYMONTH   |Limit   |Limit   |Limit  |Limit  |Limit |Limit  |Expand|
   +----------+--------+--------+-------+-------+------+-------+------+
   |BYWEEKNO  |N/A     |N/A     |N/A    |N/A    |N/A   |N/A    |Expand|
   +----------+--------+--------+-------+-------+------+-------+------+
   |BYYEARDAY |Limit   |Limit   |Limit  |N/A    |N/A   |N/A    |Expand|
   +----------+--------+--------+-------+-------+------+-------+------+
   |BYMONTHDAY|Limit   |Limit   |Limit  |Limit  |N/A   |Expand |Expand|
   +----------+--------+--------+-------+-------+------+-------+------+
   |BYDAY     |Limit   |Limit   |Limit  |Limit  |Expand|Note 1 |Note 2|
   +----------+--------+--------+-------+-------+------+-------+------+
   |BYHOUR    |Limit   |Limit   |Limit  |Expand |Expand|Expand |Expand|
   +----------+--------+--------+-------+-------+------+-------+------+
   |BYMINUTE  |Limit   |Limit   |Expand |Expand |Expand|Expand |Expand|
   +----------+--------+--------+-------+-------+------+-------+------+
   |BYSECOND  |Limit   |Expand  |Expand |Expand |Expand|Expand |Expand|
   +----------+--------+--------+-------+-------+------+-------+------+
   |BYSETPOS  |Limit   |Limit   |Limit  |Limit  |Limit |Limit  |Limit |
   +----------+--------+--------+-------+-------+------+-------+------+

For secondly, you can see that no BYxxx rule part will expand the set of recurrence instances so BYSETPOS won't have an effect. However, for minutely, hourly, and daily frequencies you can see that some BYxxx rule parts will expand the set of recurrences, which would allow BYSETPOS to select the nth one in an interval and thus alter the final set of recurrence instances.

describe WeeklyRule, 'BYSETPOS' do
it 'should behave correctly' do
# Weekly on Monday, Wednesday, and Friday with the week starting on Wednesday, the last day of the set
schedule = IceCube::Schedule.from_ical("RRULE:FREQ=WEEKLY;COUNT=4;WKST=WE;BYDAY=MO,WE,FR;BYSETPOS=-1")
Copy link

@jkehres jkehres Jan 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For each of the frequencies, it would be nice to have the following test cases:

  • With a BYxxx that expands the set of recurrence instances...
    • BYSETPOS with a positive value
    • BYSETPOS with a negative value
    • BYSETPOS with multiple positive values
    • BYSETPOS with multiple negative values
    • BYSETPOS with multiple values both positive and negative
    • BYSETPOS with multiple repeated values - spec does not define behaviour; rrule.js does some weird things and outputs the same occurrence multiple times (bug or maybe just undefined behaviour?)
    • BYSETPOS with a value that matches in some intervals but not others (e.g. selecting the 5th Monday of every month since some months will have only 4 Mondays)
    • BYSETPOS with a positive value beyond than the number of recurrence instances in any interval (e.g. the 10th Monday of every month); rrule.js returns an empty set of occurrences
    • BYSETPOS with a negative value bigger than the number of recurrence instances in any set (e.g. the -10th Monday of every month); rrule.js returns a non-empty set of occurrences, which is weird (bug or maybe just undefined behaviour?)
  • BYSETPOS with each BYxxx that expands the set of recurrence instances for the current frequency
  • BYSETPOS with BYxxx that limits the set of recurrence instances instead of expanding it
  • BYSETPOS with no BYxxx - spec says BYSETPOS "MUST only be used in conjunction with another BYxxx rule part" so not sure if this should be an exception or an empty set of occurrences returned; rrule.js does the latter
  • BYSETPOS with multiple BYxxx rules parts that expand the recurrence instances
  • BYSETPOS with multiple BYxxx rules parts that both expand and limit the recurrence instances
  • BYSETPOS with a value outside the allowed range (covered in the from_ical tests)

@@ -75,7 +75,7 @@ def self.rule_from_ical(ical)
when "BYYEARDAY"
validations[:day_of_year] = value.split(",").map(&:to_i)
when "BYSETPOS"
# noop
params[:validations][:by_set_pos] = value.split(',').collect(&:to_i)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps use the style validations[:by_set_pos] to match everything else in this case statement.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, existing code uses double quotes instead of single quotes for string in split and collect instead of map. Picky but just for consistency.

Comment on lines +1 to +2
require 'date'
require 'time'
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing the type of quotes seems unnecessary. Double quotes are consistent with the existing style.

], step_time
)
new_schedule = IceCube::Schedule.new(start_of_week_adjusted) do |s|
s.add_recurrence_rule(IceCube::Rule.from_hash(rule.to_hash.except(:by_set_pos, :count, :until)))
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


# Needs to start on the first day of the month
new_schedule = IceCube::Schedule.new(IceCube::TimeUtil.build_in_zone([start_of_month.year, start_of_month.month, start_of_month.day, step_time.hour, step_time.min, step_time.sec], start_of_month)) do |s|
s.add_recurrence_rule(IceCube::Rule.from_hash(rule.to_hash.except(:by_set_pos, :count, :until)))
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto my comment about except.


# Needs to start on the first day of the year
new_schedule = IceCube::Schedule.new(IceCube::TimeUtil.build_in_zone([start_of_year.year, start_of_year.month, start_of_year.day, step_time.hour, step_time.min, step_time.sec], start_of_year)) do |s|
s.add_recurrence_rule(IceCube::Rule.from_hash(rule.to_hash.except(:by_set_pos, :count, :until)))
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto my comment about except.

@jkehres
Copy link

jkehres commented Jan 11, 2023

Using BYSECOND with MONTHLY does not work. I tried this:

ical = "RRULE:FREQ=MONTHLY;COUNT=4;BYSECOND=1,2,3,4;BYSETPOS=2"
schedule = IceCube::Schedule.from_ical(ical)
schedule.start_time = Time.new(2015, 5, 28, 12, 0, 0)
schedule.occurrences_between(Time.new(2015, 01, 01), Time.new(2017, 01, 01))

and got an empty array. I expected to get:

2015-05-28 12:00:02
2015-06-28 12:00:02
2015-07-28 12:00:02
2015-08-28 12:00:02

I suspect there is an assumption in the code that you will only use BYDAY with MONTHLY and not other BYxxx rule parts.

@rgalanakis
Copy link

First I want to thank @nehresma @jkehres and others for all the work on this.

I believe BYSETPOS support is still needed for daily, hourly, and minutely frequencies to be compliant with the RFC, which I think should be the goal of this PR.

The current code already doesn't implement the RFC, and it fails to do so silently (just spent over an hour finding the this as the source of a bug). I think partial RFC compliance is better than none. And the users who use daily/hourly/minutely frequencies would continue to see the current unimplemented behavior. I know I wouldn't be alone in appreciating getting this work merged.

@pacso
Copy link
Collaborator

pacso commented Jul 20, 2023

Thanks for the review @jkehres. Can we get the tests running/passing?

@jorgemanrubia
Copy link

Thank you so much for your work here. I'd love if we can get this merged. Apple Calendar feeds use BYSETPOS when selecting a day of the month. And there is no direct equivalent for the negative values (starting from the end). So this would be super handy.

@pacso
Copy link
Collaborator

pacso commented Jan 5, 2024

@jorgemanrubia - Since nothing has moved on this PR for a long time ... it might be easiest if you can fork it, get the tests passing and then we can move forwards? I have struggled for time recently otherwise would have done this myself.

rgalanakis added a commit to webhookdb/webhookdb that referenced this pull request Jan 7, 2024
The RRULE parser does not support BYPOS,
so instead of 'every 2nd tuesday of the month'
you get 'every tuesday on the month'.

Pulls in the fork
[this PR](ice-cube-ruby/ice_cube#449)
comes from for the fix, and adds a test to assert it.
@Skulli
Copy link

Skulli commented Sep 11, 2024

Monthly/Yearly would be more than enough for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.