[Release] Some implementations are not removing old Release Candidates and old Releases from dist.apache.org

[raulcd]

Describe the enhancement requested

Old Release Candidates and old releases should be removed from https://dist.apache.org/repos/dist/dev/arrow/ and https://dist.apache.org/repos/dist/release/arrow/ respectively.
We remove Arrow RC(s) and Arrow's old releases when we execute dev/release/post-09-remove-old-artifacts.sh but we do seem to have a lot of old Rust RCs, datafusion RCs and objectstore RCs. We also have old nanoarrow and datafusion releases.

Component(s)

Release

[kou]

apache/arrow-rs fixed this: apache/arrow-rs#7060

[kou]

We can remove "apache-arrow-datafusion-*" from https://dist.apache.org/repos/dist/dev/arrow/ because Apache DataFusion is a top level project not a subproject of Arrow now.

[raulcd]

should we just run the following manually? I can do if we are ok

$ cat remove_datafusion_rcs.sh 
echo "Remove datafusion RCs"
dev_base_url=https://dist.apache.org/repos/dist/dev/arrow
old_rcs=$(
  svn ls ${dev_base_url}/ | \
  grep -E '^apache-arrow-datafusion-' | \
  sort --version-sort
)
for old_rc in $old_rcs; do
  echo "Remove RC: ${old_rc}"
  svn \
    delete \
    -m "Remove old Apache Arrow Datafusion RC: ${old_rc}" \
    ${dev_base_url}/${old_rc}
done

BTW this will list:

apache-arrow-datafusion-36.0.0-rc1/
apache-arrow-datafusion-36.0.1-rc1/
apache-arrow-datafusion-37.0.0-rc1/
apache-arrow-datafusion-37.0.0-rc2/
apache-arrow-datafusion-37.1.0-rc1/
apache-arrow-datafusion-37.1.0-rc2/
apache-arrow-datafusion-python-36.0.0-rc1/

[kou]

Yes, please.

[raulcd]

ok, those have been removed now:

./remove_datafusion.sh 
Remove datafusion RCs
Remove RC: apache-arrow-datafusion-36.0.0-rc1/
Committing transaction...
Committed revision 74767.
Remove RC: apache-arrow-datafusion-36.0.1-rc1/
Committing transaction...
Committed revision 74768.
Remove RC: apache-arrow-datafusion-37.0.0-rc1/
Committing transaction...
Committed revision 74769.
Remove RC: apache-arrow-datafusion-37.0.0-rc2/
Committing transaction...
Committed revision 74770.
Remove RC: apache-arrow-datafusion-37.1.0-rc1/
Committing transaction...
Committed revision 74771.
Remove RC: apache-arrow-datafusion-37.1.0-rc2/
Committing transaction...
Committed revision 74772.
Remove RC: apache-arrow-datafusion-python-36.0.0-rc1/
Committing transaction...
Committed revision 74773.

[raulcd]

The apache-arrow-object-store-rs-*-rc should be removed once a new release of object store is done as per: apache/arrow-rs#7060

I think the only one missing is for nanoarrow to remove old releases from https://dist.apache.org/repos/dist/release/arrow/ as part of their release.

[kou]

Right.

We can remove https://dist.apache.org/repos/dist/release/arrow/arrow-ballista-0.12.0/ too because the latest Ballista release exists at https://dist.apache.org/repos/dist/release/datafusion/ .

[raulcd]

Done, ballista removed from arrow releases:

$ ./remove_ballista.sh
Remove ballista releases
Remove Release: arrow-ballista-0.12.0/
Committing transaction...
Committed revision 74778.

[kou]

Thanks!!!