Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Will a full or pruned node lose the ability to migrate receipts from the database to a static file? #13973

Closed
DevinByte opened this issue Jan 24, 2025 · 4 comments
Assignees
Labels
A-cli Related to the reth CLI A-pruning Related to pruning or full node C-docs An addition or correction to our documentation

Comments

@DevinByte
Copy link

Describe the change

just like issue: #10118

it seems by design, see the following code, crates/static-file/static-file/src/static_file_producer.rs

    /// Returns a static file targets at the provided finalized block numbers per segment.
    /// The target is determined by the check against highest `static_files` using
    /// [`reth_provider::providers::StaticFileProvider::get_highest_static_files`].
    pub fn get_static_file_targets(
        &self,
        finalized_block_numbers: HighestStaticFiles,
    ) -> ProviderResult<StaticFileTargets> {
        let highest_static_files = self.provider.static_file_provider().get_highest_static_files();

        let targets = StaticFileTargets {
            headers: finalized_block_numbers.headers.and_then(|finalized_block_number| {
                self.get_static_file_target(highest_static_files.headers, finalized_block_number)
            }),
            // StaticFile receipts only if they're not pruned according to the user configuration
            receipts: if self.prune_modes.receipts.is_none() &&
                self.prune_modes.receipts_log_filter.is_empty()
            {
                finalized_block_numbers.receipts.and_then(|finalized_block_number| {
                    self.get_static_file_target(
                        highest_static_files.receipts,
                        finalized_block_number,
                    )
                })
            } else {
                None
            },
            transactions: finalized_block_numbers.transactions.and_then(|finalized_block_number| {
                self.get_static_file_target(
                    highest_static_files.transactions,
                    finalized_block_number,
                )
            }),
        };

        trace!(
            target: "static_file",
            ?finalized_block_numbers,
            ?highest_static_files,
            ?targets,
            any = %targets.any(),
            "StaticFile targets"
        );

        Ok(targets)
    }

if any prune config is set, we will get a none receipts target:

            receipts: if self.prune_modes.receipts.is_none() &&
                self.prune_modes.receipts_log_filter.is_empty()
            {
                finalized_block_numbers.receipts.and_then(|finalized_block_number| {
                    self.get_static_file_target(
                        highest_static_files.receipts,
                        finalized_block_number,
                    )
                })
            } else {
                None
            },

Is my understanding correct?

Additional context

No response

@DevinByte DevinByte added C-docs An addition or correction to our documentation S-needs-triage This issue needs to be labelled labels Jan 24, 2025
@DevinByte
Copy link
Author

maybe add more doc content in https://reth.rs/run/pruning.html#basic-concepts

@emhane emhane added A-pruning Related to pruning or full node and removed S-needs-triage This issue needs to be labelled labels Jan 24, 2025
@emhane
Copy link
Member

emhane commented Jan 24, 2025

seems related #9784 @DevinByte , wdyt?

@DevinByte
Copy link
Author

wdyt?

yes, they are indeed the same type of problem。Or is this an issue with the default engine, which has already been resolved in NewEngine?

@emhane emhane added the A-cli Related to the reth CLI label Jan 28, 2025
@joshieDo
Copy link
Collaborator

joshieDo commented Feb 5, 2025

Will a full or pruned node lose the ability to migrate receipts from the database to a static file?

Yea, whenever there's a prune configuration for receipts, they are written to database, and remain there. Each specific static file range requires all data to be present and sequential.

  • Pruning by distance (aka gradually delete old receipts): deleting/truncating old receipts from a single static file is expensive, since they're at the head of the file. We could prune only files and not single receipts from files. (eg. delete the whole file of 0 -> 499_999 when its time to delete receipt 499_999, instead of slowling deleting receipts from it), but hasn't been worth it yet imo.
  • Pruning by logs is the most difficult one though. We rely on TxNumber to query receipts/transactions. However, this pruning type implies that receipts have gaps inbetween them and static files assume that they are sequential in order to query them.

@joshieDo joshieDo closed this as completed Feb 5, 2025
@github-project-automation github-project-automation bot moved this from Todo to Done in Reth Tracker Feb 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-cli Related to the reth CLI A-pruning Related to pruning or full node C-docs An addition or correction to our documentation
Projects
Archived in project
Development

No branches or pull requests

3 participants