Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support running parsers XOR checks separately #1836

Open
gwerbin opened this issue Oct 25, 2024 · 0 comments
Open

Support running parsers XOR checks separately #1836

gwerbin opened this issue Oct 25, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@gwerbin
Copy link

gwerbin commented Oct 25, 2024

I want to assume that the output of my parser passes all checks (for use at runtime when loading data from an external source), to avoid running a lot of expensive and unnecessary checks at runtime. And in other situations, I want to be able to run all checks (either to validate input or during my test suite) but not run the parsers, assuming that the input data is already parsed.

As far as I can tell, that's not currently possible with Pandera. I would have to define 2 separate schemas, one with all my parsers and another with all my checks, which is verbose and error-prone (or requires a helper function to introspect and generate both from the same base schema).

I propose two new interfaces for this purpose: a .parse method which only runs parsers but not checks, and a .check method which only runs checks but not parsers.

The types would look something like this:

class DataFrameModel:
    def parse(self, df: DataFrame[Any], **config_kwargs: Any) -> DataFrame[Self]: ...
    def check(self, df: DataFrame[Self], **config_kwargs: Any) -> bool: ...

(The config_kwargs are from my other feature request here: #1837)

@gwerbin gwerbin added the enhancement New feature or request label Oct 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant