Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Non-deterministic NPE in datetimeconvert #14837

Open
dang-stripe opened this issue Jan 17, 2025 · 3 comments
Open

Non-deterministic NPE in datetimeconvert #14837

dang-stripe opened this issue Jan 17, 2025 · 3 comments

Comments

@dang-stripe
Copy link
Contributor

We're running into this error non-deterministically on one of our queries which points to https://github.com/apache/pinot/blob/master/pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/BinaryOperatorTransformFunction.java#L304.

Error Code: 200

QueryExecutionError:
java.lang.NullPointerException: Cannot read field "value" because "anotherString" is null
	at java.base/java.lang.String.compareTo(String.java:2042)
	at org.apache.pinot.core.operator.transform.function.BinaryOperatorTransformFunction.fillResultString(BinaryOperatorTransformFunction.java:304)
	at org.apache.pinot.core.operator.transform.function.BinaryOperatorTransformFunction.fillResultArray(BinaryOperatorTransformFunction.java:134)
	at org.apache.pinot.core.operator.transform.function.BinaryOperatorTransformFunction.transformToIntValuesSV(BinaryOperatorTransformFunction.java:110)

We've narrowed it down to this datetimeconvert in the where clause. created and t_created are LONG column types and have no null values.

Example query.

SET enableNullHandling = true;

SELECT
    date_trunc('day', t_created, 'milliseconds', 'UTC', 'seconds') AS ts,
    t_type AS t_type,
    COUNT(*) AS ct
FROM
    example_table
WHERE
    ...
    CASE
        WHEN datetimeconvert(t_created, '1:MILLISECONDS:EPOCH', '1:DAYS:SIMPLE_DATE_FORMAT:yyyy-MM-dd', '1:DAYS') >= '2024-12-11'
        AND datetimeconvert(t_created, '1:MILLISECONDS:EPOCH', '1:DAYS:SIMPLE_DATE_FORMAT:yyyy-MM-dd', '1:DAYS') <= '2024-12-19'
        THEN floor((created - t_created) / 86400000) <= 15
        ELSE true
    END
GROUP BY 1, 2
ORDER BY 1, 2 ASC

When we switch it to the literal time values, it works fine.

    AND CASE
        WHEN t_created >= 1733875200000  -- 2024-12-11
         AND t_created < 1734566400000   -- 2024-12-19
        THEN floor((created - t_created) / 86400000) <= 15
        ELSE true
    END

Any ideas what might be happening here?

cc @Jackie-Jiang

@Jackie-Jiang
Copy link
Contributor

@yashmayya Can you help take a look?

@yashmayya
Copy link
Collaborator

@Jackie-Jiang @bziobrowski this looks like it could potentially be related to the changes from #14298?

@bziobrowski
Copy link
Contributor

bziobrowski commented Jan 27, 2025

I couldn't reproduce the error. The right hand side mentioned in the exception is always a string literal - either '2024-12-11' or '2024-12-19'.
@dang-stripe Please provide the following data:

  • pinot version
  • engine type used for querying - single or multi-stage
  • table schema/config
  • full stack trace
  • full query (I suspect the failing part belongs to other predicate)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants