Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Timestamps queried from InfluxDB 3 Core via JDBC Driver are inconsistent with those inserted #25983

Open
linghengqian opened this issue Feb 8, 2025 · 0 comments
Open

Timestamps queried from InfluxDB 3 Core via JDBC Driver are inconsistent with those inserted #25983

linghengqian opened this issue Feb 8, 2025 · 0 comments

Comments

@linghengqian
Copy link

linghengqian commented Feb 8, 2025 •
edited
Loading

Steps to reproduce:
List the minimal actions needed to reproduce the behaviour.

sdk install java 21.0.6-ms
git clone [email protected]:linghengqian/influxdb-3-core-jdbc-test.git
cd ./influxdb-3-core-jdbc-test/
sdk use java 21.0.6-ms
./mvnw -T 1C -Dtest=TimeDifferenceTest clean test
Click me to view the core logic of the unit test🥯🥨🍟🧂🥖🥚🍔🦪🍜🍘 
@Testcontainers
public class TimeDifferenceTest {
    private final Instant magicTime = Instant.now().minusSeconds(10);
    @Container
    private final GenericContainer<?> container = new GenericContainer<>("quay.io/influxdb/influxdb3-core:911ba92ab4133e75fe2a420e16ed9cb4cf32196f")
            .withCommand("serve --node-id local01 --object-store file --data-dir /home/influxdb3/.influxdb3")
            .withExposedPorts(8181);
    @Test
    void test() throws Exception {
        try (InfluxDBClient client = InfluxDBClient.getInstance(
                "http://" + container.getHost() + ":" + container.getMappedPort(8181),
                null,
                "mydb")) {
            writeData(client);
            queryDataByInfluxDbClient(client);
            queryDataByJdbcDriver();
        }
    }
    private void writeData(InfluxDBClient client) {
        Point point = Point.measurement("home")
                .setTag("location", "London")
                .setField("value", 30.01)
                .setTimestamp(magicTime);
        client.writePoint(point);
    }
    private void queryDataByInfluxDbClient(InfluxDBClient client) {
        try (Stream<PointValues> stream = client.queryPoints("select time,location,value from home order by time desc limit 10",
                QueryOptions.DEFAULTS)) {
            List<PointValues> list = stream.toList();
            assertThat(list.size(), is(1));
            PointValues p = list.getFirst();
            assertThat(p.getField("value", Double.class), is(30.01));
            assertThat(p.getTag("location"), is("London"));
            assertThat(p.getTimestamp(), is(NanosecondConverter.convert(magicTime, WritePrecision.NS)));
        }
    }
    private void queryDataByJdbcDriver() throws SQLException {
        HikariConfig hikariConfig = new HikariConfig();
        hikariConfig.setJdbcUrl("jdbc:arrow-flight-sql://" + container.getHost() + ":" + container.getMappedPort(8181) + "/?useEncryption=0&database=mydb");
        try (HikariDataSource hikariDataSource = new HikariDataSource(hikariConfig);
             Connection connection = hikariDataSource.getConnection()) {
            ResultSet resultSet = connection.createStatement().executeQuery("select time,location,value from home order by time desc limit 10");
            assertThat(resultSet.next(), is(true));
            assertThat(resultSet.getString("location"), is("London"));
            assertThat(resultSet.getString("value"), is("30.01"));
            assertThat(Timestamp.from(magicTime).getTime(), is(magicTime.toEpochMilli()));
            assertThat(resultSet.getString("time"), notNullValue());
            assertThat(resultSet.getTimestamp("time").getTime(), is(magicTime.toEpochMilli()));
        }
    }
}
  • Obviously, there is no concept of ZoneId or Offset for java.time.Instant, which is a fixed point on the continuous timeline. The timestamp of the fixed time is inserted through the InfluxDB Client, and then the same timestamp can be obtained by executing SQL to the InfluxDB Client. However, executing the same SQL through the Flight JDBC Driver will obtain a timestamp that is inconsistent with the original one. That is, this assertion will fail.
assertThat(resultSet.getTimestamp("time").getTime(), is(magicTime.toEpochMilli()));

Expected behaviour:
Describe what you expected to happen.

  • Timestamps queried from InfluxDB 3 Core via the JDBC driver are consistent with the inserted timestamps.

Actual behaviour:
Describe What actually happened.

  • Timestamps queried from InfluxDB 3 Core via JDBC Driver are inconsistent with those inserted.

Environment info:

  • Please provide the command you used to build the project, including any RUSTFLAGS.
    • It seems that this is not needed, I use the Docker Image quay.io/influxdb/influxdb3-core:911ba92ab4133e75fe2a420e16ed9cb4cf32196f directly in my unit test.
  • System info: Run uname -srm or similar and copy the output here (we want to know your OS, architecture etc).
    • Linux 5.15.167.4-microsoft-standard-WSL2 x86_64
  • If you're running IOx in a containerised environment then details about that would be helpful.
$ docker --version
Docker version 27.5.1, build 9f9e405
  • Other relevant environment details: disk info, hardware setup etc.
    • Considering that I execute unit tests under WSL, it’s hard to say there’s anything special about this. However, my WSL time zone environment is set to Asia/Shanghai.

Config:
Copy any non-default config values here or attach the full config as a gist or file.

  • It doesn't seem necessary.

Logs:
Include snippet of errors in logs or stack traces here.
Sometimes you can get useful information by running the program with the RUST_BACKTRACE=full environment variable.
Finally, the IOx server has a -vv for verbose logging.

  • The unit test will dynamically create a Docker container, so I think I only need to provide the log for the unit test.
Click me to view the complete Log🥯🥨🍟🧂🥖🥚🍔🦪🍜🍘 
$ ./mvnw -T 1C -Dtest=TimeDifferenceTest clean test
[INFO] Scanning for projects...
[INFO] 
[INFO] Using the MultiThreadedBuilder implementation with a thread count of 16
[INFO] 
[INFO] ----------< io.github.linghengqian:influxdb-3-core-jdbc-test >----------
[INFO] Building influxdb-3-core-jdbc-test 1.0-SNAPSHOT
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- clean:3.2.0:clean (default-clean) @ influxdb-3-core-jdbc-test ---
[INFO] Deleting /home/linghengqian/TwinklingLiftWorks/git/public/influxdb-3-core-jdbc-test/target
[INFO] 
[INFO] --- resources:3.3.1:resources (default-resources) @ influxdb-3-core-jdbc-test ---
[INFO] skip non existing resourceDirectory /home/linghengqian/TwinklingLiftWorks/git/public/influxdb-3-core-jdbc-test/src/main/resources
[INFO] 
[INFO] --- compiler:3.13.0:compile (default-compile) @ influxdb-3-core-jdbc-test ---
[INFO] No sources to compile
[INFO] 
[INFO] --- resources:3.3.1:testResources (default-testResources) @ influxdb-3-core-jdbc-test ---
[INFO] skip non existing resourceDirectory /home/linghengqian/TwinklingLiftWorks/git/public/influxdb-3-core-jdbc-test/src/test/resources
[INFO] 
[INFO] --- compiler:3.13.0:testCompile (default-testCompile) @ influxdb-3-core-jdbc-test ---
[INFO] Recompiling the module because of changed source code.
[INFO] Compiling 7 source files with javac [debug target 21] to target/test-classes
[INFO] 
[INFO] --- surefire:3.5.2:test (default-test) @ influxdb-3-core-jdbc-test ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running io.github.linghengqian.TimeDifferenceTest
SLF4J(W): No SLF4J providers were found.
SLF4J(W): Defaulting to no-operation (NOP) logger implementation
SLF4J(W): See https://www.slf4j.org/codes.html#noProviders for further details.
2月 08, 2025 11:51:56 上午 org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.memory.BaseAllocator <clinit>
信息: Debug mode disabled. Enable with the VM option -Darrow.memory.debug.allocator=true.
2月 08, 2025 11:51:56 上午 org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.memory.DefaultAllocationManagerOption getDefaultAllocationManagerFactory
信息: allocation manager type not specified, using netty as the default type
2月 08, 2025 11:51:56 上午 org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.memory.CheckAllocator reportResult
信息: Using DefaultAllocationManager at memory/netty/DefaultAllocationManagerFactory.class
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 3.760 s <<< FAILURE! -- in io.github.linghengqian.TimeDifferenceTest
[ERROR] io.github.linghengqian.TimeDifferenceTest.test -- Time elapsed: 3.696 s <<< FAILURE!
java.lang.AssertionError: 

Expected: is <1738986704151L>
     but: was <1738929104151L>
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8)
        at io.github.linghengqian.TimeDifferenceTest.queryDataByJdbcDriver(TimeDifferenceTest.java:83)
        at io.github.linghengqian.TimeDifferenceTest.test(TimeDifferenceTest.java:47)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)

[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   TimeDifferenceTest.test:47->queryDataByJdbcDriver:83 
Expected: is <1738986704151L>
     but: was <1738929104151L>
[INFO] 
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  7.060 s (Wall Clock)
[INFO] Finished at: 2025-02-08T11:51:58+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.5.2:test (default-test) on project influxdb-3-core-jdbc-test: There are test failures.
[ERROR] 
[ERROR] See /home/linghengqian/TwinklingLiftWorks/git/public/influxdb-3-core-jdbc-test/target/surefire-reports for the individual test results.
[ERROR] See dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@linghengqian