Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create Benchmarking Suite to measure performance #253

Open
basshelal opened this issue Jul 1, 2021 · 0 comments
Open

Create Benchmarking Suite to measure performance #253

basshelal opened this issue Jul 1, 2021 · 0 comments

Comments

@basshelal
Copy link
Contributor

In order to measure any possible performance improvements (or drops) during development (like before a PR merge or push) we need a reliable benchmarking suite.

This should test both native method calling performance AND library loading performance.

We should try to test as many scenarios as possible since we use a complicated set of strategies to bind native functions including:

  • x86asm and a64asm (used mostly for numeric functions), these end up becoming JVM native functions and behave differently performance-wise than others. This is loaded using AsmLibraryLoader
  • JVM bytecode which eventually will always (as far as I'm aware) end up calling to com.kenai.jffi.Invoker. This is also loaded using AsmLibraryLoader
  • java.lang.reflect.Proxys which is what we return and use when dealing with ReflectionLibraryLoader

All of these strategies have different method calling performance behaviors AND different loading performance (the latter being of particular importance to JRuby)

This suite should probably not be a part of the tests as we we don't want mvn test to always run them as they could be time consuming. But they are related, I would ideally prefer to have a benchmark function for every "User story" like a possible scenario or use case but that might be a bit too much.

We can probably use something like OpenJDK's JMH to help with this.

Personally, I also think this is an excellent opportunity to also improve and add to the current test suite but that's a separate topic.

We can even make the suite a part of the CI to measure changes between commits and hopefully not make us become too obsessed with minute performance changes 🤞

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant