-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add search index #21
Merged
Merged
Add search index #21
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This ends up speeding up the partial country search by around 3/10s of a second on my old computer. The slow thing was creating the index on the fly. Loading it from JSON ends up being very fast.
This moves another search index to the cache. The goal here is to move all data to the cache and remove all data dependencies. I also made some changes to the rubocop config and how the CacheGenerator class is set up. The tests had to be updated as well.
- add docs for cached files - update gemspec to include cached files
apainintheneck
commented
Dec 13, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #14.
Overview
The main change here is to pre-compute all word to country mappings for performance reasons. This adds quite a bit of complexity to the code though because we have to make sure that these mappings stay up-to-date.
Changes
script/shared/cache_generator.rb
class to make it easier to generate cache files with documentation.script/generate_search_index.rb
to facilitate generating cached search indexes and checking to make sure they are up-to-date.cache.rb
module to facilitate getting values from the cache.generate_readme.rb
to newscript/
folder.atlasq.gemspec
).Benchmarks
The performance gains come from two main areas.
This even notably speeds up tests from around 8 seconds to around 4 seconds.
This is a measurement of the worst-case scenario before this change since it required creating a partial country name index on the fly.
The results would look like this.
For normal country matching the results are still significant.
The results should look like this.
Next Steps
It would be nice to not have to bundle all of the data dependencies with the program. These could easily be added to the cache reducing the number of runtime dependencies.