Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache Data and Remove Most Runtime Dependencies #24

Merged
merged 3 commits into from
Dec 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,14 @@ source "https://rubygems.org"
# Specify your gem's dependencies in atlasq.gemspec
gemspec

gem "rake", "~> 13.0"
gem "rake"

gem "minitest", "~> 5.0"
gem "minitest"

gem "rubocop", "~> 1.21"
gem "rubocop"
gem "rubocop-minitest"
gem "rubocop-rake"

gem "countries"
gem "iso-639"
gem "money"
12 changes: 6 additions & 6 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,6 @@ PATH
remote: .
specs:
atlasq (0.2.0)
countries (~> 5.7)
iso-639 (~> 0.3)
money (~> 6.9)
tty-pager (~> 0.14)

GEM
Expand Down Expand Up @@ -67,9 +64,12 @@ PLATFORMS

DEPENDENCIES
atlasq!
minitest (~> 5.0)
rake (~> 13.0)
rubocop (~> 1.21)
countries
iso-639
minitest
money
rake
rubocop
rubocop-minitest
rubocop-rake

Expand Down
39 changes: 18 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,13 +100,12 @@ $ atlasq --country 418
* Country: The Lao People's Democratic Republic
* * * * * * * * * * * * * * * * * * * * * * * * *
(🇱🇦 | 418 | LA | LAO | Lao People's Democratic Republic)
| Search Term: 418
| Languages: Lao
| Nationality: Laotian
| Region: South-Eastern Asia
| Continent: Asia
| Currency: ₭ Lao Kip
|________________________________________
| Languages: Lao
| Nationality: Laotian
| Region: South-Eastern Asia
| Continent: Asia
| Currency: ₭ Lao Kip
|________________________________________

```

Expand All @@ -116,13 +115,12 @@ $ atlasq --country AM
* Country: The Republic of Armenia
* * * * * * * * * * * * * * * * * * *
(🇦🇲 | 051 | AM | ARM | Armenia)
| Search Term: AM
| Languages: Armenian / Russian
| Nationality: Armenian
| Region: Western Asia
| Continent: Asia
| Currency: դր. Armenian Dram
|________________________________________
| Languages: Armenian / Russian
| Nationality: Armenian
| Region: Western Asia
| Continent: Asia
| Currency: դր. Armenian Dram
|________________________________________

```

Expand All @@ -132,13 +130,12 @@ $ atlasq --country honduras
* Country: The Republic of Honduras
* * * * * * * * * * * * * * * * * * *
(🇭🇳 | 340 | HN | HND | Honduras)
| Search Term: honduras
| Languages: Spanish; Castilian
| Nationality: Honduran
| Region: Central America
| Continent: North America
| Currency: L Honduran Lempira
|________________________________________
| Languages: Spanish; Castilian
| Nationality: Honduran
| Region: Central America
| Continent: North America
| Currency: L Honduran Lempira
|________________________________________

```

Expand Down
6 changes: 5 additions & 1 deletion Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ require "rubocop/rake_task"

RuboCop::RakeTask.new

task default: %i[test lint]
task default: %i[lint test]

desc "Shortcut for `rake rubocop`"
task lint: :rubocop
Expand Down Expand Up @@ -44,10 +44,14 @@ namespace "cache" do
desc "Check if the cache needs to be regenerated"
task :outdated do
sh "bundle exec ruby script/generate_search_index.rb outdated"
sh "bundle exec ruby script/generate_formatted_output.rb outdated"
sh "bundle exec ruby script/generate_list.rb outdated"
end

desc "Regenerate the cache"
task :generate do
sh "bundle exec ruby script/generate_search_index.rb generate"
sh "bundle exec ruby script/generate_formatted_output.rb generate"
sh "bundle exec ruby script/generate_list.rb generate"
end
end
3 changes: 0 additions & 3 deletions atlasq.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,5 @@ Gem::Specification.new do |spec|
spec.bindir = "exe"
spec.executables = ["atlasq"]

spec.add_dependency "countries", "~> 5.7"
spec.add_dependency "iso-639", "~> 0.3"
spec.add_dependency "money", "~> 6.9"
spec.add_dependency "tty-pager", "~> 0.14"
end
8 changes: 8 additions & 0 deletions cache/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,14 @@ Note: Sample data can be found in the `README.md` file in each subdirectory.

As the name would suggest, this generates a bunch of search indexes that are basic JSON files with string to string mappings (no nested nonsense). These are used primarily to speed up partial matches though pre-computing things also means we don't have to pull in the internationalization libraries for 90+ languages which also makes a difference.

### script/generate_formatted_output.rb

This is where we generate the formatted output so that we don't need to include runtime dependencies to load country, currency and region information.

### script/generate_list.rb

This script generates a few simple lists of countries and regions.

## Reading

Reading from the cached files is quite easy. Just use the `Atlas::Cache` module to load the file using the namespace and file name. Each file gets lazy loaded the first time it's referenced and then memoized. Specified in `lib/atlasq/cache.rb`.
Expand Down
119 changes: 119 additions & 0 deletions cache/formatted_output/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# Cache: formatted_output

---

## Item: formatted_output/one_line_country.json

### Content Sample
Sample of the first 20 pretty printed lines of the file.

```
{
"ad": "(🇦🇩 | 020 | AD | AND | Andorra)",
"ae": "(🇦🇪 | 784 | AE | ARE | United Arab Emirates)",
"af": "(🇦🇫 | 004 | AF | AFG | Afghanistan)",
"ag": "(🇦🇬 | 028 | AG | ATG | Antigua and Barbuda)",
"ai": "(🇦🇮 | 660 | AI | AIA | Anguilla)",
"al": "(🇦🇱 | 008 | AL | ALB | Albania)",
"am": "(🇦🇲 | 051 | AM | ARM | Armenia)",
"ao": "(🇦🇴 | 024 | AO | AGO | Angola)",
"aq": "(🇦🇶 | 010 | AQ | ATA | Antarctica)",
"ar": "(🇦🇷 | 032 | AR | ARG | Argentina)",
"as": "(🇦🇸 | 016 | AS | ASM | American Samoa)",
"at": "(🇦🇹 | 040 | AT | AUT | Austria)",
"au": "(🇦🇺 | 036 | AU | AUS | Australia)",
"aw": "(🇦🇼 | 533 | AW | ABW | Aruba)",
"ax": "(🇦🇽 | 248 | AX | ALA | Åland Islands)",
"az": "(🇦🇿 | 031 | AZ | AZE | Azerbaijan)",
"ba": "(🇧🇦 | 070 | BA | BIH | Bosnia and Herzegovina)",
"bb": "(🇧🇧 | 052 | BB | BRB | Barbados)",
"bd": "(🇧🇩 | 050 | BD | BGD | Bangladesh)",
...
```

## Item: formatted_output/country_title.json

### Content Sample
Sample of the first 20 pretty printed lines of the file.

```
{
"ad": "Country: The Principality of Andorra",
"ae": "Country: The United Arab Emirates",
"af": "Country: The Islamic Republic of Afghanistan",
"ag": "Country: Antigua and Barbuda",
"ai": "Country: Anguilla",
"al": "Country: The Republic of Albania",
"am": "Country: The Republic of Armenia",
"ao": "Country: The Republic of Angola",
"aq": "Country: Antarctica",
"ar": "Country: The Argentine Republic",
"as": "Country: The Territory of American Samoa",
"at": "Country: The Republic of Austria",
"au": "Country: The Commonwealth of Australia",
"aw": "Country: Aruba",
"ax": "Country: Åland",
"az": "Country: The Republic of Azerbaijan",
"ba": "Country: Bosnia and Herzegovina",
"bb": "Country: Barbados",
"bd": "Country: The People's Republic of Bangladesh",
...
```

## Item: formatted_output/multiline_country.json

### Content Sample
Sample of the first 20 pretty printed lines of the file.

```
{
"ad": " | Languages: Catalan; Valencian\n | Nationality: Andorran\n | Region: Southern Europe\n | Continent: Europe\n | Currency: € Euro\n |________________________________________",
"ae": " | Languages: Arabic\n | Nationality: Emirian\n | Region: Western Asia\n | Continent: Asia\n | Currency: د.إ United Arab Emirates Dirham\n |________________________________________",
"af": " | Languages: Pushto; Pashto / Uzbek / Turkmen\n | Nationality: Afghan\n | Region: Southern Asia\n | Continent: Asia\n | Currency: ؋ Afghan Afghani\n |________________________________________",
"ag": " | Languages: English\n | Nationality: Antiguan, Barbudan\n | Region: Caribbean\n | Continent: North America\n | Currency: $ East Caribbean Dollar\n |________________________________________",
"ai": " | Languages: English\n | Nationality: Anguillian\n | Region: Caribbean\n | Continent: North America\n | Currency: $ East Caribbean Dollar\n |________________________________________",
"al": " | Languages: Albanian\n | Nationality: Albanian\n | Region: Southern Europe\n | Continent: Europe\n | Currency: L Albanian Lek\n |________________________________________",
"am": " | Languages: Armenian / Russian\n | Nationality: Armenian\n | Region: Western Asia\n | Continent: Asia\n | Currency: դր. Armenian Dram\n |________________________________________",
"ao": " | Languages: Portuguese\n | Nationality: Angolan\n | Region: Middle Africa\n | Continent: Africa\n | Currency: Kz Angolan Kwanza\n |________________________________________",
"aq": " | Continent: Antarctica\n | Currency: $ United States Dollar\n |________________________________________",
"ar": " | Languages: Spanish; Castilian / Guarani\n | Nationality: Argentinean\n | Region: South America\n | Continent: South America\n | Currency: $ Argentine Peso\n |________________________________________",
"as": " | Languages: English / Samoan\n | Nationality: American Samoan\n | Region: Polynesia\n | Continent: Australia\n | Currency: $ United States Dollar\n |________________________________________",
"at": " | Languages: German\n | Nationality: Austrian\n | Region: Western Europe\n | Continent: Europe\n | Currency: € Euro\n |________________________________________",
"au": " | Languages: English\n | Nationality: Australian\n | Region: Australia and New Zealand\n | Continent: Australia\n | Currency: $ Australian Dollar\n |________________________________________",
"aw": " | Languages: Dutch; Flemish\n | Nationality: Aruban\n | Region: Caribbean\n | Continent: North America\n | Currency: ƒ Aruban Florin\n |________________________________________",
"ax": " | Languages: Swedish\n | Nationality: Swedish\n | Region: Northern Europe\n | Continent: Europe\n | Currency: € Euro\n |________________________________________",
"az": " | Languages: Azerbaijani / Armenian\n | Nationality: Azerbaijani\n | Region: Western Asia\n | Continent: Asia\n | Currency: ₼ Azerbaijani Manat\n |________________________________________",
"ba": " | Languages: Bosnian / Croatian / Serbian\n | Nationality: Bosnian, Herzegovinian\n | Region: Southern Europe\n | Continent: Europe\n | Currency: КМ Bosnia and Herzegovina Convertible Mark\n |________________________________________",
"bb": " | Languages: English\n | Nationality: Barbadian\n | Region: Caribbean\n | Continent: North America\n | Currency: $ Barbadian Dollar\n |________________________________________",
"bd": " | Languages: Bengali\n | Nationality: Bangladeshi\n | Region: Southern Asia\n | Continent: Asia\n | Currency: ৳ Bangladeshi Taka\n |________________________________________",
...
```

## Item: formatted_output/one_line_currency.json

### Content Sample
Sample of the first 20 pretty printed lines of the file.

```
{
"eur": "[EUR] € Euro",
"aed": "[AED] د.إ United Arab Emirates Dirham",
"afn": "[AFN] ؋ Afghan Afghani",
"xcd": "[XCD] $ East Caribbean Dollar",
"all": "[ALL] L Albanian Lek",
"amd": "[AMD] դր. Armenian Dram",
"aoa": "[AOA] Kz Angolan Kwanza",
"usd": "[USD] $ United States Dollar",
"ars": "[ARS] $ Argentine Peso",
"aud": "[AUD] $ Australian Dollar",
"awg": "[AWG] ƒ Aruban Florin",
"azn": "[AZN] ₼ Azerbaijani Manat",
"bam": "[BAM] КМ Bosnia and Herzegovina Convertible Mark",
"bbd": "[BBD] $ Barbadian Dollar",
"bdt": "[BDT] ৳ Bangladeshi Taka",
"xof": "[XOF] Fr West African Cfa Franc",
"bgn": "[BGN] лв. Bulgarian Lev",
"bhd": "[BHD] د.ب Bahraini Dinar",
"bif": "[BIF] Fr Burundian Franc",
...
```
Loading