Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat apilinks.json generator #153

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

feat apilinks.json generator #153

wants to merge 4 commits into from

Conversation

flakey5
Copy link
Member

@flakey5 flakey5 commented Nov 29, 2024

Closes #152

Opening this as a draft currently to get feedback on the approach since it's a bit non-trivial relative to the other generators. The apilinks.json file maps things exported by modules to their source locations on Github.

Example:

{
  "SomeClass.prototype.var": "github.com/nodejs/node/tree/<hash>/lib/file.js#L100"
}

This means we need to parse the module's javascript source in addition to its markdown.

What the current approach does is doing:

  • Adds another loading & parsing step for the js source files
    • acorn is used for parsing the source files
    • This intakes .js files, so to run it you need to do pass in node/lib/*.js as the input
    • This is dependent on the Markdown source parsing since it uses the source_link metadata in the docs
  • Exposes the parsed js ast to other generators by adding the ast-js generator
  • api-links generator is based off of the ast-js result

@flakey5 flakey5 requested a review from a team as a code owner November 29, 2024 07:59
@flakey5 flakey5 marked this pull request as draft November 29, 2024 07:59
src/utils/git.mjs Fixed Show fixed Hide fixed
src/utils/git.mjs Fixed Show fixed Hide fixed
src/utils/git.mjs Fixed Show fixed Hide fixed
src/utils/git.mjs Fixed Show fixed Hide fixed
src/utils/git.mjs Outdated Show resolved Hide resolved
src/parser.mjs Outdated Show resolved Hide resolved
@ovflowd
Copy link
Member

ovflowd commented Dec 19, 2024

@flakey5 do you need review here? I think I forgot to review this PR!

@flakey5
Copy link
Member Author

flakey5 commented Dec 20, 2024

do you need review here

As far as the approach yes please

bin/cli.mjs Show resolved Hide resolved
bin/cli.mjs Outdated Show resolved Hide resolved
bin/cli.mjs Outdated Show resolved Hide resolved
src/generators.mjs Outdated Show resolved Hide resolved
src/generators.mjs Outdated Show resolved Hide resolved
src/loader.mjs Outdated Show resolved Hide resolved
src/parser.mjs Outdated Show resolved Hide resolved
src/utils/git.mjs Outdated Show resolved Hide resolved
Copy link
Member

@ovflowd ovflowd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approach in general sounds good; Just, some of these utilities are way too complex and big and the git manipulation scripts are possibly dangerous or at least an easy vulnerability hell.

@ovflowd
Copy link
Member

ovflowd commented Jan 6, 2025

Also, @flakey5 this https://github.com/syntax-tree/estree-util-visit is what I was referring to use to walk through the Nodes of the source code. Since acorn generates an AST that is compatible with estree.

Signed-off-by: flakey5 <[email protected]>
@flakey5 flakey5 marked this pull request as ready for review January 15, 2025 23:55
@ovflowd
Copy link
Member

ovflowd commented Jan 26, 2025

@flakey5 there are PR conflicts. Could you fix'em?

indirects: {},
};

if (lhs.object.name === 'exports') {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is such a big function. I'd rather have a dedicated file for each one of these "cases" or dedicated functions on a separate file, just to reduce the chunk of this function. Pretty much just thinking on reusability.

Then you could have a map:

import identifierHandler ...

const typeHandlers = {
  'Identifier: identifierHandler,
  ...,
};

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This applies for all these large function bodies here. You can create a sub directory. I think it becomes easier to read, easier to maintain. And if you have different functions for handling identifiers or expressions, etc. Simply make multiple exports for a file that handles everything related to a given type.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this applies to all the helper functions you've made on all these different files. Let's decrease the size of these functions.

Copy link
Member

@ovflowd ovflowd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far looking good. Left more comments, will review more once they are addressed-

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

urgent: feat: add apilinks.json generator
3 participants