what about private repos?

[einthusan]

I tried running my own local copy of this site, but got compile errors.

package appengine: unrecognized import path "appengine"
package appengine/datastore: unrecognized import path "appengine/datastore"
package appengine/urlfetch: unrecognized import path "appengine/urlfetch"

are appengine packages a must?
can we replace them with defined interface calls?

Would it work if I forked my own copy and made the changes myself? I didn't look into the code yet.

[CorgiMan]

The easiest thing to fix this is by installing app engine
https://cloud.google.com/appengine/downloads#Google_App_Engine_SDK_for_Go
and then to run the app with goapp serve (you can then access it through
your browser at localhost:8080)

If you don't want to use appengine you have to pull out the urlfetches
(which is appengine's own version of http.Get)
You have to remove the database part (or connect your own database).
Replace the appengine webserver part by your own.

The code is really hacky so I think your best bet is to install appengine ;)

I am not allowed to touch the code untill the judging phase of the
gophergala is over. Maybe I clean the code up after this.

Good luck

2015-01-29 23:42 GMT+01:00 Einthusan Vigneswaran [email protected]:

I tried running my own local copy of this site, but got compile errors.

package appengine: unrecognized import path "appengine"
package appengine/datastore: unrecognized import path "appengine/datastore"
package appengine/urlfetch: unrecognized import path "appengine/urlfetch"

are appengine packages a must?
can we replace them with defined interface calls?

Would it work if I forked my own copy and made the changes myself? I
didn't look into the code yet.

—
Reply to this email directly or view it on GitHub
#2.

[einthusan]

In that case i will wait for you to clean it up, and i think many others will benefit from removing the link to app engine.. however, if i do get desperate enough, I may mess around with a fork.

thanks for the awesome work!

[CorgiMan]

Thank you, it feels good to know that some people like it !

I didn't spent much time writing this so it probably makes no sense but it might help you.

If you want to change it for private repos, here are some tips:

The github crawler searches your github repo directory for go files and deeper directories.
(c Crawler) ParseFile(...) extracts the source from a specific file form the github url and puts it in the analyzer
(c Crawler) GetDirInfo(...) searches a github directory for all its go files, and deeper directories. It returns a list of strings for directory paths, and a list of strings for the go files it finds.

For a private repo I would leave out the whole github html scanning and just use os to find the files in your local file system.

Tear out all the db stuff and create a crawler to scan your file system once. Make the crawler global or something. Then, when you make a /view/.../.../.. request, find the package in cr.Analysis.Repo.Pkgs[..].
stuff that Package into a DBPackage. Stuff the DBPackage into a TemplateData. Execute the view.html template with this TemplateData.

The type conversion from Package to DBPackage is very badly designed due to the time constraints of the hackathon.

So something like this:

//Do this once, cr is a global var
cr = NewCrawler("user", "repo")
cr.Crawl(c)
cr.Analysis.ConstructGraph()

// do this for a view request, elt selects the package you want to view

p := DBPackage{Links: *cr.Analysis.Repo.Pkgs[elt].Links} 
// other fields of DBPackage don't matter too much I think

templateData := TemplateData{pkgs, p} 
// the pkgs should be a list of strings that is used for the left
// search bar, I think you could get away by setting it to nil

// execute the template with the templateData
if err := viewTemplate.Execute(w, templateData); err != nil {
    http.Error(w, err.Error(), http.StatusInternalServerError)
}

Summary:

• Change the ParseFile, and GetDirInfo functions in githubcrawler.go to scan your local filesystem.
• Do the scanning once and store globally in stead of in a db.
• Upon view request pick the correct package from cr.Analysis.Repo.Pkgs. Stuff its Links in DBPackage, stuff the DBPackage into TemplateData, execute the viewTemplate.

[einthusan]

thanks for the details. I have been building an internal closed source web framework in Go for about 1.5 year now, so I am pretty sure I'd be able to figure this out, if and when I get time.

Why were you using a database in the first place? Just out of curiosity.. if you were under time constraints, and assuming you were storing either fetched repos or using it like some sort of cache, isn't that time consuming to implement?

[CorgiMan]

• I thought the process of crawling the directories and parsing the files
  would take longer. It did on my machine.
• I didn't want to sent the entire repo as one response either, and I
  thought it was necessary to parse the entire repo to connect the functions.
• Pressure turns out to not be the best catalyst for logical thought

2015-01-31 3:03 GMT+01:00 Einthusan Vigneswaran [email protected]:

thanks for the details. I have been building an internal closed source web
framework in Go for about 1.5 year now, so I am pretty sure I'd be able to
figure this out, if and when I get time.

Why were you using a database in the first place? Just out of curiosity..
if you were under time constraints, and assuming you were storing either
fetched repos or using it like some sort of cache, isn't that time
consuming to implement?

—
Reply to this email directly or view it on GitHub
#2 (comment).

[einthusan]

:) regardless, you pulled off a very useful tool in the short time period. best of luck! :D

[CorgiMan]

Thank you