Figure out how to debug performance issues on private codebases

[codygman]

I'm mostly posting this because I had some potential problems on my work codebase whose ghcide process is sitting at 20.4GB of RES memory right now. I can't share the code, but I'm interested in helping fix this performance issue and not quite sure how.

The only thing I can currently think of is manually try and pare down a version of the codebase with the same number of modules, functions, perhaps even code body within functions, etc. Even then though, I'm not sure it will reproduce my issue.

Then I had an idea. We could have a "codebase shape" tool of some sort which tells the number of modules, constructors, and other parts of the GHC ast I have no idea about that matter to the performance of ghcide.

Then perhaps that could be extended to recreate codebases of that "shape".

That's probably a pretty complex tool though and my goal here was to answer: What is the simplest way to make a minimum reproduction of a large private codebase?

This would be simplest from the user perspective and I expect that commercial users with large codebases would have no problem submitting a file outputting the "shape" of their codebase in order to debug performance issues.

I'm curious what others with more experience with ghcide think of this idea and if there are simpler ways to accomplish this.

ghcide 2.0 has been awesome and super fast btw, outside of a few pathological (hopefully, time will tell) cases.

[codygman]

A rougher idea of the shape of our codebase can be given with cloc:

Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Haskell                       1198          12021           3157          78722

and we have 1105 modules per rg --glob="*.hs" -l "^module" | wc -l.

From just this information though, I think it's hard for ghcide devs to answer "why is my codebase and it's specific shape taking 22GB" and if that's perfectly fine or not.

[codygman]

I have a testing method that should be a good stress test for ghcide.

1. Kill ghcide and all editor windows in vscode
2. Expand all folders with Haskell files within the hie.yaml cradle in the vscode file explorer
3. Open htop or top and keep it visible
4. Open first file, notice the memory usage as soon as things become responsive like docs on hover
5. Hover over every symbol in that file, take note of memory usage at the end of file
6. Open the next file in that module
7. Repeat until you've went through every file in your codebase

I'm guessing it's by design this will use a bunch of ram, but maybe a future optimization could look something like only caching a certain number of modules in memory and discarding them once memory usage is "too great". Maybe ghcide already does something similar to this?

[pepeiborra]

Garbage collection is not implemented currently, so I can see how the testing method above would blow up memory usage since nothing is discarded.

I think that rather than an abstract notion of "shape", a concrete benchmark might be more useful. All you need to is find a public codebase (ideally a Hackage package) that has similar characteristics to the private one in terms of size, features, etc. Then you can add it to the existing benchmark suite, possibly extending the set of experiments as well if you find that they don't suffice.

We expect contributions to run the benchmark suite and attach the differential results (see haskell/ghcide#647), so adding your benchmark there will ensure that performance doesn't regress on your codebase, as well as motivate contributors to improve it.

[mpickering]

If you are using TemplateHaskell (at all) then memory usage will be a lot higher. I have a patch to make ghcide use object files rather than bytecode which makes this situation a lot better but someone will have to rebase and upstream it.

[codygman]

@mpickering is it the https://github.com/mpickering/ghcide/tree/wip/fix-th branch?

[unhammer]

87371a125acede1adb758ed6f86df293b7687f1f (https://github.com/mpickering/ghcide/tree/wip/fix-th) seems to be merged

[jneira]

@codygman did you have the opportunity of trying @pepeiborra suggestion or compare the perf of the th-fix branch (not sure if the improvement landed finally in master)?

[jneira]

I am gonna close the issue optimistically, @codygman please reopen or open a new one as you wish