Gumtree parsing is slow

[Hareem-E-Sahar]

I'm parsing java and python source code using the gumtree docker image. I have about 25000 commits and it took about 40 hours to parse those files. With javalang I was able to parse in minutes. I'm doing something wrong or is gumtree slow because its more than just a parser.
Thanks

[jrfaller]

Hi Hareem,

To establish a more precise diagnostic it would be nice to know GumTree's version, which parser, and which matching algorithm you are using. However, you are right that parsing is usually the performance bottleneck, especially for languages where there is no hand-coded optimized parser.

Cheers!

[Hareem-E-Sahar]

Thanks for replying. I am currently using Gumtree version 3.0.0. Initially, I attempted to parse Python files directly using the JAR file with the following command:

./gumtree-3.0.0/bin/gumtree parse example.py
However, I encountered an error that pythonparser is misisng (in directory "/tmp"), although I installed it with pip.
Now I've switched to using the Docker image. Here is how I am currently running it:
sudo docker run --rm -v "$SOURCE_DIR"/:/data -p 4567:4567 gumtreediff/gumtree parse /data/"${file##*/}"
Do you have recommendations to improve the parsing speed, or could possibly suggest a more optimal setup? Additionally, if there are known performance differences between using the JAR file and the Docker image, insights on that would help. Thank you.

[jrfaller]

Hi!

I recommend you to use our latest version because we integrated two tree-sitter-based backends for Python (one native, should be the fastest, one using a Python wrapper, the downloadable jar contains only the native one), maybe they are faster. I don't really know what is the overhead of using Docker, but I think the best performance is obtained by running a jar directly.

Cheers!