automatic generation

[makarov-roman]

Hi,
how do you like the idea of scraping the Steam API and generating rules for the top N popular games? This could be the core of the rules, so we can resort to manual changes only for those we cannot generate — e.g. other launchers, mods, etc.

[makarov-roman]

I did a small PoC. Here is a sample of the data. Let me know if you're interested in this direction.
The way I see it - depending on how big the rule set should be, it can be an automatic job which regenerate the rules monthly. Or a manual script.
in current implementation the usage data is taken from steamspy (because they provide a convenient endpoint, can be replace with something else), and the Launch options are directly from steam's getProductInfo.

# 1142710: Total War: WARHAMMER III
## Wine
{ "name": "Warhammer3.exe", "type": "Game" }
{ "name": "launcher.exe", "type": "Game" }
{ "name": "twmm.exe", "type": "Game" }
## Native
{ "name": "TotalWarhammer3.sh", "type": "Game" }

# 508440: Totally Accurate Battle Simulator
## Wine
{ "name": "TotallyAccurateBattleSimulator.exe", "type": "Game" }

# 2113900: Touge Shakai
## Wine
{ "name": "Touge Shakai.exe", "type": "Game" }

# 2665680: Tower of Babel: Survivors of Chaos
## Wine
{ "name": "Tower Of Babel.exe", "type": "Game" }

# 2140510: Town of Salem 2
## Wine
{ "name": "TownOfSalem2.exe", "type": "Game" }

# 24010: Train Simulator Classic
## Wine
{ "name": "RailWorks64.exe", "type": "Game" }
{ "name": "RailWorksDX12_64.exe", "type": "Game" }
{ "name": "Railworks.exe", "type": "Game" }

# 1066780: Transport Fever 2
## Wine
{ "name": "TransportFever2.exe", "type": "Game" }
## Native
{ "name": "run.sh", "type": "Game" }

# 372000: Tree of Savior (English Ver.)
## Wine
{ "name": "tos.exe", "type": "Game" }

# 858820: Tribes of Midgard
## Wine
{ "name": "TOM.exe", "type": "Game" }

# 57690: Tropico 4
## Wine
{ "name": "Tropico4.exe", "type": "Game" }

minReviews1000TopByOnline1000.txt

[Aarrayy]

I mean it's possible but Steam limits API's usage either way so it will take a long time to do so.

[makarov-roman]

@Aarrayy I used steamspy to get a list of relevant titles(the data is not perfect, but decent), and then getProductInfo, which allows to supply list of titles, so every request handled 500 titles.
It took ~20mins to scrape 20k of the most popular titles (rate limit of 1 request every min), and around 1-2m to fetch launchOptions for them.
I'll provide you the code when I'm back home, it pretty simple

[Aarrayy]

Sure, i mean if it's reliable and doesn't increase maintenance, i'm all for it

[makarov-roman]

@Aarrayy https://github.com/makarov-roman/ananicy-rules-generator/tree/main
I commited db so data fetching can be skipped, not yet sure if it belongs to this repo, I have no preference here.

[lukaogadze]

@makarov-roman thank you for the work. it's useful tool. hate to say but... it can't be trusted.

# 1222700: A Way Out
## Wine
{ "name": "1222700?platform=steam&theme=awo", "type": "Game" }

actual game process is AWayOut.exe and AWayOut_friend.exe

# 251570: 7 Days to Die
## Wine
{ "name": "7dLauncher.exe", "type": "Game" }
## Native
{ "name": "7DaysToDie.sh", "type": "Game" }

actual game process is 7DaysToDie.exe / 7DaysToDie_EAC.exe and 7DaysToDie.x86_64

# 2183900: Warhammer 40,000: Space Marine 2
## Wine
{ "name": "MagnusClientRetail.exe", "type": "Game" }
{ "name": "MagnusEOSBootstrapperProfile.exe", "type": "Game" }
{ "name": "MagnusEOSBootstrapperRelease.exe", "type": "Game" }
{ "name": "MagnusEOSBootstrapperRetail.exe", "type": "Game" }
{ "name": "MagnusEosBootstrapperProfile.exe", "type": "Game" }
{ "name": "MagnusEosBootstrapperRelease.exe", "type": "Game" }
{ "name": "MagnusEosBootstrapperRetail.exe", "type": "Game" }
{ "name": "Warhammer 40000 Space Marine 2 - Retail.exe", "type": "Game" }
{ "name": "Warhammer 40000 Space Marine 2.exe", "type": "Game" }
{ "name": "launcher_pc_profile.bat", "type": "Game" }
{ "name": "launcher_pc_release.bat", "type": "Game" }
{ "name": "launcher_pc_retail.bat", "type": "Game" }
{ "name": "open.bat", "type": "Game" }

here actual game process is Warhammer 40000 Space Marine 2 - Retail.exe

# 550650: Black Squad
## Wine
{ "name": "BSLauncher.exe", "type": "Game" }

BSLauncher.exe is just a launcher. the actual game process is BlackSquadGame.exe.
problem is that it only scrapes data from here https://steamdb.info/app/550650/config/ which is not always displays correct game process name(s).

i will say again. it's useful tool for generating title and possible process name but it will require manual checking in steamdb Depots -> https://steamdb.info/depot/550651/ to make sure that it's "correct".

only way to be 100% correct is to install game but that's not realistic. again thank you. i will use your provided file. it will make my work more easy.

[makarov-roman]

@lukaogadze
Glad to hear that it's useful.
Can you clarify where do you see the process name in https://steamdb.info/depot/550651/ or you just checking what kind of binaries game files have?

[lukaogadze]

@lukaogadze Glad to hear that it's useful. Can you clarify where do you see the process name in https://steamdb.info/depot/550651/ or you just checking what kind of binaries game files have?

yes im just checking game binaries and comparing them to configuration page data. most of the time it's biggest exe file. but there will be exceptions like this. here anti cheat exe is bigger than game exe.

giving launcher and other processes "Game" types will make things messed up. for example game performance will suffer if you give game's background process same priority as actual game.

EDIT: "most of the time it's biggest exe file(s)"

[makarov-roman]

@lukaogadze thanks. I applied some filters based on your feedback.

here is a new version. I do not expect any feedback at this point.
lines 1-12793 are autogenerated rules
lines 12794-15870 - are manual entries (data extracted from this repo, mostly for me, since I'm trying to find patterns where can I guess better).
15870 - end - entries where no LaunchOptions left after filters, mostly EA entries with their ulr entries.

minReviews1000TopByOnline5000.txt

I think we can combine educated guess and manual verification. We can apply "the biggest executable" approach for autogenerated entries, which should improve accuracy. Once the title is verified, it can be marked as proven so automatic generations won't affect it.

[lukaogadze]

@makarov-roman thank you for the work. data that you provided was very useful. i checked around 300 game and added things that were "missing". it's slow process but it was way slower without your files. THIS is my last pr. i used your first file.
this is how i work:

1. i check game depots to find binaries
2. i make "guess" based on game engines / binary names / binary locations / binary size
   it's kinda hard to find linux binaries in depots.... but it's doable. it's a slow and time consuming process but we are making progress.

[makarov-roman]

@lukaogadze glad to hear.
I didn't find an easy a way to get the depots values without scraping public sources(which is something I want to avoid). You have to own the game to request this data from steam.

I'm still not sure if it makes sense to pursue the automation route, where we should be able to generate decently accurate rules automatically for the titles which are not handled manually or provide a tool/report for people who's interested to curate the list.

I'll experiment later

[lukaogadze]

can someone help me? :( i have this file. cleaned.txt
i want to remove entries from [cleaned.txt] that are in the repo.
example:
repo contains this:

# Nioh: Complete Edition https://store.steampowered.com/app/485510/Nioh_Complete_Edition/
{ "name": "nioh_launcher.exe", "type": "BG_CPUIO" }
{ "name": "nioh.exe", "type": "Game" }

cleaned.txt contains this:

# Nioh: Complete Edition
{ "name": "nioh.exe", "type": "Game" }
{ "name": "nioh_launcher.exe", "type": "Game" }

# Nobody Wants to Die
{ "name": "detnoir.exe", "type": "Game" }

i want everything to gone starting from "# Nioh: Complete Edition" to "# Nobody Wants to Die"
clanker gpt drove me crazy.

[makarov-roman]

@lukaogadze oh, I do have a script for it already, but I haven't committed it yet.
It's on my home pc, I'll be back from vacation in 3 weeks

[lukaogadze]

@makarov-roman do you extracted all data from the repo in second file?

[makarov-roman]

@makarov-roman do you extracted all data from the repo in second file?

I didn't check thoroughly, but I extracted 3k+ entries. Obviously only for titles which are on steam