Moving this project forward

[insraq]

Hi, I am the developer of Industry Idle on Steam and have been using greenworks for the Steamwork integration via @Armaldio 's prebuilt binaries.

I realize that the library itself hasn't received any updates for a while and lacks several newer Steamworks APIs. It seems that people at @greenheartgames have moved on to their next project that does not utilize Electron/NW.js anymore. However, greenworks library remains very useful for developers like me who are stuck with web stack :-D

I wonder what would be a good way to move this project forward. I have a private fork of greenworks that fixes the node-gyp build script and adds several new features for myself. I have manually built the binaries myself on Windows/Linux/Mac for the Electron version I am using, which is quite a hassle. It would be great to be able to utilize @Armaldio 's existing CI infrastructure - also I'd like to upstream these changes and share them with wider developer communities. However, it seems this repo is mostly in hibernate state.

I am not sure whether @greenheartgames would add more maintainers to this repo. Another option is to make a community fork. The goal is to have a healthy pool of maintainers who can actively merge PRs and maybe add new Steamworks APIs. I am interested to help maintain the project in the immediate future - as I need to support my game. But I do expect when I move on to other projects with different tech stacks (less likely in the short term, but who knows), I would be less active but hopefully by that time, there will be new volunteers who will help keep this active.

[Armaldio]

I'm also interested in participating in the developement
The C++ part is not my cup of tea, but I can still help

What really interest would be a nan -> napi migration
That would help so much with the prebuilds

[patrickklug]

Thanks for posting this! We'd certainly welcome new maintainers! You are right that we are no longer actively working in this sphere and we don't have in-house skills to develop greenworks, so actively maintaining it became a bit difficult. We also don't have any pressing needs as greenworks does what we wanted it to do but I realize that other projects have different set of needs and electron support seems flaky.

[insraq]

Thanks for the quick reply!

Electron support needs the developer's understanding of Electron's new "best" practice - because of the "context-awareness" issue (and security), greenworks should not run in the renderer process - and the renderer should communicate with it via IPC. Lots of old articles/documentation are outdated. But the addon itself works fine, at least for me.

@Armaldio I am glad you are interested as well! I don't like C++ either but we don't really have other options given Steamworks API is in C++. I am not a C++ expert - the language is very complex but I know enough C++ to get by.

I think currently on my priority list is (in high to low order)

• Fix broken build (done)
• Add missing Steamworks API (I am especially interested in the new ones for Steam Deck, like dynamic cloud sync)
• Add TypeScript definition to replace the documentation

The NAN to Node-Addon-API migration would be nice. But it requires a lot of effort and wouldn't provide direct benefit to the game developers - without a pressing need, it is less likely to be finished. We could have a new branch that focuses on that for people who are interested.

@patrickklug How do you suggest we move forward? There are two approaches:

• You can add me and @Armaldio as the maintainer to the project so that we can have write access and merge PR
• Or we can fork the project to a community namespace and go from there. This repo can be left as is or redirected to the new project.

I am fine with either way.

[Armaldio]

Would be great to also have an org so I can move https://github.com/ElectronForConstruct/greenworks-prebuilds (I can directly incorporate this one into the greenworks repo) and this one https://github.com/ElectronForConstruct/greenworks-prebuilds-website

I can handle the js -> ts part, the github actions and I can look into the napi migration

[insraq]

Would be great to also have an org so I can move https://github.com/ElectronForConstruct/greenworks-prebuilds (I can directly incorporate this one into the greenworks repo) and this one https://github.com/ElectronForConstruct/greenworks-prebuilds-website

I can handle the js -> ts part, the github actions and I can look into the napi migration

Great. Looks like you have already given it some thought. Would you want to take the lead and create a new org? (Something like SteamworksJS?) We could ask @patrickklug to transfer this repo under the new org, or a simple fork would do as well.

I am happy to help update the Steamworks API version (latest is 1.5.3) and add missing APIs. We do need a better doc as well. Much as I like to spend time on this, I do have an overdue game that needs to be finished 😄

[Armaldio]

I'm okay to help, but do not want the responsibility of managing the org, not enough bandwith for that unfortunately

Waiting for your decision @patrickklug

[ceifa]

Glad to see this movement happening, I have my own fork of greenworks too with some fixes and I recently was trying to create something new from scratch on C# https://github.com/ceifa/steamworks.js but was not going well. Count on me to push this new fork forward!

[insraq]

Glad to see this movement happening, I have my own fork of greenworks too with some fixes and I recently was trying to create something new from scratch on C# https://github.com/ceifa/steamworks.js but was not going well. Count on me to push this new fork forward!

Haha, I have gone thru this myself, here are directions I have tried:

• Write a C++ server that directly invokes Steamworks API and expose it via HTTP Server/WebSocket
• Use Node FFI

Neither really works well and I concluded that the best way is to write a native plugin (Nan or N-API) if I want to expose it to JavaScript side. Ideally I don't want to work with C++ (the language doesn't spark joy) but I have accepted that is the only reasonable way to go.

[Spacetech]

I'm also glad to see some movement here. I'm a developer for Wayward, an Electron based game on Steam.

About 4 years ago, I internally forked this repository in order to add some apis that we needed. I believe this repro wasn't very active at that time so I decided to work on it independently so I could iterate quicker.

A few changes I made:

• Added SteamNetworking apis - for multiplayer via ISteamNetworking & ISteamNetworkingMessages
• Added Get / Store Stat apis
• Added PlayTime tracking - via SteamUGC
• Added support for workshop files over 100MB and fixed some misc workshop related issues
• Added support for latest steam sdk version (1.5.3a)
• Upgraded the zlib dependency to that latest from the chromium source - this fixed lots of build warnings and zip related workshop issues
• Sort of made the module context-aware since Electron started complaining
• Last but not least, I converted the module to NAPI (node-addon api)

I published the fork. I had to clean the git history a little but it should still contain most of my commits, as unorganized as they look. I'm not happy with how it's organized but it works and I wanted to spend more time on my game than greenworks.

Specifically for the NAPI conversion, check out this commit. Similar changes could be make in this repro to move to NAPI.

I will reiterate that it is forked from an old version of this repro so it's missing some code refactors & functionality that was added afterwards. I still hope this can help you all out.

[Armaldio]

@patrickklug @insraq

@patrickklug How do you suggest we move forward? There are two approaches:

You can add me and @Armaldio as the maintainer to the project so that we can have write access and merge PR
Or we can fork the project to a community namespace and go from there. This repo can be left as is or redirected to the new
project.

After thinking about it, I think that would be better to create a separate org:
I don't want to maintain a project under "greenheartgames" as I don't work for them
I already know some projects that could join the "greenworks" org and without one, we can't unify
I'm gonna create it, fork greenworks and add you as admin

That doesn't mean we can't change our minds and delete this org if it's doesn't fit, but I'd like to set things up while I'm available

-> https://github.com/orgs/greenworksjs

[patrickklug]

@Armaldio @insraq I am divided on (but not against) the org change (would that mean we have to point references to the new org?) but I would certainly caution against rebranding the project away from greenworks. Not because I'm precious about the name but Steamworks is a name owned by Valve and so Steamworks.js would imply an official status which this isn't. I think greenworks is, by now, known enough, that it has some value to keep the name and possibly the repo too.

I'd like to discuss this a little more with @Armaldio and @insraq via email. Are there any other folks here that want to be actively involved as maintainers for greenworks going forward and want to be part of this conversation?

[insraq]

I am indifferent to either approach - keep the name, change the name, keep the org or create a new org - doesn't matter to me that much. My goal is to get this library into a state where I (as well as other developers) can use the latest Steamworks API in my game 😄 So I'd like to be pragmatic about it. Since @Armaldio feels strongly about a new org, let's go with that? I am fine with GreenworksJS as the org name and keep the greenworks name for the library.

Big thanks to @Spacetech for publishing that fork. My own fork is similar but minus the Node-Addon-API/Zlib/Context Awareness change. I would say that is a good foundation to work on - I assume the current Wayward game uses that fork and it works. As long as @Armaldio gives that a test with the CI, I am happy to kickstart work based on that branch.

I'm not happy with how it's organized but it works and I wanted to spend more time on my game than greenworks.

As a gamedev, I can relate to that. That's the motivation behind my original post. To provide the community with a Steamworks integration that:

• Works out of the box across platforms and Electron ABIs without C++ toolchain (@Armaldio 's prebuilt binaries)
• Reach more or less feature parity with the latest Steamworks C++ API. This requires a good foundation to build upon. Obviously, I cannot achieve this by myself - since I'd like to focus on my game as well. But the idea is that if we have a repo that actively merges PRs and makes it easier for people to upstream their changes, then over time we will have good coverage.

[ceifa]

I'd like to discuss this a little more with @Armaldio and @insraq via email. Are there any other folks here that want to be actively involved as maintainers for greenworks going forward and want to be part of this conversation?

I want to be part of this conversation, but why not discuss here? So we can keep the discussion in public for future reads.

[Gurigraphics]

Haha, I have gone thru this myself, here are directions I have tried:

• Write a C++ server that directly invokes Steamworks API and expose it via HTTP Server/WebSocket
• Use Node FFI

My current attempt was using C# Reflection and Websockets to call all functions

public static dynamic run(string _namespace, string _class, string _method, object[] parameterObject = null)
{
  Type type = Type.GetType(_namespace + "." + _class);
  MethodInfo methodInfo = type.GetMethod(_method);
  ParameterInfo[] parameterInfo = methodInfo.GetParameters();        
  return methodInfo.Invoke(null, null);
}

So I can call any function from any browser ou client server

var a = {
  mode_: "get",
  namespace_: "Steamworks",
  class_: "SteamClient",
  method_: "Init",    
  args_:[480],
  types_: "int"
}

websocket.send(JSON.stringify(a));

With this strategy, the difficult thing is to deal with the data types

object[] parameters = new object[] { (uint)480, true };
methodInfo.Invoke(null, parameters);    

Tested with Facepunch.Steamworks and Steamworks.NET

[patrickklug]

I'll be coming back to this next week. Just wanted to reach out internally to our previous maintainer and see where things are at.
But reading @insraq and @Armaldio's comments, sounds like the org would be a good move to bring all greenworks-related repo's under one roof.

[insraq]

@Gurigraphics Facepunch.Steamworks has already done one level of wrapping that makes Steamworks API behave more or less in C# convention, which should make it easier. The C++ API has inconsistent styles: some expect you to use the return value, some ask for a buffer to copy to and some ask for a pointer. It's very hard to automatically map them in a structural way - some manual work is neded, even with the C style header that is supposed to be "automatic" steam_api_flat.h

Also now apart from the hefty Electron runtime, you have to ship C# runtime as well, potentially for 3 platforms (Win/Mac/Linux). It also adds quite some overhead, if you want to use Steamworks Network API for exampke. C++ doesn't spark joy but I suppose that's the best option we have 😄

[Gurigraphics]

@insraq I also tried to use GodotSteam to avoid C++.
But also have this problem with types and methods to multiplayer missing.
GodotSteam/GodotSteam#232

Is there any advantage to using Chromium Embedded Framework (CEF), like Spotify use?
https://www.spotify.com/br/opensource/

[podrivo]

Just wanted to say I'm really glad to see this happening! Excited to see where this is headed!

[hokein]

Thanks for raising it. Really excited to see more volunteers care about the project and want to maintain it! (I wish I could have more bandwidth on the project, sorry).

The project has been years, and my focus has been significantly shifted and Greenworks doesn't have a business need for the new developments of greenworks as it already meets their requirements.

I like the direction where the discussion goes. Moving the official repo and other greenworks-related repos to a community org sounds a good start to me, I vote the name GreenworksJS (I think we need a LOGO, idea wanted!). Would like to hear thoughts folks from @greenheartgames since they were the major funder of the project.

Base on above discussions, we have a list, and they seem to be reasonable to me.

• Fix stale issues (e.g. updating Steamworks and other 3rd dependencies);
• Enable context-aware feature, following the Electron's best practice;
• Expose more steamworks APIs;
• Better CI support (covered by @Armaldio's awesome greenworks-prebuilt service);
• napi migration;

I'll try to get some bandwidth to help with the development (doing the code review, and merging PRs).

[Armaldio]

Hey all,

I'm coming back here to know about the progress of the issue.

What have you decided ?
Do you still need to discuss about some things ?

[patrickklug]

@Armaldio Yes, give me a bit more time. I'm currently investigating whether we can still continue to fund and support the development of greenworks.

My investigation doesn't have to stop or delay any actions on your sides though. The plan sounds pretty good thus far.

[Armaldio]

Sure.

My investigation doesn't have to stop or delay any actions on your sides though. The plan sounds pretty good thus far.

So you are OK to have a community fork ?
But still investagate if you have resources to put into this ?

[insraq]

@Armaldio I see you've already forked greenheartgames/greenworks to greenworksjs. Do you think that's a good base to work on? Or maybe Spacetech's fork is a better base since it is already using Node-Addon-API? (https://github.com/Spacetech/greenworks)

I was thinking about using the latter as the base. While you can investigate moving the CI to use that fork, I can start to bring that fork to feature parity to greenheartgames/greenworks (Maybe @Spacetech can give more pointers on what's missing) and add missing APIs.

If your CI has managed to spit out some binaries, I am happy to give them a test with my game. Let me know if you need anything from Steamworks side (in case you don't have access to that).

After greenworksjs/greenworks reaches a working state, we can kindly ask @greenheartgames to point this repo to greenworksjs/greenworks 😁

Does this sound like a good plan?

[Armaldio]

@Armaldio I see you've already forked greenheartgames/greenworks to greenworksjs. Do you think that's a good base to work on? Or maybe Spacetech's fork is a better base since it is already using Node-Addon-API? (Spacetech/greenworks)

I was just preparing things. Indeed if https://github.com/Spacetech/greenworks fork is better, let's use this one in the org as base.
You have the rights to delete and fork this one with the new org right ?

While you can investigate moving the CI to use that fork

The CI was separated and runned every day because I could not run it "on push"
My initial idea was to move the CI part inside the greenworks repo to be able to run only on push.

Maybe we should also start discussing about that directly inside the org repo once you've forked the right one ?

[insraq]

@Armaldio

I have forked Spacetech/greenworks to greenworksjs/greenworks - I had to rename your previous fork of this repo to greenworks-nan due to name conflict. I have added you and @Spacetech as maintainer to that repo and I will draft a plan there to get things started.

Everyone else who is interested in the initial work, feel free to head over to https://github.com/greenworksjs/greenworks
I'd say to move quickly for initial work, let's not enforce any branch/PR rule for now. After things are more or less stable, we can have a PR process for introducing changes.

[patrickklug]

So you are OK to have a community fork ? But still investagate if you have resources to put into this ?

Yes 👍

[patrickklug]

New community fork will be at https://github.com/greenworksjs/greenworks - I'll await until it's stable and then point to this going forward.

[Elanis]

Hi !

Seeing this discussion a bit late 😄 I'm glad to see that community takes over things to work on and improve this library.

I'm using this lib in 2 productions games on Steam Extortion and Alchemistry and am working on a 3rd one.

These games are using nw.js (+Node.js for the in-developement one, on serverside), so I'd be interested to participate and/or test things with you !

Not a C++ dev either, but feel free to hit me up if you feel I can help.