I'm getting an OOM using this software. What's going on?

[MuniuDev]

Hi,

I've been running Pl3xMap for a few years on my server and I really enjoy it!

Unfortunately, since 1.20.1 update I've been experiencing a constant memory leak that results in OoM termination of the server runtime. I've verified that removing Pl3xMap fixes the issue. I've also tested that using an external webserver (nginx) does not fix the issue.

Few parameters of my server:

MC version: 1.20.1
Paper version: 68
Pl3xMap version: 1.20.1-468 (with Pl3xMapClaims 1.20.1-30, I use GriefPrevention and WorldGuard)
4 CPUs, 8GB RAM (5GB allocated to server)

Plugins (besides Pl3xMap):
  AntiBookBan-1.3.jar
  AntiPopup-6.1.jar
  Chunky-1.3.92.jar
  CoreProtect-21.3.jar
  DoubleShulkerShells.jar
  DropHeads v3.7.13.jar
  EssentialsX-2.20.0.jar
  EssentialsXChat-2.20.0.jar
  EssentialsXSpawn-2.20.0.jar
  GriefPrevention 16.18.1.jar
  LuckPerms-Bukkit-5.4.40.jar
  Multiverse-Core-4.3.1.jar
  Multiverse-Inventories-4.2.2.jar
  Multiverse-Portals-4.2.1.jar
  OpenInv 4.3.1.jar
  ProtocolLib-5.0.0.jar
  QuickShop-5.1.2.2.jar
  SurvivalInvisiframes.jar
  TabList-bukkit-v5.7.3.jar
  Vault.jar
  WanderingTrades-1.8.1.jar
  gringotts-2.12.4.jar
  sleep-most-5.3.0.jar
  worldedit-bukkit-7.2.15.jar
  worldguard-bukkit-7.0.9-beta1.jar

I'm using Multiverse to hold a total of 5 dimensions (overworld, nether, end, overworld2, and creative realm (hidden from map))

Screenshot of memory usage at the server start:

And then after 4h:

And finally after about 7h:

I'm attaching my config:
config.zip

Thank you for the work and the support with this issue.

[iTemmy]

I am getting this as well, adjusted server to have 20GB of memory and was running fine then would suddenly spike and use all 20GB after about 5-7 minutes.

Have attached heap dump from when server started and just before server crashed.

server heap dumps.zip

[BillyGalbreath]

No, there is no memory leak.

Yes, Pl3xMap uses as much ram as it can. That's by design.

What you both are seeing is the server using more ram than the server has to give it. This is due to misconfigurations of the heap allocations on your part.

Let's take Temmy's example since it's got actual numbers I can point at. He has a server with 20GB of physical ram. He's telling the MC server it can use 18GB of that by setting 18432MB as his Xmx parameter in the startup script. This leaves little to no room for the OS to operate on or the JVM to breathe (the JVM comes with overhead).

Ok, so this is all fine and dandy, because he also told the JVM to start up with only 256MB of heap. this is will below the max limit, so it starts up and is fine. Until something starts needing more heap, then the JVM will grow that 256MB until it reaches the Xmx cap you set at 18GB. Except, the JVM's been crunching heavily on some numbers (in this case, Pl3xMap), so the JVM overhead is rather high. Very probably more than 2GB of overhead. And we still havent thought about the OS, or other programs running on the machine, etc etc. All of a sudden the server has run out of ram before it can even allocate all the heap space you told it it could have.

Boom. OOM.

See, look for yourself. The physical ram has run out, and the JVM has only allocated 16GB out of the 18GB it was allowed to grab.

You'll see the same OOM almost instantly if you were to follow Aikar's advice on his blog post about JVM flags where he states to make sure Xms is the same value as Xmx. This is important, not only for his flags to operate properly, but to ensure the heap can and does get everything you tell it it can get.

Another thing you can do is lessen the burden on the JVM/heap by lowering the amount of threads Pl3xMap can use (its in the config.yml file). This will result in Pl3xMap needing less ram at one time to do what it needs to do. Since ya'll look to have only 4 cores, I recommend lowering the thread count to 1 for Pl3xMap.

And yet another thing you could do to lessen the burden on your heap is to turn on both GC options in Pl3xMap's config file. This will make the GC run way more aggressively, cleaning out old, dead objects much faster that letting the JVM do it on it's own.

So yeah. TLDR; There's no memory leak. Do the 3 things I suggest and you should be fine.

[MuniuDev]

My server hosts only this single MC server for which I have 5GB (out of total 8GB, with 300MB used by linux) assigned according to Aikar flags (Xmx == Xms). I've already been using both GC flags, and yesterday switched to 1 rendering thread (from -1). The exact same thing happened. In the past I've also experimented with 4GB assigned to the mc server but this didn't solve the issue.

To be perfectly clear, my server always takes up a constant amount of RAM unless I switch the Pl3xMap on, at that point, the RAM slowly crawls up and eventually causes server termination. To me this is a clear memory leak.

Edit: No players have been playing on the server during that test.

[iTemmy]

I have a server that has 32GB of memory and basically only has the server running, this was running fine with an older version of the plugin then when I updated to 1.20 and the latest version it started getting out-of-memory errors and as @MuniuDev mentioned the server would idle at roughly 7GB then would slowly climb until it ran out of memory and would cause it to terminate due to the JVM running out of memory.

[matipoirierg]

Hi, I had this same issue, I'm running my server under a pterodactyl docker. The problem is that with pl3xmap the server starts increasing the overhead way higher than the xmx. 2 GB should be enough to cover overhead needs, but with plexmap my server was consuming an unreal amount of ram when my XMX was 11 GB. Here's an example:

The flags that I was using (I'm on a dedicated server with 32 gb of ram):

java -Xms11264M -Xmx11264M --add-modules=jdk.incubator.vector -XX:+UseG1GC -XX:+ParallelRefProcEnabled -XX:MaxGCPauseMillis=200 -XX:+UnlockExperimentalVMOptions -XX:+DisableExplicitGC -XX:+AlwaysPreTouch -XX:G1HeapWastePercent=5 -XX:G1MixedGCCountTarget=4 -XX:InitiatingHeapOccupancyPercent=15 -XX:G1MixedGCLiveThresholdPercent=90 -XX:G1RSetUpdatingPauseTimePercent=5 -XX:SurvivorRatio=32 -XX:+PerfDisableSharedMem -XX:MaxTenuringThreshold=1 -Dusing.aikars.flags=https://mcflags.emc.gs/ -Daikars.new.flags=true -XX:G1NewSizePercent=30 -XX:G1MaxNewSizePercent=40 -XX:G1HeapRegionSize=8M -XX:G1ReservePercent=20 -jar server.jar --nogui

Server heap usage:

Server total usage including overhead (I confirmed this was the actual total usage with free -h)

(It kept rising and eventually using my whole machine ram)

The point is that it doesn't matter how much ram you set to XMS and XMX, pl3xmap will start eating ram outside of those values, and if the container doesn't have a ram limit it will ending up eating the whole dedicated server ram. In the case the container have a limit your server will constantly crash due to OOM. I removed pl3xmap and the issue was gone, with the same flags that I put there now my server consumes a max of 13gb.