RF455-150 Subject: [Bug] Gameserver crashes every 30-46 minutes with "fatal error: invalid runtime symbol table"

Game server RF Online: Dragonborn v0.5.1 high · filed by le 8d ago

Response Required

Hi,

I'm running your GoMMO bundle for RF Online (rf455) on Windows, and the gameserver keeps crashing every 30-46 minutes with a fatal runtime error. The manager auto-restarts it, but players lose all progress since the last save (level rollback).

Environment

OS: Windows (running on a dedicated server, not a dev machine)
Game: rf455 (RF Online: Dragonborn)
Database: SQLite (default config)
Manager: AUTO_RESTART=1 (so gameserver gets relaunched immediately after exit)
Settings: default, except DEBUG=0

Crash Symptoms

Gameserver runs fine for 30-46 minutes, then suddenly exits with exit code 2
No panic, no application-level error — it's a Go runtime fatal error
Windows Event Viewer shows no gameserver.exe crash record (only Edge Update crashes, unrelated)
Players get disconnected, and when they log back in, their progress is rolled back to the last save point (e.g. a player went from level 47 to 54, then crashed and rolled back to 47 — this has happened 7 times)

The Error

fatal error: invalid runtime symbol table

[recovers with a full goroutine stack dump, then exits]
[manager 03:37:48] exited: exit 2

Reproduction

It happens reliably when players are grinding mobs in the Turncoat area (map 24, around x=-4427, z=-3220). Specifically, when there's heavy monster killing (multiple kills per second, high EXP gains — e.g. Turncoat Black Knights, Turncoat Mercenery, giving ~4-6 billion EXP per kill).

I've seen it happen twice in the logs I have:

Crash at 03:37:48, after ~15 minutes of continuous grinding
Earlier crashes at 30-46 minute intervals, all under similar heavy load

What I've Tried

Checked Windows Event Viewer — no gameserver.exe crash events
Confirmed it's not a memory issue I can see (task manager doesn't show memory spiking before exit, but I haven't turned on pprof yet)
AUTO_RESTART=1 is working as intended (it does relaunch gameserver), but the sudden exit means in-progress saves are lost

Request

Could you look into this? The "invalid runtime symbol table" looks like a Go runtime issue, possibly triggered by something in the game loop under heavy load (maybe the monster kill handler, EXP calculation, or some data structure that's getting corrupted over time).

Happy to provide more logs, stack traces, or run debug builds if needed.

Thanks!

Reported on version 0.5.1

Replies 2

OSO AI Staff

Lv 17

Inspect OSO AI

4d ago

Reported: On Windows the rf455 gameserver exits every 30 to 46 minutes under heavy grinding with "fatal error: invalid runtime symbol table", and players lose their progress since the last save.

Found: That message comes from the Go runtime itself, when it walks a thread's stack and finds an address that belongs to no function. The server runs no unsafe memory code on that path and its database driver is pure Go, so we tried to reproduce it under load:

30 minutes on Windows with 64 bots killing monsters at about 3 per second at 100,000x experience, with logins, logouts and saves throughout, about three times your load. No crash.

25 minutes of the same load under Go's race detector. It found one place where two logins on the same account read each other's state unsynchronised, which is fixed in the next RF 4.55 update. That one cannot damage memory, so it does not explain the crash.

With no reproduction, something outside the server running inside its process (an antivirus, overlay or monitoring tool) or the machine's memory fits better than the server's code. The lines the runtime prints with the error name the function and address, and they decide which.

Also affected: The server saves a character when it logs out, not while it plays, which is why a crash rolls back the whole session.

Planned fix: We are adding a hot backup system in the next update: the server keeps each character's latest state on disk as it plays, and after a crash and restart it loads that state instead of the last database save, so a crash no longer rolls players back.

Needed:

The full crash output: the line before "fatal error" (it starts "runtime: invalid pc-encoded table f=") and at least the first goroutine block after it.

The server version, and the Windows version.

Any antivirus, overlay, or monitoring software running on that machine.

Whether it still crashes with the environment variable GODEBUG=asyncpreemptoff=1 set for the gameserver.

Whether a memory test of that machine, or running the server on another machine, changes anything.

👍 le

OSO AI Staff

Lv 17

Inspect OSO AI

1d ago

Fix: 0.6.0 carries two changes from looking into this crash. A race between two sessions reading the same account at login, found under a heavy kill load like the one described, is fixed. And the server now plays on a live backup of its game database, so a crash no longer rolls back the session.

Needed: Whether 0.6.0 still crashes under the same Turncoat grinding, and if it does, the full crash output from the line before "fatal error" through the first goroutine block.

👍 le