32 bit / 64 bit save compatibility - ADOM & Ultimate ADOM Forums

11-17-2012 06:41 PM

fredmn

Junior Member

I agree, and saves compatibles between mac, win and linux would be nice !

Reply

12-07-2012 04:45 AM

adom-admin adom-admin is offline

The Creator

Sorry, the amount of differences in details between the various compilers is bewildering. We do not intend to support this. Quite to the contrary we plan to add more specific game file information to the saved game to refuse games from incompatible platforms.

Reply

12-07-2012 08:31 AM

Grey

Ancient Member

Aw, that's an immense shame, as it would put an end to the popular community games that involve swapping files between computers. Is there no way this could be avoided? I don't see what benefit it adds to the game, and it would have a negative impact on the community.

Reply

12-07-2012 02:10 PM

Mobius

Ancient Member

Knowing the myriad C compilers out there, it is not a giant surprise Das (Der?) Creator chooses to spend his time coding features, rather than the mountain of pain that could be involved here.

I've don't the binary compatibility dance, and it's onerous. Byte order, structure alignment, structure padding, floating point format, etc.

Do we prefer Mist Elves or Universal Save Game format? (and which is more fun to code, from Herr Constructur's POV)
I'd do it for free for him, but that would involve being massively spoiled.

Reply

12-07-2012 03:45 PM

Al-Khwarizmi Al-Khwarizmi is offline

Ancient Member

Isn't there any portable C libraries to save structs to JSON, XML or a similar standard format? I do prefer mist elves to universal saves, but there should be a way of tackling the compiler hell without actually handling each specific case...

Reply

12-08-2012 12:26 AM

Grey

Ancient Member

And there's no reason not to ask for both mist elves and universal saves.

Honestly, I think I'd rather the universal saves in any case. Group games are a lot of fun!

Reply

12-08-2012 06:03 PM

adom-admin adom-admin is offline

The Creator

Truly universal saves are a nightmare. As we had to discover with the new slew of C compilers Jochen has been testing to support the currently supported platforms. You'd be surprised how many C errors are totally ignored even by mainstream compilers and only are discovered by niche compilers (we were). And there actually were some pretty uncool effects with the type system across the currently supported platforms. That does not even take byte ordering into account - which probably would be a major nightmare to fix.

The idea behind the more strict tests is to prevent error messages with strange errors caused by folks swapping files across platforms and versions...

Reply

12-08-2012 06:46 PM

Mobius

Ancient Member

I've been there with universal file formats. For a sense of what it does to you, see this alt-future picture of adom-admin: https://sphotos-b.xx.fbcdn.net/hphot...56044387_n.jpg

Code:

  fwrite(stream, &data, sizeof(data));

becomes (at best!)

Code:

 data_type tmp_data;
 tmp_data.field1 = SWAP32(data.fiedl1);
 ....
 tmp_data.field15 = SWAP16(data.field15);  
fwrite(stream, &tmp_data, sizeof(tmp_data));

It's do-able, but code bloat.

Originally Posted by adom-admin

Truly universal saves are a nightmare. As we had to discover with the new slew of C compilers Jochen has been testing to support the currently supported platforms. You'd be surprised how many C errors are totally ignored even by mainstream compilers and only are discovered by niche compilers (we were). And there actually were some pretty uncool effects with the type system across the currently supported platforms. That does not even take byte ordering into account - which probably would be a major nightmare to fix.

The idea behind the more strict tests is to prevent error messages with strange errors caused by folks swapping files across platforms and versions...

Reply

12-09-2012 03:49 PM

Al-Khwarizmi Al-Khwarizmi is offline

Ancient Member

And how about a common save file for at least the 2-3 most common formats?

32-bit OSes have lingered around for too long, but they finally seem to be becoming a minority. In the latest Steam survey, 64-bit Windows 7 had 58.58% market share, and the sum of 64-bit Windows XP+Vista+7+8 had around 68% market share.

I guess if the saves were compatible between 64-bit Windows, 64-bit Linux and 64-bit MacOS, most users would be able to play community games. Users of less common platforms would be out of luck with that, but at least it would be something.

Reply

12-09-2012 04:05 PM

Mobius

Ancient Member

Originally Posted by Al-Khwarizmi

And how about a common save file for at least the 2-3 most common formats?

32-bit OSes have lingered around for too long, but they finally seem to be becoming a minority. In the latest Steam survey, 64-bit Windows 7 had 58.58% market share, and the sum of 64-bit Windows XP+Vista+7+8 had around 68% market share.

I guess if the saves were compatible between 64-bit Windows, 64-bit Linux and 64-bit MacOS, most users would be able to play community games. Users of less common platforms would be out of luck with that, but at least it would be something.

Byte order / int size are not the only potential differences between compilers and the data structures they make.
Bit field alignment, field padding, field order, union alignment and handling, and general orneriness. I did it 15 years ago, needing to make 3 systems read same files. It's doable, but I imagine it's 2-4 weeks of work, including testing.

Reply

12-09-2012 04:20 PM

jt

Administrator

Just an example: the size of an "Item" struct is 164 bytes if I use the Xcode compiler. If I compile the NotEye version for OSX with the command line version of the compiler, the "Item" struct is 168 bytes. No unusual compiler options are used.
So this happens on the same system with the same compiler. And right now I have a "zoo" of compilers. :)

IMHO the only way to prevent this (and to keep save games somewhat more compatible with newer versions of ADOM) is to store all information in a different format, e.g. JSON. But converting all code to use JSON format is not an easy task and takes time.

Maybe I try to convert the highscore file to JSON first and see how this goes. We'll see about the save game later.

Reply

12-09-2012 05:46 PM

Mobius

Ancient Member

Originally Posted by jt

Just an example: the size of an "Item" struct is 164 bytes if I use the Xcode compiler. If I compile the NotEye version for OSX with the command line version of the compiler, the "Item" struct is 168 bytes. No unusual compiler options are used.
So this happens on the same system with the same compiler. And right now I have a "zoo" of compilers. :)

IMHO the only way to prevent this (and to keep save games somewhat more compatible with newer versions of ADOM) is to store all information in a different format, e.g. JSON. But converting all code to use JSON format is not an easy task and takes time.

Maybe I try to convert the highscore file to JSON first and see how this goes. We'll see about the save game later.

struct size there is possibly because compilers can boost the size of a struct to improve memory access. For 64 bit compilers, they may make the struct size be even multiple of 8 bytes. (168 vs 164)
See http://stackoverflow.com/questions/8...nment-in-xcode for some hints.
-
But yes, universal save usually means ditching the 'store memory straight to disk' mode, and instead sending fields out the stream one at a time - and reading too.

Reply

Issue Tools