Last visit was: It is currently Sun Sep 22, 2019 3:03 am


All times are UTC-05:00




Post new topic  Reply to topic  [ 105 posts ] 
Author Message
 Post subject: Re: Onlink v0.2.0-a8 r15
PostPosted: Tue Jul 21, 2009 4:11 pm 
User avatar
 

Joined: Mon Jul 13, 2009 10:38 pm
Posts: 41
Well, if that were to be taken into consideration, I'd say move to a more efficient save file format like the one SoL proposed. The savefiles already need to be fixed anyway, and I'm fairly sure that everyone who's been using the a8 already has their savefiles corrupted and/or bloated beyond recovery anyway. To me, it's a no-brainer: If you have the time and resources, and it won't completely break the game or mess up something else, you should seriously consider optimizing the save file tags.

The fact that save-files wouldn't be "easily human-readable" is the tradeoff to being able to shave off (from my estimates) around 10MBs out of an average-sized 40MBs save file. No doubt having less stuff to parse and less stuff to write would also help on loading and saving times. Of course, I'm not the most prominent expert in this domain, so I don't claim any of what I just said to be accurate.

_________________
Image


Top
Offline   
 Post subject: Re: Onlink v0.2.0-a8 r15
PostPosted: Thu Jul 23, 2009 4:50 am 
Just so you know, you brought this upon yourself. Dancefight.
User avatar
 

Joined: Sat Apr 02, 2005 3:31 pm
Posts: 1171
Location: The vicinity of an area adjacent to a location.
Ugh. I really don't like the idea of revamping the save system again. It took long enough to go through every file, and obviously there have been mistakes in the process (not entirely unexpected).

If nothing else, this really shows why we should focus more on rewriting Onlink. The code base that Onlink is based on is really quite terrible. Anyone with a few years C++ experience would tell you the same. It's also extremely limiting. We can't fix some of the problems that we have known about for years just because the code base would break horribly if we fixed it. The save format is an example of such a problem. The problem with the old save format was that there was no nice way to upgrade from save format X to save format X+1 without continually adding non-foolproof code that upgrades old formats. For instance, in the old format, if you remove something from a class (especially if it's something like a BTree), you'd have to figure out where it begins and ends which wasn't exactly simple. In the new format, you just skip over such unrecognized items and there's no harm done.

I'm sorry the XML save format has caused so much pain, but I think the save format only amplified an already existing problem. Save files are bloating on their own, and that's something that just shouldn't happen. Based on the content of DaFranker's save file, I'm pretty sure that the problem is that the log banks are going way over the capacity that I capped them at. I intend to fix this, but before I do, I need to figure out how you guys have been reproducing this bug. I've tried leaving Onlink running for a couple hours, and I've tried fast-forwarding the game time, but I can never quite reach the level of insane pain you guys get to. Care to enlighten me as to the process for this?

Once we eliminate the save bloat problem, things will get much better for this version of Onlink. Sure, we'll have occasional glitches with the save system, but I'm convinced that the save bloat issue is the biggest problem.

And finally, I've already started a prototype for an Onlink rewrite. It's pretty simple right now. I'm basically going through and cleaning up the API before we start filling the project with new code. I'm tentatively licensing the code under a two-clause BSD license, and hopefully will publish something soon. We'll get back to you on this later. I'm pretty excited about it.

_________________
- Tycho

Image


Top
Offline   
 Post subject: Re: Onlink v0.2.0-a8 r15
PostPosted: Thu Jul 23, 2009 6:20 am 
Just so you know, you brought this upon yourself. Dancefight.
User avatar
 

Joined: Sat Apr 02, 2005 3:31 pm
Posts: 1171
Location: The vicinity of an area adjacent to a location.
I believe I've found the root of the save bloat problem. I'm currently testing a fix for it. The good news is, once I publish this fix, I'm pretty sure everyone's saves will still be loadable (and correctable!). The game should be able to load them, and after two in-game weeks, the logbanks of all the systems will have gone through a pruning process, and your save file size will be back to normal.

I'm working on fixing some other bugs (like Defrag being broken), and then I'll start building some new Windows and Linux builds.

Everyone please verify that all the bugs you've seen have been reported. And if there's something that hasn't been reported yet, please file a bug report. The bug tracking system is much easier for us to follow than a bunch of complaints on a forum (although it does help to discuss it here once it's bug reported).

EDIT: I loaded DaFranker's save, fast-forwarded the game time by half a month and then saved the game. Uncompressed size went from 118MB to 19MB. :)

_________________
- Tycho

Image


Top
Offline   
 Post subject: Re: Onlink v0.2.0-a8 r15
PostPosted: Thu Jul 23, 2009 7:16 am 
User avatar
 

Joined: Sat Nov 17, 2007 8:45 am
Posts: 202
Quote:
I loaded DaFranker's save, fast-forwarded the game time by half a month and then saved the game. Uncompressed size went from 118MB to 19MB. :)

Now *that's* good to hear.


Top
Offline   
 Post subject: Re: Onlink v0.2.0-a8 r15
PostPosted: Thu Jul 23, 2009 7:38 am 
Just so you know, you brought this upon yourself. Dancefight.
User avatar
 

Joined: Sat Apr 02, 2005 3:31 pm
Posts: 1171
Location: The vicinity of an area adjacent to a location.
Aaaaaaaaand r22 is up for Windows.

_________________
- Tycho

Image


Top
Offline   
 Post subject: Re: Onlink v0.2.0-a8 r15
PostPosted: Thu Jul 23, 2009 7:43 am 
Just so you know, you brought this upon yourself. Dancefight.
User avatar
 

Joined: Sat Apr 02, 2005 3:31 pm
Posts: 1171
Location: The vicinity of an area adjacent to a location.
And Linux is up now, too.

_________________
- Tycho

Image


Top
Offline   
 Post subject: Re: Onlink v0.2.0-a8 r15
PostPosted: Thu Jul 23, 2009 9:40 am 
 

Joined: Thu Apr 28, 2005 3:57 pm
Posts: 175
AOL: DKWiz214
Yay. This is exciting. I just filed a bug report yesterday about a minor glitch with the bounce saving. If you load and save your connection continuously, each time you do so it adds 127.0.0.1 to the beginning of your bounce path. This results in (minor) bloat and also a map glitch where it shows Gateway several times. Also, keymapping still doesn't load properly. I checked the xml save and I fount it, but it doesn't load. Also there is still the glitch with buying software while an aux gateway is selected for next update which causes it to disappear/overwrite other software. I believe it's looking on the aux databanks to see where to write it, but then it writes it to the main databanks, overwriting other files, even though programs are still usable they are inaccessible for things like cracking ACS.

Just a reminder of a few bugs that have been bugging me. Nothing game-breaking (except the software bug, but I know how to avoid it now).

Update: In the process of squashing the log bug, it seems you have made a new one. I made a new user, since my old one is stuck trying to crack an ACS, and when I logged into the uplink ISM there were a total of 5 missions, none takeable. I tested again with another new user, same problem. It seems you pruned not only logs, but also missions.

_________________
The light shall be your demise.


Top
Offline   
 Post subject: Re: Onlink v0.2.0-a8 r15
PostPosted: Thu Jul 23, 2009 11:24 am 
User avatar
 

Joined: Mon Jul 13, 2009 10:38 pm
Posts: 41
Eh... the new build sounds nice, but I'm guessing that getting this when trying to load my save file is not good?

I'm also guessing that getting an almost-identical-yet-slightly-different thing under x86 is bad too?

_________________
Image


Top
Offline   
 Post subject: Re: Onlink v0.2.0-a8 r15
PostPosted: Thu Jul 23, 2009 11:36 am 
Just so you know, you brought this upon yourself. Dancefight.
User avatar
 

Joined: Sat Apr 02, 2005 3:31 pm
Posts: 1171
Location: The vicinity of an area adjacent to a location.
Don't know why that would happen. Is the profile you're loading the same as the one you uploaded?

_________________
- Tycho

Image


Top
Offline   
 Post subject: Re: Onlink v0.2.0-a8 r15
PostPosted: Thu Jul 23, 2009 11:37 am 
Just so you know, you brought this upon yourself. Dancefight.
User avatar
 

Joined: Sat Apr 02, 2005 3:31 pm
Posts: 1171
Location: The vicinity of an area adjacent to a location.
Quote:
In the process of squashing the log bug, it seems you have made a new one. I made a new user, since my old one is stuck trying to crack an ACS, and when I logged into the uplink ISM there were a total of 5 missions, none takeable. I tested again with another new user, same problem. It seems you pruned not only logs, but also missions.
I noticed there were surprisingly few missions as well. But I don't think the log pruning did that.

_________________
- Tycho

Image


Top
Offline   
 Post subject: Re: Onlink v0.2.0-a8 r15
PostPosted: Thu Jul 23, 2009 11:40 am 
Just so you know, you brought this upon yourself. Dancefight.
User avatar
 

Joined: Sat Apr 02, 2005 3:31 pm
Posts: 1171
Location: The vicinity of an area adjacent to a location.
Quote:
Quote:
In the process of squashing the log bug, it seems you have made a new one. I made a new user, since my old one is stuck trying to crack an ACS, and when I logged into the uplink ISM there were a total of 5 missions, none takeable. I tested again with another new user, same problem. It seems you pruned not only logs, but also missions.
I noticed there were surprisingly few missions as well. But I don't think the log pruning did that.
The dates on the missions are wrong, too. They say 1/1/1970.

_________________
- Tycho

Image


Top
Offline   
 Post subject: Re: Onlink v0.2.0-a8 r15
PostPosted: Thu Jul 23, 2009 11:47 am 
User avatar
 

Joined: Mon Jul 13, 2009 10:38 pm
Posts: 41
Quote:
Don't know why that would happen. Is the profile you're loading the same as the one you uploaded?
No, it's a more recent (and somehow, much more voluminous) file. It's the afore-mentioned 473MBs savefile that was created a few minutes after the one I uploaded... well, actually, now that I check, it's also linked in my previous post (linky). So yes, it's the same as the (second) one I uploaded, but you seem to have used the first one I uploaded, which was the save generated while the game was still open. This second savefile is from the same profile a few minutes later when I closed the game. The compressed file somehow went from 8MBs to 6MBs yet the uncompressed file went up by over 350MBs, after having closed the game. I smell shenanigans. Or maybe the bloating bug just exploded into a sparkle of zillions of empty log entries. Or something.

I'll try it with the older savefile, just in case. Maybe the windows compiler decided to play a prank on you or something? ;)

EDIT: Nope, whew, it's not a prank ^^

I tried with the old save file, and it loaded just fine too. I assume it's an issue related to that save file in particular, maybe because it's so damn humongous or maybe because there are just way too many entries to load. The file might somehow just be corrupted too, since I haven't opened it since then either even in the previous build. That would explain the ludicrous size.

_________________
Image


Last edited by DaFranker on Thu Jul 23, 2009 11:53 am, edited 1 time in total.

Top
Offline   
 Post subject: Re: Onlink v0.2.0-a8 r15
PostPosted: Thu Jul 23, 2009 11:52 am 
Just so you know, you brought this upon yourself. Dancefight.
User avatar
 

Joined: Sat Apr 02, 2005 3:31 pm
Posts: 1171
Location: The vicinity of an area adjacent to a location.
Quote:
The dates on the missions are wrong, too. They say 1/1/1970.
Goddamnit. See, this is what I mean by unmaintainable code!
Code:
if (mission->createdate.GetYear() == 1000) mission->SetCreateDate(&(game->GetWorld()->date));
The old default for the Date class was year 1000. But since the standard date/time functions don't work with anything pre-1970, I set the defaults to 1/1/1970. And since it didn't get 1000 as a response, it never sets the mission creation date.

Anyway, I'm going to find out why missions are so far and few between and then upload some new Windows/Linux builds. I'm kind of hoping to get something at least semi-decent up before I leave for two and a half weeks.

EDIT: Apparently that stupid Date class issue is what caused so few missions to appear. They expired before they could be used.

_________________
- Tycho

Image


Top
Offline   
 Post subject: Re: Onlink v0.2.0-a8 r15
PostPosted: Thu Jul 23, 2009 12:08 pm 
Just so you know, you brought this upon yourself. Dancefight.
User avatar
 

Joined: Sat Apr 02, 2005 3:31 pm
Posts: 1171
Location: The vicinity of an area adjacent to a location.
r23 is up.

_________________
- Tycho

Image


Top
Offline   
 Post subject: Re: Onlink v0.2.0-a8 r15
PostPosted: Thu Jul 23, 2009 12:12 pm 
User avatar
 

Joined: Mon Jul 13, 2009 10:38 pm
Posts: 41
Quote:
Goddamnit. See, this is what I mean by unmaintainable code!
Code:
if (mission->createdate.GetYear() == 1000) mission->SetCreateDate(&(game->GetWorld()->date));
Eh... I have to agree that that's pretty ugly. I'm just a rookie modding scripter, but even I can see that using a variable, maybe something default to the mission object, would've been much cleaner and caused less problems. Well, any hardcoded number is always bad in the first place... with extremely few exceptions (such as /0 checks). At least, so I was taught.

Also, two more bugs:
Bugzilla #128 and
Bugzilla #129

_________________
Image


Top
Offline   
Display posts from previous:  Sort by  
Post new topic  Reply to topic

All times are UTC-05:00


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
Theme created by Miah with assistance from hyprnova