View Full Version : Problem reading NGBH ressource
23rd Jun 2012, 8:32 PM
I wish to make a tool like SimTracker with a spécial feature.
This feature is that, the software will gather data directly from the game datas. For now, I'm exploring the packages files. Thanks to the SimsWiki, I've made pieces of code to extract datas from package files. Thanks to SimPe, I can check that data obtained with my code are correct. But...
In my investigations, I block on a problem: How to make the link between a sim instance and a sim GUID? Further investigations led me to the NGHB ressource. Here is the problem
The format of NGBH described in the SimsWiki do not match what I read. The header is Ok, but, not what that follows.
Here's a piece of an output log:
From File \Neighborhoods\N001\N001_Neighborhood.package
48 42 47 4e < type
cb 00 00 00 < version
00 00 00 00 < deprecated
80 00 00 00 < height
80 00 00 00 < width
09 00 00 00 < next string legnt
54 65 6d 70 65 72 61 74 65 < terrain type
00 00 00 00
00 00 00 00
00 00 00 00
00 00 00 00
00 00 00 00
00 00 00 00
00 00 00 00 < 28 reserved bytes Ok
?? According SimWicki for res 4E474248, Next should come a slot (Global)
00 00 00 00 <- instance
be 00 00 00 <- version // NO match should be: cb 00 00 00
?? assume no global to continue exploration and...
?? a SLot count (Families)
00 00 00 00 <- empty => I've 118 families recorded in the hood (SimPE export),
00 00 00
00 be 00 00 00 03 00 00 00 dd d3 a9 f0 00 00 00
00 00 00 00 00 00 00 02 00 00 00 01 00 00 00 c9
// continues ... real bock is 582844 bytes long...
After Header I expect a NgbhSlot ( for Global Tokens) entry, then a Sections entry. I've made something wrong but I don't understand what.
If you have any idea that could help thanks !
23rd Jun 2012, 10:10 PM
If you'd like, you can get the source code for SimPE or the HoodChecker to see how we get the connection between the Sim Instance and Sim ID. I believe that we use basically the same method; the HoodChecker code may be slightly simpler to understand if you haven't done any work with SimPE. However, I must tell you that I often use the SimPE source code as a reference because we know that it works.
I don't see any problem with the NgbhSlot that you are examining. It contains an instance number of 0 and, since the version is >= 0xBE (nightlife), an additional NgbhSlot version of 0xBE, implying that the NgbhSlot was never updated to a later version. There is nothing which says that the version must match the previous version. The comment is "apparently never used; same values as Version above". In other words, it has the same value list as the previous version, but can be ignored because it is (apparently) never used.
The next 4 bytes are the item count of 0, meaning that there are no global NgbhItem Special Simulator tokens.
The next 4 bytes are the item count of 0, meaning that there are no global NgbhItem Standard Tokens.
I'm not sure why you split the next DWORD off into 3 bytes and placed the remaining byte on the next line, since that makes the rest of the array difficult to read. However, now that you are done with the NgbhSlot, you will be seeing the data for the sections for Lots, Families, and Sims, in order.
24th Jun 2012, 2:14 AM
First, thanks for your answer and thanks for your CC's :)
Ignore the version was my first idea, but the Wiki indicate that version is repeated in NgbhSlot, so if versions does not match there may be a bug, but I dont find this version near. I Suspect a problem. The Wicky may be wrong, but I prefer think that I'am wrong first.
The datas don't seem to have any problem, the best proof is that the game handle them perfectly. The problem stand in the way I process this entry. Reading the Wiki, I understand that first there's a header, then a NgbhSlot containing global data and follow a section block with two list NgbhSlot. this ressourcre weight about 600k and I get nothing .
I already have a source set of Simpe, but it does not compile, I'm not familiar whith C# .Net so I don't really understand what goes wrong. I will check the HoodChecker.
24th Jun 2012, 3:13 AM
Each version number is independent and does not need to match. There is no problem. Large packages like neighborhoods and lots are not updated to a new version on-mass, but only as required.
24th Jun 2012, 12:48 PM
Problem solved, it was a nasty bug in in my NgbhItem processing routine. Now I have my datas. I didn't really find what I expected (a link between guid and instance) for now, but I can continue my investigations.
Thanks for your help :)
24th Jun 2012, 7:26 PM
There is no link. If I remember correctly, you need to open every user package and try to correlate the Sim Instance and Sim ID in the Sim Description in the neighborhood package with the Sim GUID in the OBJD of the user package.
29th Jun 2012, 2:47 AM
By user package do you mean packages that ares located in the Characters subdirectory?
I didn't know where to search and by lurking around, I've found there some JPG images of sims I was able to identify without mistake. Instance of JPG matches an age transition which is 1 for toddler, 2 for child, 4 for teen, 8 for adult, 16 for elder, 32 for baby animals, and 64 for young adults (I didn't find human babies images, it should be exists one because I'd a birth in my hood test, I expected to find it, it's high instance ID could have been of some help...). Somes images are duplicated in differents files (I will try to check these ones). When an age change occurs in game, the two images corresponding to each age sections are present in the file.
As far as I've seen, other instance values are 256,512,1024... to 16384. Images whose instances are greater than 64 are black squares. Somes package have one and some does not, but it seem's that they have only one image whose instance is greater than 64. I've no image with instance 128.
So far I've found no link between the XXXXX number and sim or family instance, Burbs family for exemple have instance 8, sims 7,10 and 48 and user 162,163 and 164. Goth family have instance 1, sims have instance 1,15 and 16 and user ares 0,2 and 3. I've checked user 1 it 's seems that it is Bella Goth sim instance 19 family instance 0.
I think that those package are interesting to investigate for what I plan to do, but I have no way to link them to FAMh or PDAT entries, except for current age section image selection.
I've not tried to read OBJD entries yet, I didn't thought they were linked with sims, or families. I've noticed that package in Characters/ have this entry, I will explore it.
29th Jun 2012, 3:21 AM
Yes, the Characters subfolder contains the User packages, which have most of the data about individual sims.
Which number are you calling XXXXX? Do you mean the Sim ID? The FAMI - Family Information record correlates Family Instance with a number of Sim IDs. The SDSC - Sim Description (aka PDAT) correlates a Sim Instance with a Sim ID.
When you open the User file, the Sim ID is in the OBJD record and the sim's name is in the CTSS - Catalog Description. What other information are you interested in?
29th Jun 2012, 7:25 PM
I call XXXXX Number, the number that is part of the user package file name, for exemple
I've started to look at OBJD. I've located the Sim GUID, first name, last name and description. Everything become much more clear thanks to you :).
The link that is not completly clear for me is the link sim instance <-> sim guid. I think I've found it, but I'm not sure for now. I've noticed that PDAT entries were bigger in the Neighborhood.package, than in suburbs packages. In this file, PDAT seems to hold GUID at offset 0x1DC. For now, I've checked with only one sim, I must extend my tests to validate the link.
29th Jun 2012, 8:37 PM
First run, the PDAT entries in Neighborhood.package are big enough to hold offset 0x1DC, manual check with GUID I know are positive. I think I've my link :)
29th Jun 2012, 9:29 PM
As far as I can tell, the file number isn't stored anywhere. I suspect that you could renumber all of your User files and the game would continue to work just fine. Note that I've never tried this, but it might be an interesting test. Basically, the file number is irrelevant.
30th Jun 2012, 1:42 PM
I've run the test, first, the only thing I've noticed stands on loading time. I feel, game loads slower the first time after renumbering, but it's only a feeling.
I've played with the renumbered sim, it was fine. I've created a new sim, no problems too, it took the user file number left free by the renumbering. But there's something strange anyway, two maxis premade families are in the Sim Box and are installed on lots! Both works! It seems that the game have recreated instances for these two families. I'm not sure that it is due to the renumbering but...
30th Jun 2012, 3:29 PM
I comfirm, that there are side effects, User00000.package have no TGA records (I renumberd user package 2, same family) it have some before renumber, and no new user package were created when I played with family copies. The game seems to handle it well. I've created another sim and played it without errorr and a corresponding user file was created. I'll take a backup and retry the test.
30th Jun 2012, 11:42 PM
That certainly seems to indicate that the User number is basically irrelevant. The Sim ID and Sim Description instance number are used in a lot of places, so those are the ones that you want to focus on.
3rd Jul 2012, 12:11 AM
I aggree with you, user number is probably irrelevant. I'haven't run the tests further for now, this part seems to work fine. For now, I won't go further this way, what I have already, seems to work fine. Link which are missing to me are now FAMt and SREL. SREL don't seems to be a problem, I've already a basic processor to handle them, the only interrorgation I get is about "chemistry", my game is in french, so I don't know if chemistry is the right term. I refer to the litlle lightning icon appearing on other Sim Icon appearing in relation tabs (no lightning means nothing, one litghtning means small chémistry, two means strong chemistry...). FAMt, decoding will probably not be a problem too, structure is quite simple even if I don't know what is stored in the "Type of tie" DWORD.
Currently, I'm focusing on establishing a link between families and lots. For now it doesn't work, because I've à bug in LDEF decoding, I've to fix it, prior I can go further. I'll let you know about my progress. I'll not fix it tonight, I'm too tired ;)
5th Jul 2012, 8:31 PM
Hi, some news about my research,. I now have the connections with families and lots, FAMt and SREL too. I'm now focusing on deep SDSC exploration. I've encountered a problem that I think i've solved. The problem is about primary an secondary aspiration, I first thought that offset 0x0068 holded only primary aspiration. Further investigations showed that it is in fact a bitfield:
0x0001 => Romance
0x0002 => Family
0x0004 => Money
0x0008 => I've seen this one only on sims that have a non zero ghost flag (offset 0x0094), all were unlink too.
0x0010 => Reputation
0x0020 => Knowledge
0x0040 => Growth (children toddlers and babies)
0x0080 => Pleasure ? (seen on sims that have the vampire traits), but not only on them.
0x0100 => Grilled Cheese
Bits ares combined when a sim have a secondary aspiration. But how to determine which bit holds the primary aspiration ?
I think that I've found something, but I'm not sure. I've noticed that value of offset 0x14C and 0x150 where strange according the comments on the wiki. The first value is labeled "Aspiration Score" and the second "Aspiration Score Raw (divide by 10)". So I suspected that the comment of offset 0x0150 was wrong. I tried differents things and it seems that a bitwise and between aspiration at offset 0x0068 and aspiration score raw (offset 0x0150) give the primary aspiration. It worked on five sims that have a secondary aspiration set. On my test hood there are not many sims I've played with, so there are only a few sims that have a secondary aspiration, I think I've tested all of them.
I will extend my tests by assigning more secondary aspiration.
The end of the SDSC looks like
0x01DA WORD => Sim instance
0x01DC DWORD => Sim GUID
0x01E0 DWORD => always 3
0x01E4 DWORD => count
then, count DWORD (I d'on't know what they are for).
9 bytes set to 0.
I will now focus on talents (selling, toymaking ...). For what I've understood, they are stored in NGBH, as GUID entries. I will play a bit to get sims that have talents.
5th Jul 2012, 11:12 PM
Well, it's seem that renumbering have some issues. In my chase of talent, I've planned an aggressive commercial policy. I first created a community lot devoted to production, objective was to get talents in production (toymaking, potery, sewing...), then I've made a Selling lot to improve ofb talents, but I've made a mistake when I created it, I created it as a residential lot. I tried to change it with cheat codes and I've lost my producing family, my producing lot and my selling lot too! So be carefull, it may not be harmfull.
PS I've backups ;)
8th Aug 2012, 11:02 PM
Hi there :)
No questions tonight, things are going well ;)
I'm currently working on strings and images ressources gathering in game directories. This serves me as test and optimization purposes for my DBPF low level processing routines (package holds many datas). I've also done somes GUI embryos to help investigations and help me find a way to organize datas.
vBulletin v3.0.14, Copyright ©2000-2013, Jelsoft Enterprises Ltd.