Replies: 209 (Who?), Viewed: 60403 times.
Page 1 of 9
velocitygrass
Inventor
Original Poster
#1 Old 13th Sep 2014 at 11:10 AM Last edited by plasticbox : 29th May 2015 at 8:47 PM.
Default Sims4XmlExtractor: extracts individual tuning files from the combined tuning resource (2014-10-29: v0.0.0.6: fix for STBL issue and support for object info)
--

Moderator note 2015-05-08: velocitygrass hasn't logged in for quite a while, so this tool should be regarded as currently not supported. In this thread scumbumbo has provided an alternative that works with the current patch level -- you can use that for now.

--

2014-10-29: v0.0.0.6: Fixed STBL refs not being used when they had missing leading zeros. Added support for object info (see FAQ for info on implementation). Thanks DarkWalker!
2014-09-28: v0.0.0.5: Fixed tuning package file filter to actually be SimulationDeltaBuild0.package, not ...FullBuild.... Thanks plasticbox!
2014-09-28: v0.0.0.4: The combined tuning can now be read from the SimulationDeltaBuild0.package directly and the tool will optionally add comments with the name of reference or the text of a STBL ID.
2014-09-22: v0.0.0.3: Completely rewrote the parser with classes that should preserve the attribute order. It also speeds up the initialization and lowers memory consumption. Indentation for non-compact xmls is now 2 spaces.
2014-09-14: v0.0.0.2: fixed the module resource type (it's 0x03B33DDF, which I verified with two sample mods that change the starting funds and the starting day and time) and added {instance#} to the naming patterns (for decimal instance).

This tool resolves the references of the combined tuning xml and extracts the instance and module tunings in individual xml files.

Please let me know about errors so that I can fix them.

Source code is GPL3 and attached.



How to install
- Use 7zip to unpack the folder/file. For the update version extract that into your existing Sims4XmlExtractor folder.

How to start
- Just click the Sims4XmlExtractor.exe

How to use
  1. You need to give the tool the path to the previously exported combined tuning resource 0x62E94D38-0x00000000-0xF1352E740C4710BA or SimulationDeltaBuild0.package which contains it.
    - Click "Browse" to select the resource or SimulationDeltaBuild0.package
    - You can also simply paste the path to the file in the line edit (this option doesn't remember the path)
  2. Select an export folder for the files by browsing or pasting it. The folder must exist.
  3. Select a naming pattern for the exported files. I have convenience buttons for S4PE compatible format and the EA unpacked mod naming pattern, but you can create your own naming pattern by using one of the following placeholders:
    - {instance}, {group}, {type}: lowercase hex without leading 0x
    - {INSTANCE}, {GROUP}, {TYPE}: uppercase hex without leading 0x
    - {instance#}: decimal instance
    - {name}: the name of the resource, e.g. trait_EternalBond
    There's a preview of what the filename will look like. Please ensure that your combination results in unique file names. If a file already exists, it will simply not be written.
  4. Check or uncheck creation of subfolders. If checked the tool will group the xml files the way they are grouped in the combined tuning (tag R), e.g. in "action" or "trait" subfolders.
  5. Check or uncheck if all whitespace and line breaks should be removed. This is recommended for distribution by EA, but for browsing the files leaving them in can make them more readable.
  6. Check or uncheck if names of reference targets should be put in comments after the target, e.g.
    Code: 
    <L n="objectives"><T>24715</T></L>
    becomes
    Code: 
    <L n="objectives"><T>24715<!--Max_Astronaut--></T></L>

    Please note that I do not check if the value is actually a reference or just a normal number. E.g. it is possible that a setting has an integer value of "24715" and the <!--Max_Astronaut--> would still be added even though it's not meant as a reference to that objective.
  7. Optionally add the path to an exported STBL file or a package that contains one. If this is done and the option to print reference names in comments is selected, the tool will read the STBL file (or the first it finds in the given package) and use that to resolve all hexadecimal references and print the string in comments, e.g.
    Code: 
    <T n="descriptive_text">0x8F1F5015<!--"Have a Sim reach max level in the Astronaut career."--></T>
  8. Click "Export" to start the export

FAQ
  1. Where do I find SimulationDeltaBuild0.package or this resource?
    - The combined tuning resource has the type 0x62E94D38, the group 0x00000000, and the instance 0xF1352E740C4710BA and can be found in the SimulationDeltaBuild0.package (and the unpatched old version in the SimulationFullBuild0.package). You can use S4PE to extract the file from the package.
    - SimulationDeltaBuild0.package (and SimulationFullBuild0.package) can be found the Data/Simulation subfolder of the Sims 4 installation.
  2. What do I do with the xmls?
    - That depends entirely on your goal.
    - They can serve as a way to look up information, e.g. how much will career X earn at level Y.
    - They can serve as a base to modify tuning settings. E.g. you can change how much a sim will earn from a career at a certain level. Please be aware however that many tunings that have a rendering component (where something will be displayed on screen) will require to make the same changes in the accompanying DATA file (see my other tool for making changes to that). Also note that the .tdesc files available in the EA modding documentation include additional information and tuning settings (like the defaults which might never appear in the extracted tunings because they're not overridden). Thanks to plasticbox for pointing this out in this thread.
    - Plasticbox has posted a XML Tuning How-To here.
    - For discussion about package files vs unpackedmod, please check this thread. EA has fixed the issue with tuning in packages.
    - For helpful hints on handling loose tuning files check DarkWalker's tutorial Making better use of the unpackedmod folder.
  3. Why all the naming pattern options?
    - The two ways to work with the tuning files right now are putting them in a package file or placing them directly in the unpackedmod folder (as per SimGuruModSquad's workaround for current limitations).
    - The unpackedmod requires a special naming pattern to work (e.g. 0x00000000!0x0000000000006cb6.0x7df2169c)
    - If you use S4PE to import your resources, a different naming pattern will allow you to import from file without making any further changes (i.e. having to type in manually the IDs).
    - For a normal human just seeing the TGI combination will not be enough to make them identify the actual content of the file, so I added an option to include the name in file so that you can scroll through them and find what you're looking for (or just see what's there) without having to open each file or search.
    - The decimal instance option was added because this is how the references are done in the xml and the upper and lower case are purely for esthetics.
  4. How do I get the STBL?
    - It's the resource of type 0x220557DA in Strings_[language code].package (e.g. Strings_ENG_US.package), which can be found in the Data/Client subfolder of the Sims 4 installation. It can be exported e.g. with S4PE.
  5. How does the object info work?
    - For the object info to be retrieved you'll need to use the package option for the combined tuning and have it point to SimulationFull/DeltaBuild0.package. The tool then assumes the installation structure so that it looks for ClientDeltaBuild0.package in a "Client" folder next to the "Simulation" folder of the SimulationFull/DeltaBuild0.package.
    - It then retrieves info from all COBJ files for later use (TODO: do this as needed which should be faster)
    - In the xml it looks for certain n attributes in tags that indicate the next decimal is an object and not a tuning. Specifically it looks for: "reward", "target_game_object", "items_to_check", "actual_object", "definition".
    - The tool ignores this in the case of situation and aspiration_track tunings, where the reward is actually a tuning.

Credit: Thanks to the S4PE team, the EA devs who created the doc, the FNV site, TheHologramMan for unpyc3.py, which I used to decompile the python files, which enabled me to find the module resource type, DarkWalker for info on where to find object name and description.
Attached files:
File Type: 7z  Sims4XmlExtractor0002update.7z (33.6 KB, 777 downloads) - View custom content
Description: Update/downgrade for an existing Sims4XmlExtractor installation
File Type: 7z  Sims4XmlExtractor0003update.7z (36.6 KB, 257 downloads) - View custom content
Description: Update/downgrade for an existing Sims4XmlExtractor installation
File Type: 7z  Sims4XmlExtractor0005update.7z (51.4 KB, 384 downloads) - View custom content
Description: Update/downgrade for an existing Sims4XmlExtractor installation
File Type: 7z  Sims4XmlExtractor0006.7z (6.59 MB, 2226 downloads) - View custom content
Description: Full program with all dependencies. Download this when in doubt.
File Type: 7z  Sims4XmlExtractor0006update.7z (55.6 KB, 1174 downloads) - View custom content
Description: Update for an existing Sims4XmlExtractor installation
File Type: 7z  Sims4XmlExtractor_src0006.7z (155.1 KB, 1128 downloads) - View custom content
Description: Sims4XmlExtractor source code (Qt Creator Project)
13 users say thanks for this. (Who?)
Advertisement
Shimrod101
Mad Poster
#2 Old 13th Sep 2014 at 11:25 PM
Thanks for this! It doesn't appear to do anything incorrectly as far as I can tell so far. It also seems that default replacement XML tuning on EA's stuff is now within reach (if not just around the corner).
MedievalMods and Sims3mods: Dive Cave Reset Fix, Resort Revamp, Industrial Oven Revamp, Will O' Wisp fix, UI Sounds Disabled, No Cars, Gnome Family Planner, Townies Out on the Town, No Martial Arts Clothes, Fast Skilling, etc. http://simsasylum.com/tfm/
velocitygrass
Inventor
Original Poster
#3 Old 14th Sep 2014 at 12:10 AM
Thanks for letting me know that it works for you And yes, default replacements for tunings work. I've removed the friends requirements for my sim's career successfully, which required both Data and XML tuning.
plasticbox
Pettifogging Legalist!
retired moderator
#4 Old 14th Sep 2014 at 12:32 AM Last edited by plasticbox : 14th Sep 2014 at 3:38 AM.
Thanksthisisgreat! It truly makes it super easy to poke at things (so far I'm still in the "I have no idea what I did, but it worked! Yay! phase though =P .. I managed tomato cake and child-only townie families and other unassorted Weirder Stuff. It's fun! =) ).


Edit, to make this a somewhat more useful post: can you perhaps move the standalone download to attachments, as well? I guess I'm not the only one who downloads the wrong version otherwise -- I didn't see the actual download link up there in the main post text at first.

ETA, I also experienced the same issue you mentioned at EA forums -- XML not being read from a package, but the /unpackedmod approach works.
Stuff for TS2 · TS3 · TS4 | Please do not PM me with technical questions – we have Create forums for that.

In the kingdom of the blind, do as the Romans do.
velocitygrass
Inventor
Original Poster
#5 Old 14th Sep 2014 at 9:27 AM
Yes, I hope this makes playing with (or just looking at) tunings more accessible.

I've uploaded the full program directly now. I hesitated before because it's so large, but since you're a mod, I'll take that as permission

Hopefully the package issue will be fixed soon, so that mods can be distributed as packages again. With the way it works now, it would be a nightmare to test tuning mods and then later try to remove them again, especially once you have a bunch.
JustAnotherSimsFan
Lab Assistant
#6 Old 14th Sep 2014 at 10:32 AM
This'll definitely make things easier, thank you
plasticbox
Pettifogging Legalist!
retired moderator
#7 Old 14th Sep 2014 at 11:57 AM
Quote: Originally posted by velocitygrass
I've uploaded the full program directly now. I hesitated before because it's so large, but since you're a mod, I'll take that as permission

Hopefully the package issue will be fixed soon, so that mods can be distributed as packages again. With the way it works now, it would be a nightmare to test tuning mods and then later try to remove them again, especially once you have a bunch.


Oh sure -- 6MB are not a problem when they're doing something useful! =)

Re. the not-reading-from-package issue, I just posted again at EA forums .. hopefully they'll notice.
Stuff for TS2 · TS3 · TS4 | Please do not PM me with technical questions – we have Create forums for that.

In the kingdom of the blind, do as the Romans do.
~MadameButterfly~
Mad Poster
#8 Old 14th Sep 2014 at 10:18 PM
Thank-you! You've opened up a whole new world for me to explore.
♥ }i{ Monarch of the Receptacle Refugees }i{ ♥
velocitygrass
Inventor
Original Poster
#9 Old 15th Sep 2014 at 10:32 PM
I've updated the tool with a fix for the module type (my guess was incorrect) and added a decimal instance name pattern since the cross references to instances are done in decimal in the xmls themselves.
chuckles_82
Lab Assistant
#10 Old 17th Sep 2014 at 3:32 AM
Thank you! I'm not very experienced with game code, and have no programming skills whatsoever, so I'm just winging it here. I extracted the XMLs and am now currently importing them into their own package file in S4PE for easy browsing. I would probably recommend to use the sub folders and make it into several packages, because there are 20,518 files!! I can search these files as-is using Notepad++, too. I can't wait to find hidden nuggets of information :D
Margaret Pendragon
Top Secret Researcher
#11 Old 17th Sep 2014 at 4:44 AM
Ohmywow, thank you for this. You rock my Simsworld. Can't wait to play with this tomorrow.
pcgeekri
Test Subject
#12 Old 18th Sep 2014 at 2:01 PM Last edited by pcgeekri : 18th Sep 2014 at 5:54 PM.
Hi,

Thank you, thank you, thank you!

Probably should post the rest of this in modding discussion but someone else could be like me and downloaded this prior to the update on Sept. 14.

Downloaded this around Sept. 12. There seems to be some changes here. First I download the small version and placed it into Data Filer. I see that is gone now. (The file with the executable and the DLL) Did it turn out the XML extractor cannot use the same dependencies as Data Filer? I did download the update with only the executable only and replaced it in my Data Filer folder. Is that okay? or should I download the full version?

Now Step 3. Formats are not defined, old Bhav modder here and this is all new. Please define the naming pattern. Which is the proper button (format) to use if one wants to export, mod the XML and import into a package file? (Ever get this figured out I'll try typing a tutorial on modding packed XML files. lol)
Thank you for your hard work. *hug*

Thanks for any help,

Pcgeeri
velocitygrass
Inventor
Original Poster
#13 Old 18th Sep 2014 at 8:35 PM
pcgeekri, if you can start the new updated .exe file (v0.0.0.2) there's no reason to you to download the full version.

I know (almost) nothing about Bhav modding, so I cannot tell you anything about the comparison to tuning modding in TS4. If you want to import the files into package with S4PE, I'd recommend the S4PE naming pattern.

There are certain issues with xml tuning files in packages for now (I see you already said you found that thread), and in some cases, changes only work if you also change the corresponding DATA file.
pcgeekri
Test Subject
#14 Old 18th Sep 2014 at 9:16 PM Last edited by pcgeekri : 18th Sep 2014 at 11:28 PM.
Quote: Originally posted by velocitygrass
pcgeekri, if you can start the new updated .exe file (v0.0.0.2) there's no reason to you to download the full version.

I know (almost) nothing about Bhav modding, so I cannot tell you anything about the comparison to tuning modding in TS4. If you want to import the files into package with S4PE, I'd recommend the S4PE naming pattern.

There are certain issues with xml tuning files in packages for now (I see you already said you found that thread), and in some cases, changes only work if you also change the corresponding DATA file.


Oh thank you for your response. Well the data file is the one you mentioned at the beginning of your description. That big data file that placed XMLs into folders. I'm modding some of those.
Oh don't worry about what I said about Bhavs, just saying this is all new to me. I'm still trying to get them to work. *hugz*

Edit: Oh thank you again! *big hug* They are working, my import method wasn't right, but those new sim template overrides are working just fine! *smiles* Now to take a break and play some Sims 2!

And if you ever have a question about Bhavs, Bcons, STR text files, and "Run in my stack"/"Run tree by name" just let me know by pm or forum and I'll answer. Oh! and there are Global Bhavs, and Semi-Global Bhavs, private Bhavs, all that fun stuff. My SimPE is still going strong. Long live Sims 2!

Thanks again for your response that's a big help.

Thanks,
Pcgeekri
velocitygrass
Inventor
Original Poster
#15 Old 18th Sep 2014 at 11:25 PM
I meant DATA files (resource type 0x545AC67A). E.g. let's say you want to mod the starting funds for sims. There's an xml resource that is among the extracted tuning files: 0x00000000!0xd66f672e4576ec88.0x266ca322, which includes the 20000 as the starting funds. However, you'll also need to change the corresponding data resource S4_545AC67A_006CA304_D66F672E4576EC88%%+DATA.data (sorry for the mix of naming patterns here; it's how I have them currently exported; I hope you still get the idea) for it to actually work in the game.
pcgeekri
Test Subject
#16 Old 18th Sep 2014 at 11:29 PM
Quote: Originally posted by velocitygrass
I meant DATA files (resource type 0x545AC67A). E.g. let's say you want to mod the starting funds for sims. There's an xml resource that is among the extracted tuning files: 0x00000000!0xd66f672e4576ec88.0x266ca322, which includes the 20000 as the starting funds. However, you'll also need to change the corresponding data resource S4_545AC67A_006CA304_D66F672E4576EC88%%+DATA.data (sorry for the mix of naming patterns here; it's how I have them currently exported; I hope you still get the idea) for it to actually work in the game.


I understand, yes, yes. Thank you. I knew you meant Data. *hugz*

Thanks again,
Pcgeekri
malfus
Test Subject
#17 Old 21st Sep 2014 at 10:49 AM
Is there a way to force your extractor to put xml elements in a consistent order? For instance, the first time I extracted, I got <I s="9882" c="Achievement" i="achievement" m="achievements.achievement_tuning" n="RL_04_50_Mile_High_Club"> and the second time, I got <I m="achievements.achievement_tuning" n="RL_04_50_Mile_High_Club" s="9882" c="Achievement" i="achievement">. It would make doing mass modding so much easier if I could get a good verbatim comparison on all the files.
velocitygrass
Inventor
Original Poster
#18 Old 22nd Sep 2014 at 8:25 PM Last edited by velocitygrass : 22nd Sep 2014 at 11:12 PM. Reason: typo
@malfus: Yes, I'd noticed that too. Unfortunately the class I used to extract the data didn't support this, so I had to completely rewrite the parser, but the order should be consistent now (it was during the several test runs I had throughout development). We'll have to see how it handles the combined tuning of the next patch. I could still explicitly set an order, e.g. alphabetically, if a patched combined tuning results in different orders.
malfus
Test Subject
#19 Old 22nd Sep 2014 at 9:48 PM
Looking good. First test I ran returned the results I had expected from WinMerge. Thanks for the awesome work!
rebecah
Scholar
#20 Old 26th Sep 2014 at 8:20 PM Last edited by rebecah : 26th Sep 2014 at 8:37 PM. Reason: plasticbox explained it.
How do I "Exported combined tuning resource 0x62E94D38-0x00000000-0xF1352E740C4710BA"?

In other words where do I get these from? Please!

Nevermind plasticbox helped me, but for future use it might be helpful if you did explain where to find this.

Thank you very much for creating this tool!
Zach
Lab Assistant
#21 Old 28th Sep 2014 at 1:43 AM Last edited by Zach : 28th Sep 2014 at 3:16 AM.
*read instructions, downloaded and exported, etc.*

Okay, now what?

I know it's not velocity's responsibility to provide instructions for this, but now what do you do with the files? Also, why are we given a choice for the naming pattern? When does the choice of naming come into the picture down the line?

Case in point, as a test, I modded the TechGuru-StartUp's level 10 career (S4_2C70ADF8_00000000_0000000000007ECC_career_TechGuru_StartUp_Level4.xml) to pay 10x per hour (as a test... obviously since using the motherlode cheat would be easier :P), packaged it up with sp4e, dropped it in my mod folder, but it had no effect on my Sim (who is a Level 10 TechGuru).

I looked in the unpackedmod folder (after testing the above in-game), and it's still empty, so that implies that even though the game recognized it as a non-script mod (so the startup dialog told me), it apparently didn't know what to do with it since nothing got unpacked. (I think I read somewhere that's the point of the unpackedmod folder we have to manually create and point our resource.cfg to.)

Edit: Hmmm... never mind, I guess. Though the job panel info didn't change (still reads $516/hour), my sim actually did get paid $5160/hour.
Shimrod101
Mad Poster
#22 Old 28th Sep 2014 at 4:40 AM
The job panel info probably isn't updated because there is a DATA file to go with this XML I would think.
Edit: Nothing is supposed to get unpacked, that folder is supposed to be empty. What you read somewhere is incorrect; having that folder is simply part of the workaround to get the game to recognize XML files.
MedievalMods and Sims3mods: Dive Cave Reset Fix, Resort Revamp, Industrial Oven Revamp, Will O' Wisp fix, UI Sounds Disabled, No Cars, Gnome Family Planner, Townies Out on the Town, No Martial Arts Clothes, Fast Skilling, etc. http://simsasylum.com/tfm/
Zach
Lab Assistant
#23 Old 28th Sep 2014 at 5:39 AM
Quote: Originally posted by Shimrod101
Edit: Nothing is supposed to get unpacked, that folder is supposed to be empty. What you read somewhere is incorrect; having that folder is simply part of the workaround to get the game to recognize XML files.


Ah, okay. Good to know. Thanks for the explanation!
velocitygrass
Inventor
Original Poster
#24 Old 28th Sep 2014 at 11:23 AM
@rebecah: The tool can now read directly from the package (as well as the exported file) and I've added the location of the package to the FAQ. Thanks for pointing this out.
@Zach: I've tried to address a few of your points in the new FAQ section.
rebecah
Scholar
#25 Old 28th Sep 2014 at 3:06 PM
@velocitygrass: Thank you!
Page 1 of 9
Back to top