Locked thread | Replies: 75 (Who?), Viewed: 19335 times. | Locked by: plasticbox Reason: See Downloads > Tools!
Page 1 of 4
Alchemist
Original Poster
#1 Old 8th May 2015 at 8:53 AM Last edited by scumbumbo : 28th Aug 2015 at 6:49 AM. Reason: Version 3.3.1 released as download
Default (Released as download) Combined XML (and Binary Tuning) Extractor
Now Available in Downloads

The latest version of this program is now hosted in the Programs and Utilities download section here at MTS. You should go here to obtain it in the future.



Old Info...
Version 3.2.3 adds comments cross-referencing any XML files. Like the string table comments, this will only be performed when doing a full extract from the game install folder. The full extract will now also unpack a full build for a package if no delta build is available (new game packs that have never been patched).

Note, there was a big nasty bug in version 3.2.2 so if you downloaded that be sure to update to 3.2.3.

Version 3.1 will include the strings from the string tables in XML comments. This is only available when doing the full extract. It will load the strings from all the game string tables, index them, and dump a tab-delimited text file while it's at it. Including the string tables will take a bit extra time, but it's tough to say just how much as I ran my tests back to back and so a lot of data was still cached in RAM. I mean I can't even estimate, because the test including the strings ran a second faster. If you have a reasonably fast machine though, you will probably miss the "Extracting and indexing strings" message if you blink.

Version 3 can now process all the game XML in a single run. No more converting to binary files, then XML, and finally producing the XML extracts. Select the main game folder (it should find it automatically from the registry) and create a new empty output folder, hit extract and go for a short break. The program will find all of the SimulationDeltaBuild0 packages in the game, pull the binary tuning out, convert it to a combined XML file, and finally extract them into separate named folders, BG, EP01, etc. The temporary files are left behind in case you need them (you can still work with the individual files).

For good measure, I replaced that crappy .Net folder selector with a version that uses a common dialog.

As of version 2.0 the XML Extractor can now convert a binary tuning file into the combined XML format. The input file should be a .bnry file extract from one of the games delta packages. The binary file will be loaded and a new file written to the output folder with a .xml file extension. Expect this to take some time, but it only needs to be done once for each of the binary tuning files at each patch level. Once the combined XML has been output, it can be processed further to extract into individual folders and files.

I have written a short tutorial on using the XML Extractor.

Browse for the input file. If the input file has the .bnry file extension it will be processed as a binary tuning file and a combined XML file written to the output folder with a .xml file extension. Any other input file will be treated as a combined XML input file. If an invalid file is specified the program may crash or produce invalid results.

Select an output folder, select the game pack (this will occur automatically if a "standard" file naming is used) and click extract.

Sources are also attached, made in Visual Studio Express 2013 (VB .Net) if someone wants to change it to do something fancier.

The program requires the .NET v4.0 framework to run.

v1.0.1 - Bugfix - wasn't processing the Module instances (tun folder). Didn't update the sources, it's a really easy fix (see comment further below or PM me if you can't find it).
v1.0.2 - Additional Bugfix, the Module type number was generated wrong (thanks for pointing that out Fogity). Also made the application a bit more responsive while it's extracting (output won't hang, you can reposition the window, the buttons get disabled and if you click the close button it will prompt to exit or cancel).
v1.1.0 - Autodetects the game pack/group number if input file is named properly, all known packs are coded in
v2.0.0 - Convert a binary tuning file to the combined XML format. Added name for GP02. (Sources updated with the new code)
v3.0.0 (unreleased) - Pull the binary tuning out of a package file.
v3.0.1 - Full auto mode using the SimulationDeltaBuild0 package files found in the game install folder.
v3.1.0 - Added option to include the string table strings in the XML output when doing a full extract.
v3.2.2 - Added option to include XML cross reference comments.
v3.2.3 - Bugfix - program would crash when extracting a single combined XML.
Download - please read all instructions before downloading any files!
File Type: rar TS4 Combined XML Extractor_src.rar (56.5 KB, 272 downloads) - View custom content
Description: Source code (v3.2.3) - Visual Basic .NET project
File Type: rar TS4 Combined XML Extractor_v3.2.3.rar (26.7 KB, 656 downloads) - View custom content
Description: Current version
10 users say thanks for this. (Who?)
Advertisement
Pettifogging Legalist!
retired moderator
#2 Old 8th May 2015 at 9:23 AM
Thank you, especially for providing the source

Stuff for TS2 · TS3 · TS4 | Please do not PM me with technical questions – we have Create forums for that.

In the kingdom of the blind, do as the Romans do.
Lab Assistant
#3 Old 8th May 2015 at 9:41 AM
The tool works great, thank you Scumbumbo
Mad Poster
#4 Old 8th May 2015 at 10:10 AM
Thank-you Scumbumbo.

♥ }i{ Monarch of the Receptacle Refugees }i{ ♥
Pettifogging Legalist!
retired moderator DELETED POST
8th May 2015 at 10:12 AM
This message has been deleted by plasticbox. Reason: obsolete
Alchemist
Original Poster
#5 Old 9th May 2015 at 3:19 AM
Quote:
Originally Posted by plasticbox
Thank you, especially for providing the source

Wasn't sure if anyone would be interested in the source, but glad some folks are finding it useful. It's fully uncommented, lol, but pretty straightforward. If anyone has any questions about what it's doing I'd be glad to follow up on them.

Mostly it's just a quick n' dirty something to get going while waiting for Fogity to make a far better tool, which I believe is close from the last post I remember seeing, so I didn't want to throw too much time at the code (like error checking, or comments).
Lab Assistant
#6 Old 9th May 2015 at 7:44 PM Last edited by JustAnotherSimsFan : 10th May 2015 at 9:57 AM.
I didn't think Fogity was going to upload his extract tool, i thought he was just going to share the extracted files.
Lab Assistant
#7 Old 10th May 2015 at 1:05 AM
At first I just threw together a quick program to extract the tuning, mostly to prove to myself that I could, and also for your benefit. Then I decided to actually make a proper tool as there hadn't been any developments in that area elsewhere. But lately I have not really felt motivated to work on it, and as it seems others are on it, I will probably drop it. Will see if I drop everything or not. So do not count on my, I am not reliable.
Alchemist
#8 Old 10th May 2015 at 2:06 AM
Thank you for letting everyone know Fogity. If you aren't going to work on it then we'll continue on our end
Ms. Byte
staff: moderator
#9 Old 10th May 2015 at 1:54 PM
Thanks for the source, @Scumbumbo. I may play around with it.

Please do not PM me with mod, tutorial, or general modding questions or problems; post them in the thread for the mod or tutorial or post them in the appropriate forum.

Visit my blogs for other Sims content:
Online Sims - general mods for Sims 3
Offline Sims - adult mods for Sims 3 and Sims 4
Pettifogging Legalist!
retired moderator DELETED POST
10th May 2015 at 5:00 PM
This message has been deleted by plasticbox. Reason: obsolete
Alchemist
Original Poster
#10 Old 10th May 2015 at 5:39 PM
Quote:
Originally Posted by plasticbox
scumbumbo is your code derived from the source posted by velocitygrass or did you start from scratch? If it's the latter I shall split off the last couple posts into a new thread -- it might be confusing otherwise, particularly in case someone also starts to fiddle with velocitygrass' work.

@plasticbox - Sure, might be a good idea to split it off as it's not the same tool. The code is from scratch.
Lab Assistant
#11 Old 20th May 2015 at 4:54 AM
Thanks for this, I'm going to make a mod that prevents exes from calling and visiting once i get the XML to my Mod Workshop that I have installed!
Test Subject
#12 Old 22nd May 2015 at 2:31 PM
the program cant start cause Qt5Core.dll is missing, it popped out. why? sorry im new at this...
Pettifogging Legalist!
retired moderator
#13 Old 29th May 2015 at 7:46 PM
@itsclarkkent are you talking about velocitygrass’ tool (which this thread has been split off of) or about the one scumbumbo uploaded?

Stuff for TS2 · TS3 · TS4 | Please do not PM me with technical questions – we have Create forums for that.

In the kingdom of the blind, do as the Romans do.
Lab Assistant
#14 Old 29th May 2015 at 10:08 PM
Probably of velocitygrass' since the missing file is from QT, whose tool makes use of. Scumbumbo used .NET.

So, @itsclarkkent, forget about velocitygrass' tool as it's obsolete, it doesn't extract the files correctly any more, specially the ones from the packs.
Pettifogging Legalist!
retired moderator
#15 Old 31st May 2015 at 7:25 PM
I just extracted the combined tuning for 1.7.62 from forums.thesims.com/en_US/discussion/833758/1-7-62-combined-xml-and-tdescs (Base Game) and the /tun directory remained empty. I'm going to re-try later to make sure, but seeing as everything seemed to work flawlessly other than that (no errors or anything), could this be due to a bug?

Stuff for TS2 · TS3 · TS4 | Please do not PM me with technical questions – we have Create forums for that.

In the kingdom of the blind, do as the Romans do.
Alchemist
Original Poster
#16 Old 2nd Jun 2015 at 3:49 PM Last edited by scumbumbo : 3rd Jul 2015 at 11:39 AM.
Quote:
Originally Posted by plasticbox
the /tun directory remained empty..., could this be due to a bug?

Yeah, looks like I wasn't processing the Module ndoes in the XML that are used by the tun category so those files were skipped. I've uploaded a fixed EXE. Ran into the same issue doing the Tuning Description Browser and fixed it there so knew right away when I saw your post what the issue would be.

I didn't update the sources because it's such an easy fix, so for anyone playing with those just change the if statement in the for loop of ProcessFolder as so:
Code:
        For Each nCurrent As XmlNode In node.ChildNodes
            If nCurrent.Name = "I" Or nCurrent.Name = "M" Then
                ProcessFile(nCurrent, strTG)
            End If
        Next


ETA - Sources have been updated with the v1.1.0 update
Lab Assistant
#17 Old 2nd Jun 2015 at 4:37 PM
Quote:
Originally Posted by scumbumbo
Yeah, looks like I wasn't processing the Module ndoes in the XML that are used by the tun category so those files were skipped. I've uploaded a fixed EXE. Ran into the same issue doing the Tuning Description Browser and fixed it there so knew right away when I saw your post what the issue would be.

I didn't update the sources because it's such an easy fix, so for anyone playing with those just change the if statement in the for loop of ProcessFolder as so:
Code:
        For Each nCurrent As XmlNode In node.ChildNodes
            If nCurrent.Name = "I" Or nCurrent.Name = "M" Then
                ProcessFile(nCurrent, strTG)
            End If
        Next


I haven't looked at your code or used your program, but I think you might have another "bug" now.
For module tunings the resource type shouldn't be calculated in the same way as the other tunings, instead the value 0x03B33DDF should be used.
Pettifogging Legalist!
retired moderator
#18 Old 2nd Jun 2015 at 5:23 PM
Thank you both!

Stuff for TS2 · TS3 · TS4 | Please do not PM me with technical questions – we have Create forums for that.

In the kingdom of the blind, do as the Romans do.
Alchemist
Original Poster
#19 Old 3rd Jun 2015 at 6:10 AM
Quote:
Originally Posted by Fogity
I haven't looked at your code or used your program, but I think you might have another "bug" now.
For module tunings the resource type shouldn't be calculated in the same way as the other tunings, instead the value 0x03B33DDF should be used.

Thanks, I'd never noticed that. I'll fix that and get it uploaded yet tonight. Won't be a major change.

For those who've already done the extract and don't want to remove the tun files and reprocess - if you have Cygwin, here's a quick BASH command line you can run in that folder that will rename all of those files.
Code:
for fname in *; do mv $fname ${fname/266CA322/03B33DDF}; done
Alchemist
#20 Old 25th Jun 2015 at 7:32 PM
Maybe it's an idea to add this link: forums.thesims.com/en_US/discussion/833758/1-7-62-combined-xml-and-tdescs to the first post.
I had no idea where to look on the EA fora as I don't go there ever.

Overview all my creations (MTS and beyond): Download at once
FAQ and policy: Policy S2I
The School Project: The School Project
Over 500 new food items to come in Fast & Healthy Cooking Project: F&HCP
Pettifogging Legalist!
retired moderator
#21 Old 25th Jun 2015 at 7:53 PM
The links to the EA stuff are all in this thread: http://www.modthesims.info/showthread.php?t=533094 . They are also stickying them over there usually (and if they forget, one can ask them).

Stuff for TS2 · TS3 · TS4 | Please do not PM me with technical questions – we have Create forums for that.

In the kingdom of the blind, do as the Romans do.
Alchemist
#22 Old 25th Jun 2015 at 8:17 PM
Then maybe a link to that thread or something. A complete layman has really difficulties to find his way around here.

Overview all my creations (MTS and beyond): Download at once
FAQ and policy: Policy S2I
The School Project: The School Project
Over 500 new food items to come in Fast & Healthy Cooking Project: F&HCP
Field Researcher
#23 Old 29th Jun 2015 at 5:23 AM
Amazing! Thank you, scumbumbo!
I was wondering if it works with SPs, since there is no option for SP01 or SP02.
Alchemist
Original Poster
#24 Old 29th Jun 2015 at 5:32 AM
Quote:
Originally Posted by egureh
Amazing! Thank you, scumbumbo!
I was wondering if it works with SPs, since there is no option for SP01 or SP02.

It should, I've never unpacked those as I don't have them. The only thing that option button does is to set the Group number in the TGI, so if you unpack the SP stuff with the BG selected it will just assign group 0x00000000 and you'll have to change that manually if required for the mod. I'll have to add buttons for the new SPs. Maybe not today, maybe not tomorrow... but soon.
Field Researcher
#25 Old 29th Jun 2015 at 7:42 AM
Quote:
Originally Posted by scumbumbo
It should, I've never unpacked those as I don't have them. The only thing that option button does is to set the Group number in the TGI, so if you unpack the SP stuff with the BG selected it will just assign group 0x00000000 and you'll have to change that manually if required for the mod. I'll have to add buttons for the new SPs. Maybe not today, maybe not tomorrow... but soon.

Got it, I just need to rename the files after extracting, that's pretty okay for me. Thanks again! :D
Locked thread | Locked by: plasticbox Reason: See Downloads > Tools!
Page 1 of 4
Back to top