Hi there! You are currently browsing as a guest. Why not create an account? Then you get less ads, can thank creators, post feedback, keep a list of your favourites, and more!
MTS has all free content, all the time. Find out how YOU can help to keep it running. Tell me how...

XML Extractor for The Sims 4

by scumbumbo Posted 28th Aug 2015 at 6:15 AM - Updated 8th Dec 2018 at 12:51 AM by scumbumbo : Version 3.6.2 - December 7, 2018
 
119 Comments / Replies (Who?) - 91 Feedback Posts, 27 Thanks Posts
Hide Thanks Posts for this thread (Show only feedback)
Page 2 of 5
Lab Assistant
#26 Old 5th Dec 2015 at 12:36 AM
Broken in 64-bit patch
Test Subject
#27 Old 5th Dec 2015 at 12:39 AM
It's throwing an error at me too. Scumbumbo's been gone less than a month, hopefully it's not a permanent disappearance, as we don't currently have an alternative.
Field Researcher
#28 Old 5th Dec 2015 at 2:59 PM
Quote: Originally posted by jaybud4
It's throwing an error at me too. Scumbumbo's been gone less than a month, hopefully it's not a permanent disappearance, as we don't currently have an alternative.


I have the error too but it still extracts binary files, which can be extracted by velocitygrass' tool : http://modthesims.info/t/534316. I'm not sure everything is extracted correctly, but still better than nothing in the meantime
Forum Resident
#29 Old 5th Dec 2015 at 7:27 PM
I did the same thing.. but I had to convert the bnry file to xml before I could use velocity's tool

When an engineer says that something can't be done, it's a code phrase that means it's not fun to do.
Forum Resident
#30 Old 5th Dec 2015 at 7:49 PM
Similar to the method Neia mentioned.... here are the amateur hacky steps I took to get the files.

Use Scumbumbo's tool to extract the BNRY files to a folder.
Start with Full automatic Extract
You will get an error. But the binary files will be there.
exit the tool

Using the tool again, do single combined tuning file and browse to the binary file in your folder.

S4_62E94D38_00000000_F1352E740C4710BA.bnry (Base Game)
This will create the combined XML it takes some time... so just wait it out.

Now use Velocitygrass's tool to convert that combined XML into a filed structure.

When an engineer says that something can't be done, it's a code phrase that means it's not fun to do.
Mad Poster
#31 Old 5th Dec 2015 at 9:37 PM
I have patched Scumbumbo's code so it no longer throws an error. The problem was, the new expansion package file doesn't include any XML yet (resource 0x62E94D38). There was no error-catching in his code to work-around that. I imagine, next week when the expansion is released, they'll throw an updated package out there that has this resource so the XML for the new expansion can be unpacked as well.

For now, if you want to use his patched program, I have put it up on my Dropbox.

Source code: here
Just the Executable: here
Forum Resident
#32 Old 5th Dec 2015 at 11:18 PM
Sweet! Thanks!

When an engineer says that something can't be done, it's a code phrase that means it's not fun to do.
Pettifogging Legalist!
retired moderator
#33 Old 5th Dec 2015 at 11:36 PM
Last I heard, scumbumbo isn't gone, he's just very busy. I put a note in the first post that points to post #31 (Deaderpool patch). Thanks!

Stuff for TS2 · TS3 · TS4 | Please do not PM me with technical questions – we have Create forums for that.

In the kingdom of the blind, do as the Romans do.
Forum Resident
#34 Old 5th Dec 2015 at 11:40 PM
That's great news =)
All of your work is greatly appreciated Scumbumbo!
Don't leave us! well maybe for a little fallout.. but after that come back!

When an engineer says that something can't be done, it's a code phrase that means it's not fun to do.
Mad Poster
#35 Old 8th Dec 2015 at 4:19 PM Last edited by Deaderpool : 11th Dec 2015 at 4:39 PM.
I'm getting a crash after the Get Together update today since they added the new XML files for that update. I'm in the process of doing a debug on it and see if it's something else simple to fix. If so, I'll let plasticbox know when I post an update to it here so he can do his linking magic.

Edit: Seems like there's some sort of problem with the packed-XML files in the Sims 4 distribution. I get XML errors when I try to validate it...a missing tag or something. Not really something I'd be able to release a "patch" or anything to fix since it is a problem with the files themselves.

Edit: Edit: Ok, I figured-out the problem. Somewhere during the extraction of the binary file from the package and then the decompression of the XML from the binary, a null-character was being inserted into the final combined XML file. That caused an exception that prevented the individual XML files from being extracted. My theory is that it is in a localization of one of the dictionary entries and if I did an extraction without the text strings, it would have probably worked ok.

Anyway, I added some code to clean-up the XML files before running the individual XML extraction from the combined XML that removes any null characters. Seems to work fine and I can now see the EP02 XML files. Also, I added the Spooky Stuff and Get Together to the drop-down of Add-ons in the utility.

There are two zips attached. One includes the source files, the other is just the executable.

Thanks!
Attached files:
File Type: zip  XML Extractor for The Sims 4 - Includes Source.zip (634.7 KB, 88 downloads)
File Type: zip  XML Extractor for The Sims 4.zip (125.5 KB, 226 downloads)
Field Researcher
THANKS POST
#36 Old 10th Dec 2015 at 2:07 AM
Quote: Originally posted by Deaderpool
I'm getting a crash after the Get Together update today since they added the new XML files for that update. I'm in the process of doing a debug on it and see if it's something else simple to fix. If so, I'll let plasticbox know when I post an update to it here so he can do his linking magic.

Edit: Seems like there's some sort of problem with the packed-XML files in the Sims 4 distribution. I get XML errors when I try to validate it...a missing tag or something. Not really something I'd be able to release a "patch" or anything to fix since it is a problem with the files themselves.

Edit: Edit: Ok, I figured-out the problem. Somewhere during the extraction of the binary file from the package and then the decompression of the XML from the binary, a null-character was being inserted into the final combined XML file. That caused an exception that prevented the individual XML files from being extracted. My theory is that it is in a localization of one of the dictionary entries and if I did an extraction without the text strings, it would have probably worked ok.

Anyway, I added some code to clean-up the XML files before running the individual XML extraction from the combined XML that removes any null characters. Seems to work fine and I can now see the EP02 XML files. Also, I added the Spooky Stuff and Get Together to the drop-down of Add-ons in the utility.

The files are still located where they were previously, they've just been updated:
Source code: here
Just the Executable: here

Thanks!


I really wish to give you a hug. Thanks! :D
Test Subject
THANKS POST
#37 Old 12th Dec 2015 at 2:21 AM
Bless you, DP.
Instructor
#38 Old 13th Dec 2015 at 10:54 AM Last edited by Nukael : 13th Dec 2015 at 12:59 PM.
Whenever extracting, it uses one of the Asian languages as strings in the XML's and the Reference - all Strings (I don't know whether it's Japanese or Korean) and not the English version.

EDIT: It seems to be using the Chinese string file, which is the first string file in order. So maybe the extractor is just using the first string file it comes across? After deleting ALL string files, except the US English one, I'm getting English strings.

This space intentionally left blank...
Mad Poster
#39 Old 14th Dec 2015 at 11:22 PM
Quote: Originally posted by Nukael
Whenever extracting, it uses one of the Asian languages as strings in the XML's and the Reference - all Strings (I don't know whether it's Japanese or Korean) and not the English version.

EDIT: It seems to be using the Chinese string file, which is the first string file in order. So maybe the extractor is just using the first string file it comes across? After deleting ALL string files, except the US English one, I'm getting English strings.

Probably. I doubt when Scumbumbo wrote it that he had multiple string files. I have Spanish and German, but they were not used instead of English. It's probably only certain ones like the Asian one you saw.
Deceased
Original Poster
#40 Old 14th Jan 2016 at 5:14 AM
Quote: Originally posted by Deaderpool
Probably. I doubt when Scumbumbo wrote it that he had multiple string files. I have Spanish and German, but they were not used instead of English. It's probably only certain ones like the Asian one you saw.


Thanks for the patch @Deaderpool !!

I'll update the download. Not sure when I'll get it posted - probably not tonight, but in the next few days at least. The string language selection was added in v3.3.2, but you used v3.3.1 for your base - so I'll merge your changes into the latest version.

If there are any other fixes that are needed for this, speak up quickly while I'm here
Deceased
Original Poster
#41 Old 14th Jan 2016 at 8:13 AM
Version 3.3.3 has been uploaded fixing the identified bugs.
Test Subject
THANKS POST
#42 Old 14th Jan 2016 at 11:39 AM
Thanks for the update Scumbumbo!
Mad Poster
#43 Old 14th Jan 2016 at 9:38 PM
Quote: Originally posted by scumbumbo
Thanks for the patch @Deaderpool !!

I'll update the download. Not sure when I'll get it posted - probably not tonight, but in the next few days at least. The string language selection was added in v3.3.2, but you used v3.3.1 for your base - so I'll merge your changes into the latest version.

If there are any other fixes that are needed for this, speak up quickly while I'm here

All that I remember were the ones I already commented on and you merged in (or fixed better!) so nope, sounds good to me!
Pettifogging Legalist!
retired moderator
#44 Old 15th Jan 2016 at 6:07 AM
One can also extract binary JAZZ with this btw (I don't remember any more since when exactly that is binary; I think they changed it at the same time as XML):

* Find and export the JAZZ you want to extract with s4pe , and rename it to .bnry;
* Use the "Single Combined Tuning File" option to extract it;
* Since this won't contain any whitespace, use e.g. the "Pretty Print - with line breaks" option in npp (PLugins > XML Tools) or something similar in whatever other text editor you have to format it.

The game still reads JAZZ in XML form just fine, Maxis only deliver it as binary for performance reasons.

Stuff for TS2 · TS3 · TS4 | Please do not PM me with technical questions – we have Create forums for that.

In the kingdom of the blind, do as the Romans do.
Deceased
Original Poster
#45 Old 17th Jan 2016 at 6:56 PM
I must apologize for updating this so quickly after the v3.3.3 release fixing that bug, but I thought of a really nice new feature. Version 3.4.0 adds a file finding program which, I don't know about you, but I'm going to love!
Instructor
#46 Old 17th Jan 2016 at 11:17 PM
Hi @scumbumbo,

It's not doing automated full extraction for me.
Used to do. It throws an error, but hitting continue it extracts the compressed binary. Then it just stays there with no CPU usage at all.
Next I had to manually extract from the compressed binary (single file) it produced a (no white spaces) combined XML.
Next I had to run that XML again on single pack extract and it produced the the individual sub folders and XML files...

Finally I will do a debug next time and send you.

Thanks for the updates and new search tool which i didn't tried yet bu will soon.
Thanks Deaderpool for helping .
Deceased
Original Poster
#47 Old 17th Jan 2016 at 11:56 PM
Quote: Originally posted by azoresman
It's not doing automated full extraction for me.... I will do a debug next time and send you.


Yes, I'll definitely need more information as it's working fine for me. If you can let me know what exception is being thrown and where that'd be a minimum, but any additional information you can provide like which install version you're using (MSI, exes or compiled from sources) and what option settings you were using for the full extract would be helpful.
Instructor
#48 Old 18th Jan 2016 at 12:03 AM
Quote: Originally posted by scumbumbo
Yes, I'll definitely need more information as it's working fine for me. If you can let me know what exception is being thrown and where that'd be a minimum, but any additional information you can provide like which install version you're using (MSI, exes or compiled from sources) and what option settings you were using for the full extract would be helpful.


Just sent you PM with error code

Using latest 3.4.0 EXE didn't installed, Yesterday I tried 3.3.3 and it did the same error.

All options are selected, Strings ENG_US for output TGI+Names and overwrite in on.

I have updated Op.sys today just in case.
Deceased
Original Poster
#49 Old 18th Jan 2016 at 12:08 AM
Just to confirm -- Deaderpool's patched version was working for you? Because this looks like it might be the same error, meaning I didn't find all the changes he made, or something else that changed from version 3.3.1 (which he used for the patch version) and 3.3.2 (which was the latest version) made it break. I'll poke around at it the next chance I get.

Quote: Originally posted by azoresman
Just sent you PM with error code

Using latest 3.4.0 EXE didn't installed, Yesterday I tried 3.3.3 and it did the same error.

All options are selected, Strings ENG_US for output TGI+Names and overwrite in on.

I have updated Op.sys today just in case.
Instructor
#50 Old 18th Jan 2016 at 3:08 AM
Quote: Originally posted by scumbumbo
Just to confirm -- Deaderpool's patched version was working for you? Because this looks like it might be the same error, meaning I didn't find all the changes he made, or something else that changed from version 3.3.1 (which he used for the patch version) and 3.3.2 (which was the latest version) made it break. I'll poke around at it the next chance I get.


No, I just tried ver.3.3.3 yesterday and 3.4.0 today.
Didn't know there was a patch in the comments until yesterday.
Page 2 of 5