2014-09-07: v0.0.0.5: (Hopefully) fixed the issue with tool crashing on second save of file. Thanks to Kodek for reporting this. Added support for a STBL file so that STBL IDs will display the text in the data info and the tree. (For now you'll need to copy/paste the path to the STBL file in the input field for it.) Note: I also changed the order in the export, so that it now has the data first and the meta info second. (ETA: sorry, I accidentally uploaded the .exe a second time instead of the source code)
2014-09-03: v0.0.0.4: Support for drag and dropping files into the tool, passing a file to the executable (e.g. setting it as default program for .data files). It will also automatically load the file now after either browsing, dropping, or starting with it (though not if you just start the .exe and it recovers the last filename).
2014-09-03: v0.0.0.3: overwriting files now works (previously if the files existed, it wouldn't do anything since I forgot an additional check for that) Thanks to JettSchae for reporting this.
2014-08-31: v0.0.0.2: Editing values, cloning/deleting structs and list elements, saving of edited files
2014-08-25: updated link to Dropbox

I'm working on a tool that handles DATA (0x545AC67A) files.

I've now updated the tool to allow editing and saving (see "How to use" section on how and limitations). I haven't done much testing with edited files, so there are likely bugs. I did test editing the seed names for the CAS demo and that worked. The saving was also quite thoroughly tested. All files when loaded and left unchanged should be saved to a binary same file (except S4_545AC67A_006CA304_D89CB9186B79ACB7, which also doesn't read correctly). I won't get the game until Thursday, so I have only tested with the files available in the demo.



The data structure is complex enough that I didn't even know where to start implementing it in the s4pe wrapper (I'm capable of copy/pasting stuff to add a field, but this went beyond that and beyond my capabilities), so this is written in C++ as a quick and dirty prototype. Source code is attached and GPL3. It's a Qt Creator project, because I'm familiar with Qt from work. Unfortunately it makes the libraries the program depends on rather huge. The download is over 6MB, so I uploaded it elsewhere:

Sims 4 Data v0.0.0.5 (full version)

If you've downloaded the previous version, you can simply download the much smaller updated .exe (Sims4Data000xupdate.7z) attached to this post and overwrite the old one (the dependencies are the same and don't need to be replaced).

Caveat: I started looking at the data based on the info that s4pe already implements. The data field types are just my guesses. I'll make corrections as necessary. This will be changed to something useful once I add more columns.

How to install
- Use 7zip to unpack the folder/file. If you have the large package with the .dlls, it's enough to download the updated executable attached to this post and overwrite the old version.

How to start
- Just click the exe

How to use
There are several options to open a file. In all cases the DATA resource must have been previously exported with s4pe.
- Click "Browse" to select a DATA resource that was previously exported with s4pe.
- Pass the file path as a parameter to the executable (this is what happens when you set the .exe as default program for the .data file type)
- Drag a file onto the window
- You can also simply paste the path to the file in the line edit (this option doesn't remember the path)

Once you have set the file path in one of the above ways, the tool will mostly automatically load the file. In case of pasting the path, or if the tool restored the path from the last session, you'll have to manually click "Show File Info". (The loading of the data info might take a moment for the largest ones.)

- "Clear" clears everything
- "Save" saves the currently displayed file
- "Export All" is a convenience function that takes all .data files in the folder of the current file, then exports the meta info and data info as text files into a subfolder "as_text" (the folder needs to exist; if the files already exist nothing happens).
- Right-clicking on list elements and structs directly in the table will allow you to clone/delete them (you can't delete the last struct in the main tables, though)
- Double-clicking on values allows you to edit them (mode name is the exception)


Credit: Thanks to Rick, ChaosMageX, Kuree, Peter and Inge Jones and everyone else who helped with s4pe and in particular the DATA wrapper, which I used as a starting point.

Just a heads-up: 4shared seems to absolutely require having a Facebook, Google+, etc account -- I didn't see an option to download "just so". Or did I miss something? (I don't think I need this urgently, just wanted to let you know)

Hi velocitygrass,

I just committed a change to Data wrapper trying to fix the saving problem. The padding issue hasn't been solved entirely, but right now it works for small data files. As you can see in my commit, I'm trying to make it editable in grid view, which requires a lot of work. DATA looks very promising to tuning, but it's designed for reading rather than writing. We still have a lot of work to finish implementing DATA, but you're welcomed to join us on github.

Kuree

@plasticbox: Right, I'd forgotten that I'd signed up with a throwaway a long time ago for a reason. Thanks for the heads-up. I've put it on dropbox now.

@Kuree: I took a look at the changes you committed. As I said, I didn't know how to implement the data structure the way it's needed to work for grid view. I do have a few comments, though, from my observations of the data:

In the Data table unknown0C is the data type (same as the field data flags) and unknown10 is the field_size (for string tables this is 1).

FieldDataTypeFlags :
0x00000001 is type string, only used as type in data tables, which are referenced by fields of type 0x0000000B
0x00000009 => CAS Modifier Instance, i.e. Instance ID of a resource of type 0xC5F6763E
VFX = 0x0000000B, => this is actually a string, it's the offset to the actual string in the string table (i.e. the table of type 0x01)
0x0000000C => Mode Name (only used in censor tuning), this is string offset + hash of that string
0x0000000D => offset to structure based data (and in tables means it's data according to the structure table it has a pos stored to)
Unknown3 = 0x0000000E, => this is a list with the offset to the first element and then the number of elements
0x0000000F => two floats
RGBColor = 0x00000010, => three floats, mostly color, but e.g. the padding in the censor bone data is probably dimensions

As far as padding goes: I believe the DT header, the individual Data blobs, and the ST header all begin at byte zero, so the 16 bytes line before them is filled with zeroes as needed.

And yes, the format is definitely meant for reading more than writing.

Thank you so much. That code is based on a report provided by a user, so it might not be accurate and I didn't look at it carefully as I was busy fixing bugs. I will update the enum type to reflect the change.

Yes, there's always guesswork involved. Let's hope the DATA format will be included in the modding documentation that EA plans to release.

@velocitygrass,

I've imported the category and flag data that you provided as a sample into Color Magic 0.2.3 (not yet released, expect it Saturday 8/30) and it is amazing how various items are flagged.

Thanks a bunch for tracking the information down.

--Grant

In case someone is wondering, I'm still working on writing data files. The smallest ones come out fine, but I'm working my way through the issues with the other ones. I hope to have an update out this weekend.

@granthes: I'm glad the info helped and looking forward to seeing the flags and categories in action

ETA: I've added a new version with editing capabilities.

Tool works as intended and works overwriting files now in v0.0.0.3. Thank you for fixing that so quickly!

Thankfully it was an easy thing to fix. Thanks for letting me know it now works as expected!

Ah, I've found another small issue. I'm not sure if it's something you can fix or not, but v0.0.0.3 can't be set to be the "default opening program" for .data files. At least not for me. v0.0.0.2 did allow this, however.

Are you sure this worked with v0.0.0.2? The tool didn't recognize any parameters passed to it, which is what setting as default program does.

But I've added that now in v0.0.0.4. While I was at it, I also added drag/drop support, so you should be able to just drag .data files onto the tool. And it will then autoload the file, so the extra step of clicking "Show File Info" is not necessary anymore. (Except when pasting/typing a path and if you want to load the file from the last session.)

I downloaded this SimData Tool, and after getting errors that led me to download several dll files that were not on my computer, I am getting an "application unable to start correctly (0xc000007b)" error.

Edit: Help?

Hi velocitygrass,

I was trying to test it as a helper for S4PE, yet it only showed the files that S4PE passed in. Any ideas? And do you mind integrating it into S4PE project? :D

@cwurts00: If you needed to download dlls, you probably had the update version of the tool with only the .exe inside. If you haven't used a previous version, you'll need to download the large full package linked inside the description.

@Kuree: I hadn't even thought of adding it as a helper. I just tried it and the tool didn't show the UI and was just running endlessly in the background. After investigating it seems that S4PE doesn't export the temporary file the same way it does in the actual export to file functionality. The temporary file is corrupt (the same issue actually exists for the hex preview). What do you mean by "it only showed the files that S4PE passed in"? Does the UI come up for you?

I wouldn't mind integration into S4PE. Did you mean just adding a .helper file in which people only have to change the path or distribution with the package? I'm fine with anything, but wouldn't the Qt dependency and the fact that it's C++ be an issue (with the filesize and the way the solution is built)?

@velocitygrass: The UI came up, but it can show the details. Helper file in S4/3PE is very easy to implement and is independent, so C++ won't be any issue. I'll look at the corrupt file problem because sometimes I encountered that as well.

Hey velocitygrass! Thank You for this application, great work! I have small suggestion if that is not a problem, I would really like a shortcut to save file quicker (CTRL+S). That would make editing files a little bit quicker for me, especially when I need to save hundreds of them.

And a bug report I guess - when you save file second time without loading any other, the application will crash.

goood good would be if this could Search for Data files inside the main .package and then we could edit them and export them as separate packages.
This way it would be much easier to find what we want to edit

Your program is great!
I'm do not know English language very much, so I do not know if already asked, but a small note (I hope it's not hard to add).
Can you add a file-tree on the left of the main window? The reason - do not open each file individually. Just open the folder containing the data files in the file-tree and then view the files from the tree without opening / closing the program every time, or choosing to again file by clicking the "Browse".
I am not a programmer and do not know if it is difficult to add it, but if not - do, please.
P.S. I do not know how anyone, but I now import all the data files in a directory for a separate and yet look through almost straight (if not find an interesting group - miss). Without the file tree for me the process is slow. Of course, you can see the same thing in s4pe, but it also takes time with the transition between programs, etc.).
P.P.S. Sorry for my English. This text was translated with Google-translate.
P.P.P.S. I also had a crash when forgot to save the file or not, and hit the re-saving.

@Kuree: I'm still not sure what the problem is. Does editing not work? Version v0.0.0.2 of the tool had an issue with overwriting the existing file (which would be what the helper needs to do). Could that be the problem?

@Kodek: I hope I fixed the bug. Let me know if it still crashes. As for quicker saving: Do you want a shortcut instead of button or do you want to avoid the dialog coming up? I.e. would distinct Save/Save As buttons be okay? Shortcut shouldn't be a problem. I tend to be paranoid when overwriting things, but I see that it can get annoying. Out of curiosity: What changes are you making on so many files? Because this seems like something that might work better with a dedicated tool. Is it always the same change, just in different files?

@IceM: Yes, I'd like to be able to read from and write to package files. My first test failed to decompress the files and I haven't tried it again since then (though I assume this shouldn't be too hard to do). I can't say when I'll get to that.

@Kasany: Yes, I've also considered having a list of files for browsing a directory, which isn't quite a directory, though that should also be possible. I'm not sure yet how I'll handle it. If I get the package reading done quickly, that might make reading .data files directly almost obsolete. As for finding information in the data files: I find it much easier to do a text search on the info, which is why I implemented the "Export All" functionality to generate text files with all the data info. Those files are then easily searchable by keyword (and should also be easy to open in a text editor) with a normal search application (or even the windows search). The crash is hopefully fixed in the latest version 0.0.0.5. Let me know if it still crashes.

Great work, saving the same file second time or more doesn't crash application anymore. And I love the STBL integration, it's so much better now. Thank You!

Save and Save As buttons would make using the application easier, because anyway most of times you need to save the file with the same name. Shortcut for saving should be something for quick saving, so dialog probably should not come up then, but even if it would, the quicker way to save will be appreciated.

There are many files for tuning the same category of elements (like careers) and with scripting it will be much quicker and easier, but I wanted to change some things in that right now. It's just a small thing, nothing special.

Where did the thank you button go?

Im New at this, so very possible I am missing something, but not seeing an exe file here

sorry I found it! Thanks

I can't run this application :c
I had aleady download dozen of dlls. And this programm was finally dead :c

"The procedure entry point not found in library"
I know, this troubles are all because of Qt