Hi there! You are currently browsing as a guest. Why not create an account? Then you get less ads, can thank creators, post feedback, keep a list of your favourites, and more!
Test Subject
Original Poster
#1 Old 16th Sep 2018 at 3:22 PM Last edited by nixylvarie : 16th Sep 2018 at 4:00 PM.
E0FF Compression Type?
I'm working on a python script which unpackages Sims 4 DBPF files, fixes mod conflicts in XML files, and repackages them.

So far the unpackaging part of the script works perfectly, unless one of the files has a E0FF compression type.

Some output from an experiment with ClientDeltaBuild0.package:

Code:
>>> dbpf.debugFIndex(False, False, limitSize=10)
[FIELD]	 [0:TYPE] [1:GRP ] [2:I_HI] [3:I_LO] [4:OFST] [5:SIZE] [6:REAL] [7:CMPR]
[0] 	 101ACA81 00000000 00000000 00000000 11B30C3C 98790280 64620500 425A0100
[1] 	 F3C4206B 00000000 0ED99D43 E91AE499 182B9B32 08400080 34710000 425A0100
[2] 	 5DE7D001 00000000 53273A2F 3AB5D421 AFD7E913 C8000080 54010000 425A0100
[3] 	 340FD101 01000100 A3F07916 F8F0423E 00000000 00000080 00000000 E0FF0100     <---
[4] 	 44504ABC 00000000 F266B4D3 18C65E9F 9AE11D3C 5E010080 F9040000 425A0100
[5] 	 F3C4206B 00000000 F266B4D3 18C65E9F 065AF721 23540080 77A00000 425A0100
[6] 	 5DE7D001 00000000 5B4E7613 24316E81 86F5E613 F5000080 A8010000 425A0100
[7] 	 340FD101 00000100 07C1FD5B 2BEB6D31 47B66404 48020080 80040000 425A0100
[8] 	 7B3E3A05 00000000 9CC4597C 1A7F46A2 DD077214 73010080 C3020000 425A0100
[9] 	 49185A01 EB790B00 B6B547E6 DFE67BCE 003EFF05 7A600080 5DD40000 425A0100


What is this and what do I do with it?

EDIT: After looking more closely, I realized the file is empty. (*facepalm*) That explains the decompression error, but what does the E0FF mean?
Advertisement
Deceased
#2 Old 16th Sep 2018 at 6:18 PM
Deleted record. Here's the compression types from EA's binary template documentation for the package files:
Code:
string ReadCompressionType(UINT16& t)
{
    switch(t)
    {
    case 0x0000: return "Uncompressed";
    case 0xfffe: return "Streamable compression";
    case 0xffff: return "Internal compression";
    case 0xffe0: return "Deleted record";
    case 0x5a42: return "ZLIB";
    }
    return "Unknown"; 
}
Test Subject
Original Poster
#3 Old 16th Sep 2018 at 7:01 PM Last edited by nixylvarie : 16th Sep 2018 at 8:16 PM.
Quote: Originally posted by scumbumbo
Deleted record. Here's the compression types from EA's binary template documentation for the package files:
Code:
string ReadCompressionType(UINT16& t)
{
    switch(t)
    {
    case 0x0000: return "Uncompressed";
    case 0xfffe: return "Streamable compression";
    case 0xffff: return "Internal compression";
    case 0xffe0: return "Deleted record";
    case 0x5a42: return "ZLIB";
    }
    return "Unknown"; 
}


Oh, thanks! Streamable would be some kind of RLE, right? What's 0xFFFF?

Are there any other documentation files I can look at?
Deceased
#4 Old 16th Sep 2018 at 8:48 PM
Here's that binary template I quoted from earlier, if that could be of help. There was some stuff on a wiki somewhere (sorry, don't remember where) discussing the various compression types and I used that heavily in decompression for the internal compression. From what I can remember the streamable compression type is never used - don't remember if I implemented it in my programs but I think I skipped it. s4pi likely does handle that as it's more complete in what it's designed to do.

ETA - Regarding the streaming compression - I may also never run into that as I was mostly concerned with decompressing XML types of resources.
Attached files:
File Type: rar  PackageTemplate.rar (1.4 KB, 22 downloads) - View custom content
Deceased
#5 Old 16th Sep 2018 at 9:07 PM
Here's the VB code I used to handle the two compression types I handled, in a spoiler as it's fairly long.
Deceased
#6 Old 16th Sep 2018 at 9:10 PM
And the document I used to figure out that internal decompression was right on this site

http://modthesims.info/wiki.php?tit...BPF/Compression
Test Subject
Original Poster
#7 Old 16th Sep 2018 at 10:24 PM
Quote: Originally posted by scumbumbo
And the document I used to figure out that internal decompression was right on this site

http://modthesims.info/wiki.php?tit...BPF/Compression


What's with the DDS files? I unpackaged and decompressed several but couldn't view them. I saw another thread where they mentioned skipping a two-byte header before decompressing a .DDS...don't remember where that was. It was a very old thread. It's not like I'll need to use them - I'm just using the XML files - but it'd be nice to have a sense of completeness.

Thank you for all your help, by the way. I'm enjoying this project so much I forgot about the whole actually making mods part. Messing around with all these file formats and encodings hurts my head, but I'm enjoying the challenge. I learned a lot about compression in the past 48 hours!
Deceased
#8 Old 17th Sep 2018 at 5:47 AM
Quote: Originally posted by nixylvarie
What's with the DDS files?

The most I do with DDS files with the projects I've done is to compare hashes of the resource themselves (in the Package Conflict Detector), I never actually decompress them - if the two compressed resources hash to the same result then I'm happy. The only reason I had to work out the compression for the zlib and internal compression stuff was in the process of writing the XML Extractor and read the string tables.
Back to top