Home | Download | Discussion | Help | Site Map | New Posts | Sign in

Latest Site News

MTS Movie Night #3 - posted on 20th Oct 2017 at 10:54 PM
Replies: 36 (Who?), Viewed: 38554 times.
Page 2 of 2
Test Subject
#26 Old 8th Feb 2016 at 11:12 PM
Thanks A Lot!
Lab Assistant
#27 Old 13th Feb 2016 at 2:41 PM Last edited by DarkWalker : 25th Feb 2016 at 4:30 PM.
I've build a Python script that does what the batch file does, but by finding the folders inside the game files instead of using a fixed list of them, so it should be future-proof. You need Python 3.3 for it to work, but since that is the same requirement as for unpyc3.py, I don't see any issues here. Just put Decompile_nodeps.py in the same folder as unpyc3.py, edit it to reflect where you have the game installed and where you want the extracted scripts to go, and run it.

I don't think I will be providing much support for it, but for what matters I'm releasing this script into public domain, so do whatever you want with it.

Edit: just found out that my batch errors out with the stock unpyc3.py; sorry for that. I am using Aren's tweaked unpyc3, which is able to decompile a few files the stock one doesn't (including tunable.pyo). I recommend using that one, but in any case I've updated my batch so it can work with the stock unpyc3.py.
Download - please read all instructions before downloading any files!
File Type: zip Decompile_nodeps.zip (727 Bytes, 133 downloads) - View custom content
Description: Fixed the error with the stock unpyc3.py
Test Subject
#28 Old 2nd Mar 2016 at 7:52 PM Last edited by kk21 : 3rd Mar 2016 at 11:14 AM.
I'm trying to come up with a version of this for mac os, but I'm running into problems with the results of unpyc3. The gist of the process is 1. Copy base.zip core.zip simulation.zip into a temporary directory. 2. Navigate into that directory and unpack them. 3. Run the following:

for x in `find . | grep .pyo`; do python3 unpyc3.py $x > ${x/.pyo/.py}; rm $x; done

The problem is that, regardless of whether it was able to extract any code, every single file starts with *** Warning: file has wrong magic number *** . I'm not sure whether it's an indication that the process was unsuccessful, or an error I can simply remove.

Has this come up for Windows users as well?

EDIT:

If it really is safe just to comment this line out, the script to do it is:

for x in `find . | grep .pyo`; do python3 unpyc3.py $x | sed 's/\*\*\*/\#\*\*\*/' > ${x/.pyo/.py}; rm $x; done

All that does is changes the first instance of "***" to "#***", so that that line gets commented out. (It does this per line, so in my case there are exactly 14 instances where it matched on an additional line. All of them were print() statements, enclosed within strings, so at least for now, I'm not worrying about tweaking the regex to address this) That one line replaces the previous example, as the substitution happens before it writes the .py
Test Subject
#29 Old 3rd Mar 2016 at 9:56 AM Last edited by kk21 : 3rd Mar 2016 at 10:09 AM.
I think, also, it's not succeeding in extracting code that it looks like other people have been able to get. In fact, I think only a small fraction of the .pyo files had any success. The error that seems to occur on most of the .pyo files is AttributeError: 'SuiteDecompiler' object has no attribute 'GET_YIELD_FROM_ITER'

Any thoughts would be most appreciated. If I can get this working, I'd be most happy to provide a shell script companion to this .bat file.

EDIT: I grabbed the aforementioned Aren's tweaked unpyc3, linked to in the post above mine. *Significantly* better results, although the warning about the magic number is still there.
Test Subject
#30 Old 3rd Mar 2016 at 12:40 PM
Quote:
Originally Posted by DarkWalker
I've build a Python script that does what the batch file does, but by finding the folders inside the game files instead of using a fixed list of them, so it should be future-proof. You need Python 3.3 for it to work, but since that is the same requirement as for unpyc3.py, I don't see any issues here. Just put Decompile_nodeps.py in the same folder as unpyc3.py, edit it to reflect where you have the game installed and where you want the extracted scripts to go, and run it.

I don't think I will be providing much support for it, but for what matters I'm releasing this script into public domain, so do whatever you want with it.

Edit: just found out that my batch errors out with the stock unpyc3.py; sorry for that. I am using Aren's tweaked unpyc3, which is able to decompile a few files the stock one doesn't (including tunable.pyo). I recommend using that one, but in any case I've updated my batch so it can work with the stock unpyc3.py.


Very nice! I made a couple changes to the file to make it os-independent, and provide examples for both Mac and Windows. Thanks tons!

Code:
"""

Look through the following options, and uncomment (remove the # at the start of the line) one option each for folderGame and folderScripts.

If you are running Windows, and are happy with the options 'D:\\Origin\\Games\\The Sims 4' (folderGame)  and 'x:\\Scripts\\' (folderScripts), you don't have to do anything else!

This script was provided by DarkWalker on ModTheSims http://modthesims.info/showpost.php?p=4971513&postcount=27

The only modifications I've made are the addition of Mac/os-independent examples, and changing foldersGameplay to be os-independent

Big thanks to DarkWalker for this script!

"""



#########

"""
folderGame -- Where The Sims 4 is installed on your system
"""

# Windows Defaults
folderGame = 'D:\\Origin\\Games\\The Sims 4'

# Mac Defaults -- Mac users: comment out the line above this (under Windows Defaults), and uncomment the line below
#folderGame = "/Applications/The Sims 4.app/Contents" 


"""
folderScripts -- Where you want the decompiled scripts to reside
"""

# Windows Example
folderScripts = 'x:\\Scripts\\'

# Uncomment the following lines to have the extractor place the scripts in a folder "Scripts" beneath your home directory
#import os
#folderScripts = os.path.join(os.path.expanduser("~"),"Scripts")


# Uncomment the following lines to have the extractor place the scripts in a folder "Scripts" beneath your current directory
#import os
#folderScripts= os.path.join(os.getcwd(),"Scripts")


"""
Make sure you have only one line uncommented for each option (folderGame, folderScripts)
If you are using an option that uses os, such as os.path or os.getcwd(), make sure to uncomment `import os` *BEFORE* the line in question
You should not need to modify any of the lines following
"""

#########




import os, subprocess, zipfile, codecs, unpyc3

foldersGameplay = os.path.join(folderGame, 'Data','Simulation','Gameplay')

for i in ['base.zip', 'core.zip', 'simulation.zip']:
    file = zipfile.ZipFile(os.path.join(foldersGameplay, i))
    file.extractall(os.path.join(folderScripts, os.path.splitext(i)[0]))
# Walk down all script folders and decompile all script files
for root, subFolders, files in os.walk(folderScripts):
    for file in [f for f in files if os.path.splitext(f)[1].lower() == '.pyo']:
        pyoFile = os.path.join(root,file)
        pyFile = os.path.splitext(pyoFile)[0]+'.py'
        try:
            with codecs.open(pyFile, 'w', encoding='UTF-8') as o:
                o.write(str(unpyc3.decompile(pyoFile)))
            os.remove(pyoFile)
        except:
            os.remove(pyFile)
            print("Failed to decompile {}".format(pyoFile))
Lab Assistant
#31 Old 3rd Mar 2016 at 11:52 PM
Quote:
Originally Posted by kk21
The problem is that, regardless of whether it was able to extract any code, every single file starts with *** Warning: file has wrong magic number *** . I'm not sure whether it's an indication that the process was unsuccessful, or an error I can simply remove.

Has this come up for Windows users as well?

The error you mentioned should happen if you use any version of python other than 3.3 to run unpyc3. And, if you didn't change to the correct python version, you should still be getting that error even when using something based on my script, only the error should be going to the console instead of the decompiled files.

Also, given the way unpyc3 works, running it under any other version of python is bound to cause issues. Which might explain why you were getting half-decompiled files.
Test Subject
#32 Old 4th Mar 2016 at 2:52 PM Last edited by kk21 : 4th Mar 2016 at 3:02 PM.
Quote:
Originally Posted by DarkWalker
The error you mentioned should happen if you use any version of python other than 3.3 to run unpyc3. And, if you didn't change to the correct python version, you should still be getting that error even when using something based on my script, only the error should be going to the console instead of the decompiled files.

Also, given the way unpyc3 works, running it under any other version of python is bound to cause issues. Which might explain why you were getting half-decompiled files.


Well, I *thought* I had 3.3, but I had some difficulties w/homebrew and had to reinstall, and it looks like I only had 3.5

I managed to install 3.3 (homebrew no longer had the formula), and it was a serious pain, but success! However, now I'm on to a new hurdle: if I eg, try to import (as in the Absolute Beginners tutorial) sims4.commands, I get the error,
Code:
ImportError: No module named '_trace'


Has this come up for you?

(Thanks so much for the help!)

EDIT: I tried using placeholder files for the missing modules, to see what happened if it could get past that step, and it seems it then moves on to complaining about a syntax error in graph_algos.py, where it uses `break` outside of a loop

Code:
  File "./graph_algos.py", line 50
    break
    ^
SyntaxError: 'break' outside loop


Do you think the file incorrectly decompiled, or some other issue?

Code:
import collections
__all__ = ['strongly_connected_components', 'topological_sort']

def topological_sort(node_gen, parents_gen_fn):
    sccs = strongly_connected_components(node_gen, parents_gen_fn)
    result = []
    for scc in sccs:
        if len(scc) != 1:
            raise ValueError('Graph has a strongly connected cycle ({})'.format(','.join([str(item) for item in scc])))
        result.append(scc[0])
    return result

def strongly_connected_components(node_gen, parents_gen_fn):
    index = 0
    indices = {}
    lowlinks = {}
    stack = []
    stack_members = set()
    nodes = set(node_gen)
    sccs = []
    for node in nodes:
        while node not in indices:
            index = _strongconnect(node, sccs, nodes, parents_gen_fn, indices, lowlinks, stack, stack_members, index)
    return sccs

def _strongconnect(node, sccs, nodes, parents_gen_fn, indices, lowlinks, stack, stack_members, index):
    indices[node] = index
    lowlinks[node] = index
    index += 1
    stack.append(node)
    stack_members.add(node)
    parents = parents_gen_fn(node)
    if parents is not None:
        for parent in parents:
            if parent not in nodes:
                pass
            if parent not in indices:
                index = _strongconnect(parent, sccs, nodes, parents_gen_fn, indices, lowlinks, stack, stack_members, index)
                lowlinks[node] = min(lowlinks[node], lowlinks[parent])
            else:
                while parent in stack_members:
                    lowlinks[node] = min(lowlinks[node], indices[parent])
    if lowlinks[node] == indices[node]:
        scc = []
        sccs.append(scc)
        v = stack.pop()
        stack_members.remove(v)
        scc.append(v)
        if v is node:
            break
            continue
    return index
Lab Assistant
#33 Old 4th Mar 2016 at 4:51 PM
Quote:
Originally Posted by kk21
Well, I *thought* I had 3.3, but I had some difficulties w/homebrew and had to reinstall, and it looks like I only had 3.5

I managed to install 3.3 (homebrew no longer had the formula), and it was a serious pain, but success!

Not completely sure about Mac, but on both Windows and Linux you can have multiple versions of Python installed, you just need to call the right version to use the decompiler (and to compile files for use in Sims 4).

Quote:
Originally Posted by kk21
However, now I'm on to a new hurdle: if I eg, try to import (as in the Absolute Beginners tutorial) sims4.commands, I get the error,
Code:
ImportError: No module named '_trace'


Has this come up for you?

What are you attempting to do, compile a file for use in Sims 4? It will give an error, but the compiled file (if using Python 3.3 to compile) should be perfectly usable. Something like "python -O -m <filename>.py" should generate the compiled .pyo files you need inside a __pycache__ folder, just remove the extraneous ".cpython-33" added to the filename and use them.

BTW, the decompiled files will likely have many errors. Nothing we can do about it, apart from being on the lookout for things like while loops used in place of if conditionals and other such issues.
Test Subject
#34 Old 4th Mar 2016 at 7:01 PM
Quote:
Originally Posted by DarkWalker
Not completely sure about Mac, but on both Windows and Linux you can have multiple versions of Python installed, you just need to call the right version to use the decompiler (and to compile files for use in Sims 4).


Yes, that's correct. The current Mac OS ships w I think 2.6 and 2.7. The package on the official python site for 3.3 is incompatible with OSX El Capitan (the current OS). Homebrew offers a package manager, intended to mirror the functionality of apt-get on Ubuntu/Debian. Unfortunately, the only version available there was 3.5. With homebrew, you can also use your own 'formula', and I was able to find one for 3.3. (here) Someone repeating these steps may have problems if they had a different version of python3 which they'd then removed. One solution is to create a symbolic link to the version you have installed.


Quote:
Originally Posted by DarkWalker
What are you attempting to do, compile a file for use in Sims 4? It will give an error, but the compiled file (if using Python 3.3 to compile) should be perfectly usable. Something like "python -O -m <filename>.py" should generate the compiled .pyo files you need inside a __pycache__ folder, just remove the extraneous ".cpython-33" added to the filename and use them.

BTW, the decompiled files will likely have many errors. Nothing we can do about it, apart from being on the lookout for things like while loops used in place of if conditionals and other such issues


I was trying to use IPython to interactively test and develop. As it is, it can't import the module, which makes it tricky to use it interactively In these situations, do you guys then just modify the source code of the modules you're using to make it work in its uncompiled form?

Thanks again!
Lab Assistant
#35 Old 5th Mar 2016 at 12:24 AM
I can't tell about others, but I use external tools to just guarantee that there is no syntax error and test things in game. Being able to reload scripts without restarting the game helps, of course (http://modthesims.info/showthread.php?t=534451).
Test Subject
DELETED POST
13th Jun 2016 at 12:01 AM Last edited by Joyius : 13th Jun 2016 at 12:24 AM.
This message has been deleted by Joyius. Reason: figured it out
Test Subject
Original Poster
#36 Old 6th Jun 2017 at 3:58 PM
Nobody has posted in awhile, I asked a mod about the rules on that, not got a reply yet, if I get yelled at *Shrugs*, Okay I accidentally posted a broken version of v7.3, for all of you who downloaded it, please re download it, as I re posted the fixed version. As always check the spoiler Changelog at the very bottom is the latest information on whats new, etc
Test Subject
#37 Old 19th Jun 2017 at 5:18 PM
Quote:
Originally Posted by pcgeekri
Hopefully this will help you, read the instructions here:
http://www.modthesims.info/showthread.php?t=533907

If you left click on the "unpyc3.py" your browser will open it and you see text so instead right click on it and "Save Target As" and leave the filename as "unpyc3.py".

Darkkitten30 *hugz* above mentioned her (I'm guessing a her, oops) installs were on the "D" drive. Fortunately so are mine.
Probably a good idea to install Python on the same drive as "The Sims 4". Python was installed in the D:\Python33 folder. During the installation just changed the "C" to a "D". This is the folder where the unpyc3.py file was placed, in that same D:\Python33 folder with the Python executable.

The Batch File must be placed in the root directory of where your game resides. Fortunately Both Python and Sims 4 are both installed on the "D" drive so the batch file was placed in the root directory of my "D" drive or in D:\ . And ran or double-clicked at that location.

Hopefully this helps you.

PcGeek

omg, that avatar is still on my account? Haven't seen that in years! lmao Member since 2005, my how time flies.


I realize i sound dumb but at least i'm making an effort to learn.
"Batch file placed in root directory of where the game resides" Would that be under origingames/sims4/game/bin ect or origingames/sims4/data/simulation ect?
I know how to recolor and download mods, i'm really trying to learn how everything works so i can start modding. Baby steps love, so windows 10, origin game, how do i find the root directory you speak of? Teach me great master
Page 2 of 2
Back to top