As I did not see anything in the rules regarding switching between games, I did not think it would be an issue. Given the fact that I had very little time to reconstruct this due to my current class load, it is far from my best house. There were also structural issues and other limitations while rebuilding between games.