It’s never a dull moment when making a game and this story represents one such incident. It was early Saturday morning and I thought I would get some stuff done regarding NPCs in my Main Town zone.
While I didn't need to load in the town to get the work done I always enjoy having the zone I’m working on being in the backdrop for inspiration. After giving it a few good minutes of attempting to load it was clear Unreal Engine 4(UE4) had crashed.
It had been several months since I had last loaded the Main Town zone. I also had not done anything that I’m aware that should have affected it. I did not make any changes to the meshes in the zone, scripting, or other files. It should have just loaded as always expect this time it refused.
My first thought was to restart UE4 itself. After that, I did a full system reboot. Sometimes you just get these strange unexplained issues and the zone in question has been the most difficult of them all with the way it was created in the first place. Most of the time they clear up on their own and you can move on with your day. This was also not one of those cases.
In an attempt to understand what was going wrong I wanted to see if this was an isolated incident or a wider-reaching problem. I have 22 zones made for my game so I went one by one checking to see if they would load or had any issue. Outside of just my Main Town, all the others loaded. While this was a bit of a relief it still left me wondering what was going on.
Not really understanding the issue I was having. I thought I would google it and see if I would get any leads on a possible direction to take things next. When you are googling things regarding game development there are quite a few things to keep in mind.
It’s doubtful you will find the same issue and need to think in terms of the broader problem.
If you do find people asking similar questions it’s doubtful any of them have been resolved.
Many people will share all kinds of theories and the original asker never confirmed if anything worked.
Someone else found a solution and it does not work for your situation.
Google tells you to go fish no results found.
If you can’t tell I’ve been down this road before with other issues. It’s not that uncommon to find someone asking a similar question 5 years ago on a game development form and it got zero replies or a solution was never discovered.
A lot of the time it is also how you paraphrase the search query. You might know the technical term for something but the general people who are having issues did not. The exact opposite could be true as well if only you know the correct word would you find the goods. People of all skill levels ask different kinds of questions.
My first search result gave me some technical UE4 information that I did not need to know, did not fully understand, and to be frank, did not need to know! It was 3 am and my brain did not need that!
My second search thankfully netted much better results. It had to do in general around zones not loading in the game you are making in UE4. Many of the leading theories are a file must have become corrupted and Unreal is unable to bypass the issue to finish loading the zone.
After looking at some logs in Unreal it still was not clear exactly what the main culprit was. The zone loading was hanging at several different parts issues. Sometimes it would be the same thing and other times it was at random it seemed at least at the time. This leads me to some false positives that were not the root issue in resolving what I was experiencing.
My game itself also has over 30k files and past 60 gigs. While a fair amount of those files have to do with UE4 itself there are still quite a lot. I suspected from what information I could gather one of my assets such as a mesh or texture had become corrupted or along those lines. While this could help narrow things down by a lot there are quite a lot of files an individual asset could be using that could be the culprit instead.
So I cloned my project so I could take the next few steps I had come up with in narrowing down the issue. While I already had a full backup I’d rather not have to dig that out of storage if things went wrong. It takes a while to move that large of a project.
To ensure my current working theory had some validity to it I deleted just about everything from the game. I wanted to confirm the zone could load and it was not a deeper unknown issue. It did load and I moved forward restoring all the files so I could track down the issue.
Just to rule out a bunch of files I did not suspect there is an issue with I deleted them out of my game. It sounds a bit sinful and even heartbreaking to be deleting that much work on the cloned version of the project but it helped narrow things down greatly.
After that, I simply followed the same process of deleting some folders, opening UE4, my cloned project, and the main town to see if it would load. While some game developers might cringe at doing it this way I was not looking to do this step clean I just needed to find the issue.
Eventually I got down to a folder with 121 files that amount to about a gig. In hopes of speeding the process up and resolving it then and there I pulled a copy of the same folder from a 6-month-old backup. I knew for sure the zone was working back then. Also, I knew I did not do any work on those files since then. Sadly using them to replace the other files still did not solve my issue. There was something more going on.
I then took an educated guess based on what files were remaining and narrowed it down to just 5 files that are the trouble makers. In the town, I have a couple of trees around it to add to the look and feel of things. They sadly were the issue and without them, the zone loads just fine.
I did not suspect the trees that were giving me an issue where being used in other zones. I however wanted to make sure. So I double-checked in UE4 to make sure that was the case and it was.
I then repeated this process a couple of times of fixing the issue on the copy of the project. I wanted to ensure I had found the solution. It was working. That there was not a better approach I could come up with at the time to save any losses.
It was now time to patch my original files and resolve this. For that, I opted to allow UE4 to do the cleanup of the unwanted files for me by telling it to delete each file. While UE4 sure takes its time removing files at the root and resolving some issues it was the best way to go forward.
Once it was done the zone started to load again. I tested loading the zone in a couple of different ways. This included loading it into the viewport and entering the zone through a portal as the player from another zone. Everything worked out fine.
That leaves the question of what was wrong? It could have been a caching issue, lighting rendering, foliage, shadows, or a couple of other things. Even with hindsight and the tools I know how to use, it’s still not 100% clear.
While everything is working now I’m not 100% out of the woods yet. I first need to replace some of the appearances that were lost in the town. I also need to re-render the lighting which I know for this zone takes about an hour and a half.
While I know it’s working in my development environment. I’m not quite certain yet if this fix resolves any issues that could crop when I go to build and my next alpha build. That would simply take me all weekend to do and I have plans to do that already in a few months anyway. As such I’m simply adding it to a list of things so if there are issues I have an idea where to look when it's time to do so.
Could This Have Been Avoided?
The answer is maybe. The current source control (version control) l have setup has failed me in this instance. I am however a solo indie developer making a game with a tiny yearly budget. There is only so much that can be done for now. Some complications add to the difficulty of having a fuller source control in places such as time, skill level to maintain, size, and cost.
While many developers are used to using something like GitHub and they do have some great things if you are a team or a smaller project. I believe with the size of my project I would already be pushing the enterprise tier. On top of that while UE4 can use GitHub there are a lot of things not saved when it comes to version control.
There is a good chance the issue I was having would still have been an issue. This could have happened months ago and there is a good chance it would have been requiring me to down the same path I did anyway.
As far as I’m aware and I could be wrong GitHub is also not that big when it comes to game development at the scale I’m making a game. At least from what I’ve seen. Due to how complex things get we tend to need a more heavy solution.
For that, there are things like Subversion (SVN) and Perforce. Which tend to require dedicated servers or to meet certain legal requirements to apply depending on the solution you are picking. Might be something I further explore later this year.
Other Posts:
Information
Screenshots were taken and content was written by @Enjar. Screenshots are from Unreal Engine 4.
Game roadmap.
