Sunday, February 17, 2013

Dismantling the UCScript Parser and Interface


The intent of this small series of posts is to not only show a breakdown of the UCScript Parser but to also look at a process of how it was made, some python parsing techniques used, and a bit on Kivy UI programming.

In this first part I am going to be going over the general data structure of the parser, the ideas behind it and a little on the intended Kivy UI. Along with this talking briefly on what defaultProperties are in an Unreal Script and some problems involved.

//Parser


The idea behind this parser is to be able to break down an Unreal Script into its default properties and have it return a dictionary of these properties and values to the UI. The way these returns are structured will be explain shortly.

//DefaultProperties


So to start off we should get a general idea of what we will be parsing which is the defaultProperties. Default properties, simply put, are just properties that relate to the script that they are in and can control aspects of it. (ie. In a vehicle script you are bound to find a property called groundSpeed which controls how fast a car can go while on the ground.)

For the parser we don't need to know all the names of these because there are hundreds of them but do need to know that they exist in the defaultProperties section. This section is usually located at the end of a script and is structured in the following way.

//Header
//Functions

defaultproperties
{
     PropertyName=Value
     PropertyName=Value
     ...
}

As you can see from above it is a fairly simple construction as all default properties consist of a Property Name and a Value. These are what we are going to be extracting from the script, the only trick though is that all Properties are not as simple as a name and a single value. This is where things will get a bit complex when parsing.

//Property Types





Looking at the diagram above, I have found there to be 6 types of properties, 3 of which are simple and the other 3 complex. They are differentiated this way because of the way the data will need to be parsed and stored into the dictionaries.

The first 3 types (string, bool, and int) are simple because they are only Property Name and Value entries. The way these will be structured in their dictionaries is:

type_defaultProperties = { PropertyName : Value }

The other 3 types is where it will be a bit messy, first in getting the value information we need and secondly being able to reassemble it when converting it back into code. Lets take a look at each type real quick.

Vector properties encapsulate anything that takes an X, Y, or Z value. Something that would fall into this category would be boneOffsets since they take a 3 vector input. The way we would want this structured is:

#Structure we will use in the parser for storing
vector_defaultProperties = { PropertyName: { X : 0.0, Y : 0.0, Z : 0.0} }

#Not used because of vagueness
# vector_defaultProperties = { PropertyName : [X, Y, Z] }

The reason behind storing these values in a nested dictionary is because some offsets or things to be considered vector properties may only have a Z value listed. This will allow it to store it correctly and then when trying to access it again it can be done more precisely as opposed to storing it in a list. This will all be explained more in depth when writing out the function for actually parsing these data values.

The second of the complex properties are the curve properties. The big challenge with these properties is making sure that all the information is collected property and organized in such a way that we can access any of the variables with great accuracy. Because of this a curve dictionary entry will be structured as:

curve_defaultProperties = { CurvePropertyName : { 0 : [ inVal, outVal ], 1 : [ inVal, outVal] } }

In this case, curve properties are stored as property:points. The property is equal to a dictionary of points, the keys being which point it is and their values a list of the inVal and outVal. This way we can rearrange points, change values and write back out with ease.

Finally are possibly the most complex I find for parsing which are class properties. These objects are simple to get but the real challenge will be in getting which properties they contain. Right now there is not a definite structure for them but it will post likely be a prefix added to the property type. The classes dictionary will contain a list of all the classes and link their values to the preprocessed defaultProperties. It will look something like this

class_defaultProperties = { ClassName : [ Link to property, Link to property ] } 

This way we will be able to reference classes and their internal properties that only apply to them.


//How the Data is Stored


Just a quick review of how the data will be returned to the UI.

Simple Property types include strings, bools and ints and their dictionaries will look like:

str_defaultProperties = { PropertyName : Value }
bool_defaultProperties = { PropertyName : Value }
int_defaultProperties = { PropertyName : Value }

Complex Property types include vectors, curves and classes.

vector_defaultProperties = { PropertyName : { X : 0.0, Y : 0.0, Z : 0.0 } }
curve_defaultProperties = { PropertyName : { 0 : [ inVal, outVal ], 1 : [ inVal, outVal ] } }
class_defaultProperties = { ClassName : [ ClassName_PropertyName ] }

All of these dictionaries will be sub-dictionaries of a main defaultProperties dictionary.

//Intended Kivy UI


Finally I'll talk a bit about the intended Kivy UI. The most important part of this UI in particular is the ability to be dynamic in nature, purely because we want this to work with any Unreal Script the user plugs into it. It should be able to receive any of the properties dictionaries and depends on the entries, generate widgets to control those inputs along with saving them back out. Another thing that will be added is a propertyInfo dictionary so that when a user is using this tool, they can click on a property name and see a brief description of it.

That's it for now, a lot of talk but in the next post I'll start going into detail on some of the code and how the main parser is constructed, along with beginning the UI.


No comments:

Post a Comment