CRYENGINE CINEBOX

I am giving a talk at SIGGRAPH 2012 entitled ‘Film/Game Convergence: What’s Taking So Long?‘ where I discuss the inherent differences between games and film and go over a few case studies of projects that attempted to use a game engine for film previs. I also talk a bit about the development of our CINEBOX application, the decisions we had to make, and how we dealt with many of the issues previous attempts have run into.

STUDIO WORKSHOPS

I will be giving two more Studio Workshops this year, the first is a followup to last year’s Introduction to Python, entitled ‘Python Scripting in Maya‘. The other workshop is ‘Building a Game Level‘, which is the same basic workshop I gave last year where I show people how to make a playable game level in CryEngine in an hour. Studio Workshops are hands-on sessions where each attendee has a computer and follows along with the instructor. It’s a great chance for people of all ages to learn new things.

I am still finding my feet in Maya, on my project, some files have grown to 800mb in size. Things get corrupt, hand editing MAs is common; I am really learning some of the internals.

In the past week I have had to do a lot of timeline walking to switch coord spaces and get baked animations into and out of hierarchies. In 3dsMax you can do a loop and evaluate a node ‘at time i’, and there is no redraw or anything. I didn’t know how to do this in Maya.

I previously did this with looping cmds.currentTime(i) and ‘walking the timeline’, however, you can set the time node directly like so: cmds.setAttr(“time1.outTime”, int(i))

Unparenting a child with keyed compensation (1200 frames)
10.0299999714 sec – currentTime
2.02 sec – setAttr

There are some caveats, whereas in a currentTime loop you can just cmds.setKeyframe(node), I now have to cmds.setKeyframe(node, time=i). But when grabbing a matrix, I don’t need to pass time and it works, I don’t think you can anyway.. I guess it gets time from the time node.

Here’s a sample loop that makes a locator and copies a nodes animation to world space:

#function feeds in start, end, node
	if not start: start = cmds.playbackOptions(minTime=1, q=1)
	if not end: end = cmds.playbackOptions(maxTime=1, q=1)
	loc = cmds.spaceLocator(name='parentAlignHelper')
	for i in range(start, (end+1)):
		cmds.setAttr("time1.outTime", int(i))
		matrix = cmds.xform(node, q=1, ws=1, m=1)
		cmds.xform(loc, ws=1, m=matrix)
		cmds.setKeyframe(loc, time=i)

Maybe it’s me, but I often find myself parsing weird ascii text files from others. Sometimes the authors knew what the data was and there’s no real markup. Take this joint list for example:

143 # bones
root ground
-1
0 0 0
root hips
0
0 0.9512207 6E-08
spine 1
1
4E-08 0.9522207 1.4E-07
spine 2
2
3E-07 1.0324 8.3E-07
spine 3
3
5.6E-07 1.11357 1.53E-06
spine 4
4
8.2E-07 1.194749 2.22E-06
head neck lower

So the first line is the number of joints then it begins in three line intervals stating from the root outwards: joint name, parent integer, position. I used to make a pretty obtuse loop using a modulus operator. Basically, modulus is the remainder left over after division. So X%Y gives you the remainder of X divided by Y; here’s an example:

for i in range(0,20+1):
	if i%2 == 0: print i
#>> 0
#>> 2
#>> 4
#>> 6
#>> 8
#>> 10

The smart guys out there see where this is goin.. so I never knew range had a ‘step’ argument. (Or I believe I did, I think I actually had this epiphany maybe two years ago, but my memory is that bad.) So parsing the above is as simple as this:

jnts = []
for i in range(1,numJnts*3+1,3):
	jnt = lines[i].strip()
	parent = int(lines[i+1].strip())
	posSplit = lines[i+2].strip().split(' ')
	pos = (float(posSplit[0])*jointScale, \
	float(posSplit[1])*jointScale, float(posSplit[2])*jointScale)
	jnts.append([jnt, parent, pos])

Thanks to phuuchai on #python (efnet) for nudging me to RTFM!

I gave a workshop/talk at SIGGRAPH geared toward introducing people to Python. There were ~25 people on PCs following along, and awkwardly enough, many more than that standing and watching. I prefaced my talk with the fact that I am self-taught and by no means an expert. That said, I have created many python tools people use every day at industry-leading companies.

Starting from zero, in the next hour I aimed to not only introduce them to Python, but get them doing cool, usable things like:

• Iterating through batches/lists
• Reading / writing data to excel files
• Wrangling data from one format to another in order to create a ‘tag cloud’

Many people have asked for the notes, and I only had rough notes. I love Python, and I work with this stuff every day, so I have had to really go back and flesh out some of what I talked about. This tutorial has a lot less of the general chit-chat and information. I apologize for that.

Installation / Environment Check

Let’s check to see that you have the tools properly installed. If you open the command prompt and type ‘python’ you should see this:

So Python is correctly installed, for the following you can either follow along in the cmd window (more difficult) or in IDLE, the IDE that python ships with (easier). This can be found by typing IDLE into the start menu:

Variables

Variables are pieces of information you store in memory, I will talk a bit about different types of variables.

Strings

Strings are pieces of text. I assume you know that, so let’s just go over some quick things:

string = 'this is a string'
print string
#>>this is a string
num = '3.1415'
print num
#>>3.1415

One thing to keep in mind, the above is a string, not a number. You can see this by:

print num + 2
#>>Traceback (most recent call last):
#>>  File "basics_variables.py", line 5, in
#>>    print num + 2
#>>TypeError: cannot concatenate 'str' and 'int' objects

Python is telling you that you cannot add a number to a string of text. It does not know that ‘3.1415’ is a number. So let’s convert it to a number, this is called ‘casting’, we will ‘cast’ the string into a float and back:

print float(num) + 2
#>>5.1415
print str(float(num) + 2) + ' addme'
#>>5.1415 addme

Lists

Lists are the simplest ways to store pieces of data. Let’s make one by breaking up a string:

txt = 'jan tony senta michael brendon phillip jonathon mark'
names = txt.split(' ')
print names
#>>['jan', 'tony', 'senta', 'michael', 'brendon', 'phillip', 'jonathon', 'mark']
for item in names: print item
#>>jan
#>>tony
#>>senta
#>>michael
...

Split breaks up a string into pieces. You tell it what to break on, above, I told it to break on spaces txt.split(‘ ‘). So all the people are stored in a List, which is like an Array or Collection in some other languages.
You can call up the item by it’s number starting with zero:

print names[0], names[5]
#>>jan phillip

TIP: [-1] index will return the last item in an array, here’s a quick way to get a file from a path:

path = 'D:\\data\\dx11_PC_(110)_05_09\\Tools\\CryMaxInstaller.exe'
print path.split('\\')[-1]
#>>CryMaxInstaller.exe

Dictionaries

These store keys, and the keys reference different values. Let’s make one:

dict = {'sascha':'tech artist', 'harry': 142.1, 'sean':False}
print dict['sean']
#>>False

So this is good, but these are just the keys, we need to know the values. Here’s another way to do this, using .keys()

dict = {'sascha':'tech artist', 'harry': 142.1, 'sean':False}
for key in dict.keys(): print key, 'is', dict[key]
#>>sean is False
#>>sascha is tech artist
#>>harry is 142.1

So, dictionaries are a good way to store simple relationships of key and value pairs. In case you hadn’t notices, I used some ‘floats’ and ‘ints’ above. A float is a number with a decimal, like 3.1415, and an ‘int’ is a whole number like 10.

Creating Methods (Functions)

A method or function is like a little tool that you make. These building blocks work together to make your program.

Let’s say that you have to do something many times, you want to re-use this code and not copy/paste it all over. Let’s use the example above of names, let’s make a function that takes a big string of names and returns an ordered list:

def myFunc(input):
	people = input.split(' ')
	people = sorted(people)
	return people
txt = 'jan tony senta michael brendon phillip jonathon mark'
orderedList = myFunc(txt)
print orderedList
#>>['brendon', 'jan', 'jonathon', 'mark', 'michael', 'phillip', 'senta', 'tony']

Basic Example: Create A Tag Cloud From an Excel Document

So we have an excel sheet, and we want to turn it into a hip ‘tag cloud’ to get people’s attention.
If we go to http://www.wordle.net/ you will see that in order to create a tag cloud, we need to feed it the sentences multiple times, and we need to put a tilde in between the words of the sentence. We can automate this with Python!

First, download the excel sheet from me here: [info.csv] The CSV filetype is a great way to read/write docs easily that you can give to others, they load in excel easily.

file = 'C:\\Users\\chris\\Desktop\\intro_to_python\\info.csv'
f = open(file, 'r')
lines = f.readlines()
f.close()
print lines
#>> ['always late to work,13\n', 'does not respect others,1\n', 'does not check work properly,5\n', 'does not plan properly,4\n', 'ignores standards/conventions,3\n']

‘\n’ is a line break character, it means ‘new line’, we want to get rid of that, we also want to just store the items, and how many times they were listed.

file = 'C:\\Users\\chris\\Desktop\\intro_to_python\\info.csv'
f = open(file, 'r')
lines = f.readlines()
f.close()
dict = {}
for line in lines:
	split = line.strip().replace(' ','~').split(',')
	dict[split[0]] = int(split[1])
print dict
#>>{'ignores~standards/conventions': 3, 'does~not~respect~others': 1, 'does~not~plan~properly': 4, 'does~not~check~work~properly': 5, 'always~late~to~work': 13}

Now we have the data in memory in an easily readable way, let’s write it out to disk.

output = ''
for key in dict.keys():
	for i in range(0,dict[key]): output += (key + '\n')
f = open('C:\\Users\\chris\\Desktop\\intro_to_python\\test.txt', 'w')
f.write(output)
f.close()

There we go. In one hour you have learned to:

• Read and write excel files
• Iterate over data
• Convert data sets into new formats
• Write, read and alter ascii files

If you have any questions, or I left out any parts of the presentation you liked, reply here and I will get back to you.

As many of you know, I feel the whole ‘autorigging’ schtick is a bit overrated. Though Bungie gave a great talk at GDC09 (Modular Procedural Rigging), Dice was to give one this year at SIGGRAPH (Modular Rigging in Battlefield 3), but never showed up for the talk.

At Crytek we are switching our animation dept from 3dsMax to Maya. This forces us to build a pipeline there from scratch; in 3dsMax we had 7 years of script development focused on animation and rigging tools. So I am looking at quite a bit o Maya work. The past two weeks focusing on a ‘rigging system’ that I guess could be thought of as ‘procedural’ but is not really an ‘autorigger’. My past experience was always regenerating rigs with mel cmds.

Things I would like to solve:

• Use one set of animator tools for many rigs – common interfaces, rig block encapsulation (oh god i said ‘block’)
• Abstract things away, thinking of rigging ‘units’ and character ‘parts’ instead of individual rig elements, break reliance on naming, version out different parts
• Be fluid enough to regenerate the ‘rigging’ at any time

First Weekend: Skeleton ‘Tagging’

I created a wrapper around the common rigging tools that I used, this way, when I rigged, it would automagically markup the skeleton/elements as I went. This looked like so:

The foundation of this was marking up the skeleton or cons with message nodes that pointed to things or held metadata.  This was cool, and I still like how simple it was, however, it didn’t really create the layer of abstraction I was after. There wasn’t the idea of a limb that I could tell to switch from FK to IK.

Second Weekend: Custom Nodes

That Bungie talk got a lot of us all excited, Roman went and created a really cool custom node plugin that does way more than we spec’d it out to do. I rewrote the rigging tools to create ‘rigPart’ nodes, which could be like an IK chain, set of twist joints, expression, or constraint. These together could form a ‘charPart’ like an arm or leg. All these nodes sat under a main ‘character’ node. I realize that many companies abstract their characters into ‘blocks’ or ‘parts’, but I had never seen a system that had another layer underneath that. Roman also whipped up a way that when an attr on a customNode changes, you could evaluate a script. So whether it’s a human arm or alien tentacle arm, the ‘charPart’ node can have one FK/IK enum. I am still not sure if this is a better idea, because of the sheer legwork involved..

Third Weekend: A Mix of Both?

So a class like ‘charParts.gruntLeg()’ not only knew how to build the leg rigParts, but also only the leg ‘rigging’ if needed. This works pretty well, but the above was pretty hard to read. I took some of my favorite things about the tree-view-based system and I created a ‘character’ outliner of sorts. This made it much easier to visualize the rigParts that made up individual ‘systems’ of the character, like leg, spine, arm, etc. I did it as a test, but in a way that I easily swap it out with the treeWidget in the rigging tools dialog.

So how do you guys solve some of these issues?

I am volunteering again in the Studio; giving three small talks at SIGGRAPH, drop me a line if you will be in Vancouver.

Rigging Characters for CryENGINE

How to rig, skin, and export a character for CryENGINE 3. Topics include physics setup, building characters from many skinned meshes, and creating Character Definitions and Character Parameter files. These rigging basics are applicable to most run-time game engines.

Introduction to Python Scripting

In this introduction to Python, a powerful scripting language used by many 3D applications, attendees learn the basics and explore small example scenarios gleaned from actual game and film productions. The sessions are taught in a way that should empower attendees to immediately begin creating time-saving python scripts and applications.

World Creation in CryENGINE

Have you ever wanted to make a videogame? This session shows how to build a small level in the freely available CryENGINE 3 SDK. Topics include: world building and tools (FlowGraph, CryENGINE’s visual scripting language, and Trackview, the camera sequencing and directing tools). In less than an hour, attendees create their own playable video games.

I recently wrote a custom tool to diff CryEngine layer files in P4, and was surprised how simple it was. What follows is a quick tutorial on adding custom python tools to Perforce.

Start by heading over to Tools>Manage Custom Tools… Then click ‘New’:

You can pass a lot of information to an external tool, here is a detailed rundown. As you see above, we pass the client spec (local) file name (%f) to a python script, let’s create a new script called ‘custom_tool.py’:

import sys
from PyQt4 import QtGui    
 
class custom_tool(QtGui.QMessageBox):
	def __init__(self, parent=None):
		QtGui.QMessageBox.__init__(self)
		self.setDetailedText(str(sys.argv))
		self.show()
 
if __name__ == "__main__":
	app = QtGui.QApplication(sys.argv)
	theTool = custom_tool()
	theTool.show()
	sys.exit(app.exec_())

What this does is simply spits out the sys.argv in a way you can see it. So now you can feed any file you right click in Perforce into a python script:

If you would like to actually do something with a file or revision on the server and are passing the %F flag to get the depot file path, you then need to use p4 print to redirect the file contents (non-binary) to a local file:

p4.run_print('-q', '-o', depotFile, localFile)

Perforce is a wily beast. A lot of companies use it, but I feel few outside of the IT department really have to deal with it much. As I work myself deeper and deeper into the damp hole that is asset validation, I have really been writing a lot of python to deal with certain issues; but always scripts that work from the outside.

Perforce has a system that allows you to write scripts that are run, server side, when any number of events are triggered. You can use many scripting languages, but I will only touch on Python.

Test Environment

To follow along here, you should set up a test environment. Perforce is freely downloadable, and free to use with 2 users. Of course you are going to need python, and p4python. So get your server running and add two users, a user and an administrator.

Your First Trigger

Let’s create the simplest python script. It will be a submit trigger that says ‘Hello World’ then passes or fails. If it passes, the item will be checked in to perforce, if it fails, it will not. exiting while returning a ‘1’ is considered a fail, ‘0’ a pass.

print 'Hello World!'
print 'No checkin for you!'
sys.exit(1)

Ok, so save this file as hello_trigger.py. Now go to a command line and enter ‘p4 triggers’ this will open a text document, edit that document to point to your trigger, like so (but point to the location of your script on disk):

Triggers:
	hello_trigger change-submit //depot/... "python X:/projects/2010/p4/hello_trigger.py"

Close/save the trigger TMP file, you should see ‘Triggers saved.’ echo’d at the prompt. Now, when we try to submit a file to the depot, we will get this:

So: awesome, you just DENIED your first check-in!

Connecting to Perforce from Inside a Trigger

So we are now denying check-ins, but let’s try to do some other things, let’s connect to perforce from inside a trigger.

from P4 import P4, P4Exception
 
p4 = P4()
 
try:
	#use whatever your admin l/p was
	#this isn't the safest, but it works at this beginner level
	p4.user = "admin"
	p4.password = "admin"
	p4.port = "1666"
	p4.connect()
	info = p4.run("info")
	print info
	sys.exit(1)
 
#this will return any errors
except P4Exception:
	for e in p4.errors: print e
	sys.exit(1)

So now when you try to submit a file to depot you will get this:

Passing Info to the Trigger

Now we are running triggers, accepting or denying checkins, but we really don’t know much about them. Let’s try to get enough info to where we could make a decision about whether or not we want the file to pass validation. Let’s make another python trigger, trigger_test.py, and let’s query something from the perforce server in the submit trigger. To do this we need to edit our trigger file like so:

Triggers:
	test change-submit //depot/... "python X:/projects/2010/p4/test_trigger.py %user% %changelist%"

This will pass the user and changelist number into the python script as an arg, the same way dragging/dropping passed args to python in my previous example. So let’s set that up, save the script from before as ‘test_trigger.py’ as shown above, and add the following:

import sys
from P4 import P4, P4Exception
 
p4 = P4()
describe = []
 
try:
	p4.user = "admin"
	p4.password = "admin"
	p4.port = "1666"
	p4.connect()
 
except P4Exception:
	for e in p4.errors: print e
	sys.exit(1)
 
print str(sys.argv)
describe = p4.run('describe',sys.argv[2])
print str(describe)
 
p4.disconnect()
sys.exit(1)

So, as you can see, it has returned the user and changelist number:

However, for this changelist to be useful, we query p4, asking the server to describe the changelist. This returns a lot of information about the changelist.

Where to Go From here

The few simple things shown here really give you the tools to do many more things. Here are some examples of triggers that can be  created with the know-how above:

• Deny check-ins of a certain filetype (like deny compiled source files/assets)
• Deny check-ins whose hash digest matches an existing file on the server
• Deny/allow a certain type of file check-in from a user in a certain group
• Email a lead any time a file in a certain folder is updated

Did you find this helpful? What creative triggers have you written?

In Python, a Decorator is a type of macro that allows you to inject or modify code in functions or classes. I was turned onto this by my friend Matt Chapman at ILM, but never fully grasped the importance.

class myDecorator(object):
	def __init__(self, f):
		self.f = f
	def __call__(self):
		print "Entering", self.f.__name__
		self.f()
		print "Exited", self.f.__name__
 
@myDecorator
def aFunction():
	print "aFunction running"
 
aFunction()

When you run the code above you will see the following:

>>Entering aFunction
>>aFunction running
>>Exited aFunction

So when we call a decorated function, we get a completely different behavior. You can wrap any existing functions, here is an example of wrapping functions for error reporting:

class catchAll:
	def __init__(self, function):
		self.function = function
 
	def __call__(self, *args):
		try:
			return self.function(*args)
		except Exception, e:
			print "Error: %s" % (e)
 
@catchAll
def unsafe(x):
  return 1 / x
 
print "unsafe(1): ", unsafe(1)
print "unsafe(0): ", unsafe(0)

So when we run this and divide by zero we get:

unsafe(1):  1
unsafe(0):  Error: integer division or modulo by zero

Using decorators you can make sweeping changes to existing code with minimal effort, like the error reporting function above, you could go back and just sprinkle these in older code.

I have really been trying to learn some Python fundamentals lately, reading some books and taking an online class. So: wow. I can’t believe that I have written so many tools, some used by really competent people at large companies, without really understanding polymorphism and other basic Python concepts.

Here’s an example of my sequence method from before, but making it a class using special class methods:

class imSequence:
	def __init__(self, file):
		dir = os.path.dirname(file)
		file = os.path.basename(file)
		segNum = re.findall(r'\d+', file)[-1]
		self.numPad = len(segNum)
		self.baseName = file.split(segNum)[0]
		self.fileType = file.split('.')[-1]
		globString = self.baseName
		for i in range(0,self.numPad): globString += '?'
		self.images = glob.glob(dir+'\\'+globString+file.split(segNum)[1])
 
	def __len__(self):
		return len(self.images)
 
	def __iter__(self):
		return iter(self.images)

Here’s an example of use:

seq = imSequence('seq\\test_00087.tga')
print len(seq)
>>94
print 'BaseName: %s  FileType: %s  Padding: %s' % (seq.baseName, seq.fileType, seq.numPad)
>>BaseName: test_  FileType: tga  Padding: 5
for image in seq: print image
>>seq\test_00000.tga
>>seq\test_00001.tga
>>seq\test_000002.tga
...

[More info and examples: Dive Into Python: Special Class Methods]

I have been parsing through the files of other people a lot lately, and finally took the time to make a little function to give me general information about a sequence of files. It uses regex to yank the numeric parts out of a filename, figure out the padding, and glob to tell you how many files in the sequence. Here’s the code and an example usage:

#returns [base name, padding, filetype, number of files, first file, last file]
def getSeqInfo(file):
	dir = os.path.dirname(file)
	file = os.path.basename(file)
	segNum = re.findall(r'\d+', file)[-1]
	numPad = len(segNum)
	baseName = file.split(segNum)[0]
	fileType = file.split('.')[-1]
	globString = baseName
	for i in range(0,numPad): globString += '?'
	theGlob = glob.glob(dir+'\\'+globString+file.split(segNum)[1])
	numFrames = len(theGlob)
	firstFrame = theGlob[0]
	lastFrame = theGlob[-1]
	return [baseName, numPad, fileType, numFrames, firstFrame, lastFrame]

Here is an example of usage:

print getSeqInfo('E:\\data\\data\\Games\\Project\\CaptureOutput\\Frame000547.jpg')
>>['Frame', 6, 'jpg', 994, 'E:\\data\\data\\Games\\Project\\CaptureOutput\\Frame000000.jpg', 'E:\\data\\data\\Games\\Project\\CaptureOutput\\Frame000993.jpg']

I know this is pretty simple, but I looked around a bit online and didn’t see anything readily available showing how to deal with different numbered file sets. I have needed something like this for a while that will work with anything from OBJs sent from external contractors, to images from After Effects…

So I have always been wondering how you can create almost like a ‘droplet’ to steal the photoshop lingo, from a python script. A while ago I came across some sites showing how to edit shellex in regedit to allow for files to be dropped on any python script and fed to it as args (Windows).

It’s really simple, you grab this reg file [py_drag_n_drop.reg] and install it.

Now when you drop files onto a python script, their filenames will be passed as args, here’s a simple script to test.

import sys
 
f = open('c:\\tmp.txt', 'w')
for arg in sys.argv:
    f.write(arg + '\n')
f.close()

When you save this, and drop files onto its icon, it will create tmp.txt, which will look like this:

X:\projects\2010\python\drag_and_drop\drag_n_drop.py
X:\photos\2010.04 - easter weekend\fuji\DSCF9048.MPO
X:\photos\2010.04 - easter weekend\fuji\DSCF9049.MPO
X:\photos\2010.04 - easter weekend\fuji\DSCF9050.MPO
X:\photos\2010.04 - easter weekend\fuji\DSCF9051.MPO
X:\photos\2010.04 - easter weekend\fuji\DSCF9052.MPO

The script itself is the first arg, then all the files. This way you can easily create scripts that accept drops to do things like convert files, upload files, etc..

I got some good feedback from the last post and updated the script to export JPEG Stereo (JPS) and PNG Stereo (PNS, really.) This way you can convert your images into a single lossless image that you can pop into photoshop and adjust hsv/levels, etc.

import mpo
mpo.makePNS('DSCF9463.MPO')
#>>Saving image: DSCF9463.PNS
#>>Save complete.

This is a super simple python script, no error padding. Also, keep in mind that coming from most modern camera rigs, you are saving like a 20-40 megapixel PNG compressed file here, wait until it says it is done saving, it may take a few seconds.

Many stereo cameras are using the new MPO format to store multiple images in a file. Unfortunately, nothing really works with these files (Other than Stereo Photo Maker). Here is a simple python wrapper around ExifTool that will extract the Right and Left image, and return EXIF data as a dict. I think this is probably easier than explaining how to use ExifTool, but you can see from looking at the simple wrapper code.

import mpo
 
#Name of MPO file, name of output, whether or not you want all EXIF in a txt log
mpo.extractImagePair('DSCF9463.MPO', 'DSCF9463', True)
#>>Created DSCF9463_R.jpg
#>>Created DSCF9463_L.jpg
#>>Writing EXIF data

The above leaves you with two images and a text file that has all the EXIF data, even attributes that xnView and other apps do not read:

exif =  getExif('DSCF9463.MPO')
print exif["Convergence Angle"]
#>>0
print exif["Field Of View"]
#>>53.7 deg
print exif["Focal Length"]
#>>6.3 mm (35 mm equivalent: 35.6 mm)

I have been really amazing myself at how much knowledge I have forgotten in the past five or six months… Most of the work I did in the past year utilized the UIC module to load UI files directly, but I can find very little information about this online. I was surprised to see that even the trusty old Rapid GUI Programming with Python and Qt book doesn’t cover loading UI files with the UIC module.

So, here is a tiny script with UI file [download] that will generate a pyqt example window that does ‘stuff’:

import sys
from PyQt4 import QtGui, QtCore, uic
 
class TestApp(QtGui.QMainWindow):
	def __init__(self):
		QtGui.QMainWindow.__init__(self)
 
		self.ui = uic.loadUi('X:/projects/2010/python/pyqt_tutorial/pyqt_tutorial.ui')
		self.ui.show()
 
		self.connect(self.ui.doubleSpinBox, QtCore.SIGNAL("valueChanged(double)"), spinFn)
		self.connect(self.ui.comboBox, QtCore.SIGNAL("currentIndexChanged(QString)"), comboFn)
		self.connect(self.ui.pushButton, QtCore.SIGNAL("clicked()"), buttonFn)
 
def spinFn(value):
	win.ui.doubleSpinBoxLabel.setText('doubleSpinBox is set to ' + str(value))
def buttonFn():
	win.ui.setWindowTitle(win.ui.lineEdit.text())
def comboFn(value):
	win.ui.comboBoxLabel.setText(str(value) + ' is selected')
 
if __name__ == "__main__":
	app = QtGui.QApplication(sys.argv)
	win = TestApp()
	sys.exit(app.exec_())

Change the path to reflect where you have saved the UI file, and when you run the script you should get this:

EDIT: A few people have asked me to update this for other situations

PySide Inside Maya:

import sys
from PySide.QtUiTools import *
from PySide.QtCore import *
from PySide.QtGui import *
 
class TestApp(QMainWindow):
	def __init__(self):
		QMainWindow.__init__(self)
 
		loader = QUiLoader()
		self.ui = loader.load('c:/pyqt_tutorial.ui')
		self.ui.show()
 
		self.connect(self.ui.doubleSpinBox, SIGNAL("valueChanged(double)"), spinFn)
		self.connect(self.ui.comboBox, SIGNAL("currentIndexChanged(QString)"), comboFn)
		self.connect(self.ui.pushButton, SIGNAL("clicked()"), buttonFn)
 
def spinFn(value):
	win.ui.doubleSpinBoxLabel.setText('doubleSpinBox is set to ' + str(value))
def buttonFn():
	win.ui.setWindowTitle(win.ui.lineEdit.text())
def comboFn(value):
	win.ui.comboBoxLabel.setText(str(value) + ' is selected')
 
win = TestApp()

PyQT Inside Maya:

import sys
from PyQt4 import QtGui, QtCore, uic
 
class TestApp(QtGui.QMainWindow):
	def __init__(self):
		QtGui.QMainWindow.__init__(self)
 
		self.ui = uic.loadUi('c:/pyqt_tutorial.ui')
		self.ui.show()
 
		self.connect(self.ui.doubleSpinBox, QtCore.SIGNAL("valueChanged(double)"), spinFn)
		self.connect(self.ui.comboBox, QtCore.SIGNAL("currentIndexChanged(QString)"), comboFn)
		self.connect(self.ui.pushButton, QtCore.SIGNAL("clicked()"), buttonFn)
 
def spinFn(value):
	win.ui.doubleSpinBoxLabel.setText('doubleSpinBox is set to ' + str(value))
def buttonFn():
	win.ui.setWindowTitle(win.ui.lineEdit.text())
def comboFn(value):
	win.ui.comboBoxLabel.setText(str(value) + ' is selected')
 
win = TestApp()

I have gotten back into some pyqt in my spare time, just because it’s what I used on a daily basis at the last place I worked at. However, I had trouble getting it to run in my text editor of choice. (SciTE)

I couldn’t find a solution with like 45 minutes of googling. When trying to import PyQt4 it would give me a dll error, but I could paste the code into IDLE and it would execute fine. I found a solution by editing the python preferences of SciTE. I noticed that it wasn’t running python scripts the way IDLE was, but compiling them (?). I edited the last line to just run the script, and viola! It worked.

Find this line (usually the last):

command.1.*.py=python -c "import py_compile; py_compile.compile(r'$(FilePath)')"

And change it to:

command.1.*.py=python "(r'$(FilePath)')"

I don’t really know if this messes anything else up, but it does allow the PyQt4 libs to load and do their thing.

The Vatican recently put up an interactive Sistine Chapel flash application. You can pan around the entire room and zoom in and out in great detail.

The Vatican is not very open with it’s art, the reason they scream ‘NO PHOTO’ when you pull a camera out in the chapel is that they sold the ability to take photos of it to a Japanese TV Station (Nippon TV) for 4.2 million dollars. Because the ceiling has long been in the public domain, the only way they can sell ‘the right to photograph’ the ceiling is by screwing over us tourists who visit. If you take a photo, they have no control over that image –because they don’t own the copyright of the work.

Many of you who know me, know I am a huge fan of Michelangelo’s work, this data was just too awesomely tempting and when I saw it posted publicly online, I really wanted to get my hands on the original assets.

Here is a python script to grab all of the image tiles that the flash app reads, and then generate the 8k faces of the cubemap. In the end you will have a 32,000 pixel cubemap.

First we copy the swatches from the website:

def getSistineCubemap(saveLoc):
	import urllib
	#define the faces of the cubemap, using their own lettering scheme
	faces = ['f','b','u','d','l','r']
	#location of the images
	url = 'http://www.vatican.va/various/cappelle/sistina_vr/Sistine-Chapel.tiles/l3_'
	#copy all the swatches to your local drive
	for face in faces:
		for x in range(1,9):
			for y in range(1,9):
				file = (face + '_' + str(y) + '_' + str(x) + '.jpg')
				urllib.urlretrieve((url + face + '_' + str(y) + '_' + str(x) + '.jpg'), (saveLoc + file))
				urllib.urlcleanup()
				print "saved " + file

Next we use PIL to stitch them together:

def stitchCubeMapFace(theImage, x, y, show):
	from PIL import Image, ImageDraw
	from os import path
 
	file = theImage.split('/')[-1]
	fileSplit = file.split('_')
	im = Image.open(theImage)
	#create an 8k face from the first swatch
	im = im.resize((8000, 8000), Image.NEAREST)
	thePath = path.split(theImage)[0]
 
	xPixel = 0
	yPixel = 0
	#loop through the swatches, stitching them together
	for y_ in range(1, x+1):
		for x_ in range(1,y+1):
			if yPixel == 8000:
				yPixel = 0
			nextImage = (thePath + '/' + fileSplit[0] + '_' + str(x_) + '_' + str(y_) + '.jpg')
			print ('Merging ' + nextImage + ' @' + str(xPixel) + ',' + str(yPixel))
			loadImage = Image.open(nextImage)
			im.paste(loadImage, (xPixel, yPixel))
			yPixel += 1000
		xPixel += 1000
	saveImageFile = (thePath + '/' + fileSplit[0] + '_face.jpg')
	print ('Saving face: ' + saveImageFile)
	#save the image
	im.save(saveImageFile, 'JPEG')
	#load the image in default image viewer for checking
	if show == True:
		import webbrowser
		webbrowser.open(saveImageFile)

Here is an example of the input params:

getSistineCubemap('D:/sistineCubeMap/')
stitchCubeMapFace('D:/sistineCubeMap/r_1_1.jpg', 8, 8, True)

So, before we looked at just outputting a list of the files that were on device1 and not device2, now I will copy the files to a folder on the main device.

The tricky thing about this is I want the directory structure intact. After looking as os.path, and pywin32, I didn’t see anything like ‘mkdir’ where it would make all the folders deep needed to recreate the branch that a file was in. I did however find a function online:

def mkdir(newdir):
    if os.path.isdir(newdir):
        pass
    elif os.path.isfile(newdir):
        raise OSError("a file with the same name as the desired " \
                      "dir, '%s', already exists." % newdir)
    else:
        head, tail = os.path.split(newdir)
        if head and not os.path.isdir(head):
            mkdir(head)
        if tail:
            os.mkdir(newdir)

To copy the files and create the directories, I altered the previous script a bit:

for (path, dirs, files) in os.walk(path2):
	for file in files:
		if os.path.basename(file) not in filenames:
			newPath = os.path.abspath(os.path.join(path,file)).replace(path2,(path1 + 'isItThere//'))
			fileFull = os.path.abspath(os.path.join(path,file))
			print fileFull + " not found in " + path1 + " file cloud"
			print "Copying " + fileFull + " >>> " + newPath
			if os.path.isdir(os.path.dirname(newPath)) == False:
				mkdir(os.path.dirname(newPath))
			win32file.CopyFile (fileFull, newPath, 0))

The results printed should look like below, the files should have been copied accordingly and the directories created.

U:\photos\Crystal\Orlando - Lauras wedding\P0003270.jpg not found in D:\photos\Crystal\ file cloud
Copying U:\photos\Crystal\Orlando - Lauras wedding\P0003270.jpg >>> D:\photos\Crystal\isItThere\Orlando - Lauras wedding\P0003270.jpg
U:\photos\Crystal\Orlando - Lauras wedding\P0003271.jpg not found in D:\photos\Crystal\ file cloud
Copying U:\photos\Crystal\Orlando - Lauras wedding\P0003271.jpg >>> D:\photos\Crystal\isItThere\Orlando - Lauras wedding\P0003271.jpg
U:\photos\Crystal\Orlando - Lauras wedding\P0003272.jpg not found in D:\photos\Crystal\ file cloud

If I had time, or perhaps when I have time, I’ll add MD5 checks.

I am anal-retentive about data retention. There, I said it. There are many times when I find myself in the situation of having two storage devices, that may or may not have duplicate files. I then want to erase one, but do I have all those files backed up?

I use two existing programs to aid me in my anal-retentivity: TerraCopy and WinMerge. Terracopy replaces the windows default copy with something much better (can hash check files when they are copied, etc). With WinMerge, I can right click a folder and say ‘Compare To…’ then right click another and say ‘Compare’. This tells me any differences between the two file/folder trees.

However, here’s an example I have not yet found a good solution for:

I want to erase a camera card I have, I am pretty certain I copied the images off –but how can I be sure! I took those images and sorted them into folders by location or date taken.

So I wrote a small and I am sure inefficient python script to help:

import os
 
filenames = []
 
i = 1
 
path1 = 'D://photos//south america//'
path2 = 'N://DCIM//100ND300//'
 
if os.path.isdir(path1):
	if os.path.isdir(path2):
		print "creating index.."
 
for (path, dirs, files) in os.walk(path1):
	for file in files:
		filenames.append(os.path.basename(file))
 
for (path, dirs, files) in os.walk(path2):
	for file in files:
		if os.path.basename(file) not in filenames:
			print os.path.abspath(os.path.join(path,file)) + ' not found in ' + path1 + ' file cloud'

This will print something like this:

N:/DCIM/100ND300/image.NEF not found in D:/photos/south america/ file cloud

I don’t use python that often at all, please lemme know if there’s a better way to be doing this.

This is a simple proof-of-concept showing how to implement a perforce animation browser via python for MotionBuilder. Clicking an FBX animation syncs it and loads it.

The script can be found here: [p4ui.py], it requires the [wx] and [p4] libraries.

Clicking directories goes down into them, clicking fbx files syncs them and loads them in MotionBuilder. This is just a test, the ‘[..]’ doesn’t even go up directories. Opening an animation does not check it out, there is good documentation for the p4 python lib, you can start there; it’s pretty straight forward and easy: sure beats screen scraping p4 terminal stuff.

You will see the following, you should replace this with the p4 location of your animations, this will act as the starting directory.

	path1 = 'PUT YOUR PERFORCE ANIMATION PATH HERE (EXAMPLE: //DEPOT/ANIMATION)'
	info = p4i.run("info")
	print info[0]['clientRoot']

That should about do it, there are plenty of P4 tutorials out there, my code is pretty straight forward. The only problem was where I instanced it, be sure to instance it with something other than ‘p4’, I did this and it did not work, using ‘p4i’ it did without incident:

p4i = P4.P4()
p4i.connect()

In 2007, my friend Jason gave an AutoDesk Masterclass entitled: Python Scripting for MotionBuilder Artists. It has been available online and I would like to mention it for anyone who is interested in Python and MotionBuilder.

Here’s what you get for only 40 bucks:

118 page PowerPoint presentation
72 page Full Documentation
21 Scripts
6 Scenes
2 text files
8 .mov videos capturing 1 hour 20 minute lecture

Buy it here: Python Scripting for MotionBuilder Artists

Facial motion capture stabilization is basically where you isolate the movement of the face from the movement of the head. This sounds pretty simple, but it is actually a really difficult problem. In this post I will talk about the general process and give you an example facial stabilization python script.

Disclaimer: The script I have written here is loosely adapted from a MEL script in the book Mocap for Artists, and not something proprietary to Crytek. This is a great book for people of all experience levels, and has a chapter dedicated to facial mocap. Lastly, this script is not padded out or optimized.

To follow this you will need some facial mocap data, there is some freely downloadable here at www.mocap.lt. Grab the FBX file.

Stabilization markers

Get at least 3 markers on the actor that do not move when they move their face. These are called ’stabilization markers’ (STAB markers). You will use these markers to create a coordinate space for the head, so it is important that they not move. STAB markers are commonly found on the left and right temple, and nose bridge. Using a headband and creating virtual markers from multiple solid left/right markers works even better. Headbands move, it’s good to keep this in mind, above you see a special headrig used on Kong to create stable markers.

It is a good idea to write some tools to help you out here. At work I have written tools to parse a performance and tell me the most stable markers at any given time, if you have this data, you can also blend between them.

Load up the facial mocap file you have downloaded, it should look something like this:

In the data we have, you can delete the root, the headband markers, as well as 1-RTMPL, 1-LTMPL, and 1-MNOSE could all be considered STAB markers.

General Pipeline

As you can see, mocap data is just a bunch of translating points. So what we want to do is create a new coordinate system that has the motion of the head, and then use this to isolate the facial movement.

This will take some processing, and also an interactive user interface. You may have seen my tutorial on Creating Interactive MotionBuilder User Interface Tools. You should familiarize yourself with that because this will build on it. Below is the basic idea:

You create a library ‘myLib’ that you load into motionbuilder’s python environment. This is what does the heavy lifting, I say this because you don’t want to do things like send the position of every marker, every frame to your external app via telnet. I also load pyEuclid, a great vector library, because I didn’t feel like writing my own vector class. (MBuilder has no vector class)

Creating ‘myLib’

So we will now create our own library that sits inside MBuilder, this will essentially be a ‘toolkit’ that we communicate with from the outside. Your ‘myLib’ can be called anything, but this should be the place you store functions that do the real processing jobs, you will feed into to them from the outside UI later. The first thing you will need inside the MB python environment is something to cast FBVector3D types into pyEuclid. This is fairly simple:

#casts point3 strings to pyEuclid vectors
def vec3(point3):
	return Vector3(point3[0], point3[1], point3[2])
 
#casts a pyEuclid vector to FBVector3d
def fbv(point3):
	return FBVector3d(point3.x, point3.y, point3.z)

Next is something that will return an FBModelList of models from an array of names, this is important later when we want to feed in model lists from our external app:

#returns an array of models when given an array of model names
#useful with external apps/telnetlib ui
def modelsFromStrings(modelNames):
	output = []
	for name in modelNames:
		output.append(FBFindModelByName(name))
	return output

Now, if you were to take these snippets and save them as a file called myLib.py in your MBuilder directory tree (MotionBuilder75 Ext2\bin\x64\python\lib), you can load them into the MBuilder environment. (You should have also placed pyEuclid here)

In the image above, I get the position of a model in MBuilder, it returns as a FBVector3D, I then import myLib and pyEuclid and use our function above to ‘cast’ the FBVector3d to a pyEuclid vector. It can now be added, subtracted, multiplied, and more; all things that are not possible with the default MBuilder python tools. Our other function ‘fbv()‘ casts pyEuclid vectors back to FBVector3d, so that MBuilder can read them.

So we can now do vector math in motionbuilder! Next we will add some code to our ‘myLib’ that stabilizes the face.

Adding Stabilization-Specific Code to ‘myLib’

One thing we will need to do a lot is generate ‘virtual markers’ from the existing markers. To do this, we need a function that returns the average position of however many vectors (marker positions) it is fed.

#returns average position of an FBModelList as FBVector3d
def avgPos(models):
	mLen = len(models)
	if mLen == 1:
		return models[0].Translation
	total = vec3(models[0].Translation)
	for i in range (1, mLen):
		total += vec3(models[i].Translation)
	avgTranslation = total/mLen
	return fbv(avgTranslation)

Here is an example of avgPos() in use:

Now onto the stabilization code:

#stabilizes face markers, input 4 FBModelList arrays, leaveOrig  for leaving original markers
def stab(right,left,center,markers,leaveOrig):
 
	pMatrix = FBMatrix()
	lSystem=FBSystem()
	lScene = lSystem.Scene
	newMarkers = []
 
	def faceOrient():
		lScene.Evaluate()
 
		Rpos = vec3(avgPos(right))
		Lpos = vec3(avgPos(left))
		Cpos = vec3(avgPos(center))
 
		#build the coordinate system of the head
		faceAttach.GetMatrix(pMatrix)
		xVec = (Cpos - Rpos)
		xVec = xVec.normalize()
		zVec = ((Cpos - vec3(faceAttach.Translation)).normalize()).cross(xVec)
		zVec = zVec.normalize()
		yVec = xVec.cross(zVec)
		yVec = yVec.normalize()
		facePos = (Rpos + Lpos)/2
 
		pMatrix[0] = xVec.x
		pMatrix[1] = xVec.y
		pMatrix[2] = xVec.z
 
		pMatrix[4] = yVec.x
		pMatrix[5] = yVec.y
		pMatrix[6] = yVec.z
 
		pMatrix[8] = zVec.x
		pMatrix[9] = zVec.y
		pMatrix[10] = zVec.z
 
		pMatrix[12] = facePos.x
		pMatrix[13] = facePos.y
		pMatrix[14] = facePos.z
 
		faceAttach.SetMatrix(pMatrix,FBModelTransformationMatrix.kModelTransformation,True)
		lScene.Evaluate()
 
	#keys the translation and rotation of an animNodeList
	def keyTransRot(animNodeList):
		for lNode in animNodeList:
			if (lNode.Name == 'Lcl Translation'):
				lNode.KeyCandidate()
			if (lNode.Name == 'Lcl Rotation'):
				lNode.KeyCandidate()
 
	Rpos = vec3(avgPos(right))
	Lpos = vec3(avgPos(left))
	Cpos = vec3(avgPos(center))
 
	#create a null that will visualize the head coordsys, then position and orient it
	faceAttach = FBModelNull("faceAttach")
	faceAttach.Show = True
	faceAttach.Translation = fbv((Rpos + Lpos)/2)
	faceOrient()
 
	#create new set of stabilized nulls, non-destructive, this should be tied to 'leaveOrig' later
	for obj in markers:
		new = FBModelNull(obj.Name + '_stab')
		newTran = vec3(obj.Translation)
		new.Translation = fbv(newTran)
		new.Show = True
		new.Size = 20
		new.Parent = faceAttach
		newMarkers.append(new)
 
	lPlayerControl = FBPlayerControl()
	lPlayerControl.GotoStart()
	FStart = int(lPlayerControl.ZoomWindowStart.GetFrame(True))
	FStop = int(lPlayerControl.ZoomWindowStop.GetFrame(True))
 
	animNodes = faceAttach.AnimationNode.Nodes
 
	for frame in range(FStart,FStop):
 
		#build proper head coordsys
		faceOrient()
 
		#update stabilized markers and key them
		for m in range (0,len(newMarkers)):
			markerAnimNodes = newMarkers[m].AnimationNode.Nodes
			newMarkers[m].SetVector(markers[m].Translation.Data)
			lScene.Evaluate()
			keyTransRot(markerAnimNodes)
 
		keyTransRot(animNodes)
 
		lPlayerControl.StepForward()

We feed our ‘stab‘ function FBModelLists of right, left, and center stabilization markers, it creates virtual markers from these groups. Then ‘markers’ is all the markers to be stabilized. ‘leavrOrig’ is an option I usually add, this allows for non-destructive use, I have just made the fn leave original in this example, as I favor this, so this option does nothing, but you could add it. With the original markers left, you can immediately see if there was an error in your script. (new motion should match orig)

Creating an External UI that Uses ‘myLib’

Earlier I mentioned Creating Interactive MotionBuilder User Interface Tools, where I explain how to screenscrape/use the telnet Python Remote Server to create an interactive external UI that floats as a window in MotionBuilder itself. I also use the libraries mentioned in the above article.

The code for the facial stabilization UI I have created is here: [stab_ui.py]

I will now step through code snippets pertaining to our facial STAB tool:

def getSelection():
	selectedItems = []
	mbPipe("selectedModels = FBModelList()")
	mbPipe("FBGetSelectedModels(selectedModels,None,True)")
	for item in (mbPipe("for item in selectedModels: print item.Name")):
		selectedItems.append(item)
	return selectedItems

This returns a list of strings that are the currently selected models in MBuilder. This is the main thing that our external UI does. The person needs to interactively choose the right, left, and center markers, then all the markers that will be stabilized.

At the left here you see what the UI looks like. To add some feedback to the buttons, you can make them change to reflect that the user has selected markers. We do so by changing the button text.

Example:

def rStabClick(self,event):
	self.rStabMarkers = getSelection()
	print str(self.rStabMarkers)
	self.rStab.Label = (str(len(self.rStabMarkers)) + " Right Markers")

This also stores all the markers the user has chosen into the variable ‘rStabMarkers‘. Once we have all the markers the user has chosen, we need to send them to ‘myLib‘ in MBuilder so that it can run our ‘stab‘ function on them. This will happen when they click ‘Stabilize Markerset‘.

def stabilizeClick(self,event):
	mbPipe('from euclid import *')
	mbPipe('from myLib import *')
	mbPipe('rStab = modelsFromStrings(' + str(self.rStabMarkers) + ')')
	mbPipe('lStab = modelsFromStrings(' + str(self.lStabMarkers) + ')')
	mbPipe('cStab = modelsFromStrings(' + str(self.cStabMarkers) + ')')
	mbPipe('markerset = modelsFromStrings(' + str(self.mSetMarkers) + ')')
	mbPipe('stab(rStab,lStab,cStab,markerset,False)')

Above we now use ‘modelsFromStrings‘ to feed ‘myLib’ the names of selected models. When you run this on thousands of frames, it will actually hang for up to a minute or two while it does all the processing. I discuss optimizations below. Here is a video of what you should have when stabilization is complete:

Conclusion: Debugging/Optimization

Remember: Your stabilization will only be as good as your STAB markers. It really pays off to create tools to check marker stability.

Sometimes the terminal/screen scraping runs into issues. The mbPipe function can be padded out a lot and made more robust, this here was just an example. If you look at the external python console, you can see exactly what mbPipe is sending to MBuilder, and what it is receiving back through the terminal:

Sending>>> selectedModels = FBModelList()
Sending>>> FBGetSelectedModels(selectedModels,None,True)
Sending>>> for item in selectedModels: print item.Name
['Subject 1-RH1', 'Subject 1-RTMPL']

All of the above can be padded out and optimized. For instance, you could try to do everything without a single lPlayerControl.StepForward() or lScene.Evaluate(), but this takes a lot of MotionBuilder/programming knowhow; it involves only using the keyframe data to generate your matrices, positions etc, and never querying a model.

I was talking to my friend Marco the other day.  As he is a real programmer, he is somewhat equipped with the needed skills required to decode MotionBuilder’s procedurally-generated Python documentation.  We were both frustrated, fighting with the ‘Python Console Tool’, when I showed him the telnet interface he was like “why don’t you just use that?”

And this is what I started doing. I now do much of my tests and work in the telnet console, because, unlike the built in console that Motion Builder offers, the telnet window at least offers copy/paste, and you can press the up arrow to cycle through previous arguments that you have entered. I would suggest using this until Autodesk adds usable features to their ‘Python Console Tool’.

Here’s an example:

I have been researching quick ways to output MotionBuilder data visually, which I might post about later (doing some matplotlib tests here at home). The following is probably a ‘no-brainer’ to people with a programming background, but I found it interesting. Below I am using simple hashes to graph values visually in the console.

data = [20, 15, 10, 7, 5, 4, 3, 2, 1, 1, 0]
for i in data: print '#' * i

This will output something like so:

####################
###############
##########
#######
#####
####
###
##
#
#

Here’s a better example referencing some data names, and it’s output in the MB pyConsole:

for i in range(0,len(data)): print data1[i] + ' ' + ('#' * data[i])

This is another post in my short quest to profile per-frame functions in Motion Builder. This script below uses hotshot, the built in python profiler, to profile not only a function I write, but built in MBuilder c++ modules (FBPlayerControl().StepForward()).

8
9
10

#wrap motionbuilder c module functions you want to profile in python functions
def stepForward():
	lPlayerControl.StepForward()

Above is a snippet of the script below. We are wrapping the StepForward() MBuilder call in a function called stepForward(). Hotshot can profile any python function, so we trick it by wrapping a function around the c module.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

from pyfbsdk import *
import hotshot
import hotshot.stats
import sys
 
lPlayerControl = FBPlayerControl()
 
#wrap motionbuilder c module functions you want to profile in python functions
def stepForward():
	lPlayerControl.StepForward()
 
#this is our function that we want to profile
def myFn():
	FStart = int(lPlayerControl.ZoomWindowStart.GetFrame(True))
	FStop = int(lPlayerControl.ZoomWindowStop.GetFrame(True))
 
	FBPlayerControl().GotoStart()
	for frame in range(FStart,FStop):
		stepForward()
 
#here we set up the hotshot profiler to create a profile file (binary)
#by running our function (myFn)
prof = hotshot.Profile("c:\\myFn.prof")
prof.runcall(myFn)
prof.close()
 
#now we load the profile stats
stats = hotshot.stats.load("c:\\myFn.prof")
stats.strip_dirs()
stats.sort_stats('time', 'calls')
 
#and finally, we print the profile stats to the disk in a file 'myFn.log'
saveout = sys.stdout
fsock = open('c:\\myFn.log', 'w')
sys.stdout = fsock
stats.print_stats(20)
sys.stdout = saveout
fsock.close()

This is what the file we generated (myFn.log) looks like:

         1305 function calls in 15411.020 CPU seconds
 
   Ordered by: internal time, call count
 
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1304 15398.358   11.809 15398.358   11.809 speedtest.py:9(stepForward)
        1   12.662   12.662 15411.020 15411.020 speedtest.py:13(myFn)
        0    0.000             0.000          profile:0(profiler)

As you can see, FBPlayerControl().StepForward() is very resource intensive, and should rarely be used when crunching a lot of keyframe data. I hope this helps anyone in finding resource bottlenecks in large scripts, here’s an example of output from a larger script where I am trying to see how long scene.evaluate() and stepforward() are taking, and get rid of them by querying the keyframe data the way KxL showed before:

         58745 function calls in 26273.622 CPU seconds
 
   Ordered by: internal time, call count
 
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1304 14243.406   10.923 14243.406   10.923 stabtest.py:57(stepForward)
     5217 8448.493    1.619 8448.493    1.619 stabtest.py:52(evalScene)
     5216 1347.949    0.258 1347.949    0.258 stabtest.py:99(keyTransRot)
     5223 1028.827    0.197 1028.827    0.197 stabtest.py:23(getPosition)
        1  672.045  672.045 26273.394 26273.394 stabtest.py:62(stab)
     1305  268.026    0.205 3963.387    3.037 stabtest.py:67(faceOrient)
     5226   53.200    0.010   68.283    0.013 stabtest.py:16(vec3)
     5220   40.443    0.008   40.443    0.008 euclid.py:485(__abs__)
     2610   31.510    0.012   36.667    0.014 euclid.py:384(__sub__)
     5220   30.552    0.006   70.996    0.014 euclid.py:497(normalize)
    13058   30.549    0.002   30.549    0.002 euclid.py:285(__init__)
     2610   27.759    0.011   32.958    0.013 euclid.py:519(cross)
     3918   22.071    0.006  796.673    0.203 stabtest.py:34(avgPos)
     1306   16.727    0.013   19.413    0.015 euclid.py:439(__div__)
     1306   11.738    0.009   14.161    0.011 euclid.py:354(__add__)
        1    0.228    0.228 26273.622 26273.622 stabtest.py:152(stabCall)
        4    0.100    0.025    0.100    0.025 stabtest.py:20(fbv)
        0    0.000             0.000          profile:0(profiler)

As you can see above, most of the time is spent on scene evaluation and stepforward.