Flash Player 10 3D: Beware of Nested 3D Transforms

Working on my Flash on the Beach presentation. Slightly sidetracked into fixing up some code in a game toolbox I’m creating. Doing 3D transitions.

The thing is, that applying any 3D transform turns an object into a 3D object, where cacheAsBitmap is always on. It has a serious effect on the performance of the game. A very noticeable effect. So no problem, I think. When the transform is done, I simply assign a new Matrix to the container’s transform.matrix. A display object can have either a transform.matrix or a transform.matrix3D. Assigning either one nulls out the other one. Done and verified that the matrix3D is gone. But performance still sucks. Turns out that a container within the container also suddenly has a matrix3D. But I never did that!

A quick test confirms that assigning a 3D property to a container does NOT effect the matrix/matrix3D of any of its children. Nor does assigning a 3D property to a child effect the matrix of the parent container. So what’s going on? After much testing and attempts at reproducing the bug, I found it.

Basically, if you have a series of three or more nested objects, and apply 3D transformations to TWO of them, any clips nested between those two will become 3D objects (i.e. have a matrix3D property).

Example. Sprite A contains Sprite B, which contains Sprite C. Do a 3D transform on any one of them, and none of the others are effected. But transform A and C, and suddenly Sprite B now has a matrix3D. It turns out that in my game, some things way down at the bottom of the display list had some 3D properties set. So when I did a 3D transition on the top container, every single display object on down the line between the two became a 3D object. That’s what was killing my performance.

Worse yet, removing the outer transforms does not reset the inner ones. Basically, I’d have to cache the transform.matrix of every object on the list and when the 3D transition was done, walk the tree and reset every object’s matrix. So, for now anyway, 3D transitions are out.

As part of my trying to figure out this problem, I created this little tester:

[kml_flashembed publishmethod=”static” fversion=”10.0.0″ movie=”http://www.bit-101.com/blog/wp-content/uploads/2009/09/3dmess.swf” width=”400″ height=”200″ targetclass=”flashmovie”]

[/kml_flashembed]

I happened to be talking to Grant Skinner as I was going through this and mentioned it to him. He mentioned that there are other graphical glitches he ran into while nesting 3D transforms. Shortly thereafter, I ran into these as well – things disappearing, half disappearing, random glitchy pixels. However, I don’t see them on this web version. I guess the Flash Player version in my browser has fixed a lot of those issues. I was seeing the issues in the CS4 internal Flash Player. If you have an older player version, you might see some issues.

Nevertheless, the original problem I had is pretty evident. Transform the inner and the outer clips and you’ll see that the middle clip becomes 3D. Clear the inner and outer and the middle stays 3D. I’m not sure I’d call this a bug, per se. I can see why it would be that way, but it’s far from optimal. In an ideal world, if the matrix3D were an identity matrix, the object would know that and not be 3D.

Oh, and here’s the code in case you are interested what’s going on. It’s not too pretty but it was never meant to be more than a test to figure out what’s going on.

[as3]import com.bit101.components.*;

new PushButton(this, 10, 10, “Rotate inner”, onInner);
new PushButton(this, 10, 35, “Rotate middle”, onMiddle);
new PushButton(this, 10, 60, “Rotate outer”, onOuter);

new PushButton(this, 10, 95, “Clear inner”, onClearInner);
new PushButton(this, 10, 120, “Clear middle”, onClearMid);
new PushButton(this, 10, 145, “Clear outer”, onClearOuter);

var innerLight:IndicatorLight = new IndicatorLight(this, 300, 10, 0xff0000, “Inner is 3d”);
var middleLight:IndicatorLight = new IndicatorLight(this, 300, 35, 0xff0000, “Middle is 3d”);
var outerLight:IndicatorLight = new IndicatorLight(this, 300, 60, 0xff0000, “Outer is 3d”);

var s:Sprite = new Sprite();
s.graphics.lineStyle(0);
s.graphics.drawRect(-50, -50, 100, 100);
s.x = 200;
s.y = 100;
addChild(s);

var t:Sprite = new Sprite();
t.graphics.lineStyle(0);
t.graphics.drawRect(-25, -25, 50, 50);
s.addChild(t);

var u:Sprite = new Sprite();
u.graphics.lineStyle(0);
u.graphics.drawRect(-10, -10, 20, 20);
t.addChild(u);

output();

function onInner(event:Event):void
{
u.rotationZ = 45;
output();
}

function onMiddle(event:Event):void
{
t.rotationY = 45;
output();
}

function onOuter(event:Event):void
{
s.rotationX = 45;
output();
}

function onClearInner(event:Event):void
{
u.transform.matrix = new Matrix();
output();
}

function onClearOuter(event:Event):void
{
s.transform.matrix = new Matrix(1, 0, 0, 1, 200, 100);
output();
}

function onClearMid(event:Event):void
{
t.transform.matrix = new Matrix();
output();
}

function output():void
{
innerLight.isLit = u.transform.matrix3D != null;
middleLight.isLit = t.transform.matrix3D != null;
outerLight.isLit = s.transform.matrix3D != null;
}[/as3]