I read a really good "intro" level article by Ted Patrick on performance in the flash player. I'm sure we've all read this stuff before, but it was a really good reminder for me, and the article is short.
Here are the best parts:
"The Flash Player uses a logical unit of work called a frame which consists of 2 phases which occur one after another in a loop:
1. Process ActionScript
2. Render Graphics
3. GOTO 1
After each of these phases is a buffer to handle delays ( say you render to much or loop over a ton of data ). When both of these phases have completed we call that a frame. Each SWF file you create has a framerate (frames/second) which represents the fastest rate that the Flash Player will loop over these phases. When Flash Player encounters delays in processing either phase it will elongate the time to process the frame and thus your framerate will slow but will never exceed the framerate set in the SWF file."
"Flex was built to handle application behavior in a minimalist fashion if used properly. I continue to see developers execute to much ActionScript on a single event and not think in terms of processing data across frames. Using events properly is a great place to start. Make sure you know how long and how often methods are being called from events."
17 comments:
The idea of a "frame" is outmoded and Adobe needs to ditch it. It's a false and misleading analogue to frames of film, and doesn't get to the heart of the issue.
Adobe needs to replace the frame with a system of dynamic process slicing, with an API hook to determine the balance between cycles to code and cycles to display rendering. There is no reason to make the developer do this task manually, and whenever I see a company peep like Ted spout off, "You developers aren't doing it the right way", that usually says to me that the system itself is flawed and needs to be reformed.
Are there other graphics-oriented frameworks out there that provide that type of "API hook"? In practice, does it look much different from frames?
I think Flex and AS3 do a pretty good job of keeping me out of the timeline -
most flex developers without a background don't even know what the timeline is. Maybe Adobe just needs to do a better job balancing the processing load on compile time?
It seems to me like a pretty tough problem to solve. You can only allow the processor to do so much before updating the screen. Since we're talking about a fundamentally graphics-oriented architecture, the idea of a "frame" seems to make a lot of sense. I'm not saying that there isn't a better way; just that if I tried to solve the problem from scratch, I'd probably land on something like Frames, and I don't come from a Flash background.
I'd hate to see someone like Ted say something like that too, but I don't think that's the message here. He's noticing that great things are being done with flash that could be better, and so he's trying to remind everyone how the system works underneath - it's a lot less condescending.
Brett and RJ,
The frame model is very flexible and has allowed Flash Player to be leveraged to make the best of animation (frame and time based), video, gaming, and now applications. The concept of a frame in Flex is 100% abstracted away so that you do not see this but in edge performance cases when the implementation becomes visible.
What I find interesting is that the speed of each element (code execution and graphics rendering) are speeding up. Currently in Flash Player 9 the code execution is waiting for the rendering to complete. In a future player we are GPU accelerating all rendering operations thus we will see a big jump in player performance. Elementally this will hide the paradigm even further as overall rendering rates of 100 fps are possible with GPU based rendering and caching. Also desktop computers are making huge jumps in capability and performance which just makes Flash Player far better. Frames may sound simple but it actually scales really well and fits the needs of many different types of use.
Frames do not sound very cool though, they sound rather archaic but between GPU rendering and VM improvements a 20X faster player is possible.
Today Flash Player is good enough for 90% of the application use cases and with the future performance improvements it will make another big leap.
Don't dismiss the power of frames!
Cheers,
Ted :)
Brett, I gotta disagree.
First. Aside from some potential intention of Macromedia way back when to appease the artists using Flash, there are a lot more applicable "frame" analogies than "frames of film" (see: networking).
Second. If Adobe did your "dynamic process slicing with API", I'd bet it would kill the product. Why? Because if that was something they could do/expose without difficulty or consequence, I'd bet that would have happened with the AVM rewrite.
So you've got a choice. Do you want it to be easy(er) to code an application or do you want the application to run with all the goodness we've come to expect from Flash Player?
There are much "friendlier" APIs out there, but there must be SOME reason you care enough about Flash/Flex/Flash Player to be bothered as much as you are by this. Why aren't you using some other system? And by the way, which one would that be?
Coming from the C++ desktop development world, it is a bit frustrating not to have a decade or three of incredibly intelligent analysis of esoteric performance issues for Actionscript development. But I realize that for many years, Actionscript was the scripting language for a graphics app (Flash), not the basis for a robust and capable enterprise-class development and deployment system. It'll take time to get the analysis.
Ted has got it tough. He's got to communicate with people like you who put easy coding at the top of your "wants" list, but also with people like me who put easy coding after other issues like "performance", "rendering quality", "functionality". Some days he pleases you more, some days he pleases me. And he generally does so with a lot more class than many other "spouting company peeps".
Go Ted Go!
Eric,
It's not necessarily about making coding "easy." It's about making the framework elegant and it's about the path of least resistance.
There are a lot of performance problems with graphically-intense Actionscript apps out there because developers don't understand the underlying beast. You have "onEnterFrame" events to cycle through iterative code, but there's nothing more granular by which to measure the length of code execution. So you're left with approximations and hand-tweaking. A lot of people won't bother going that far.
So, from a platform perspective (and as an "Evangelist" Ted should be concerned by this), badly-performing AS apps are a detriment to the perceived speed of the platform.
Sure, for some it may be about making the code easier. For others, like the people on this blog, it's about fine-tuning the balance between display rendering and background execution. But the overall effect is that maximizing performance on the platform becomes effortless, and thus the platform is seen as more robust.
Ted: Glad to hear about hardware-accelerated rendering, and definitely look forward to that.
OK Brett, when you explain it that way, I agree to a point...
Where are the continuing dialogues about the high-profile apps Adobe's purchased? Not releasing the code (even in part), but simply discussing why decisions were made, and whether, in the final analysis, they were the right decisions, etc... in the context of performance and rendering.
Buzzword? Amgen Tour Tracker? Hullo? Even the Employee Directory v2... what is meaningful about it as an example?
And while we're at it, how about hearing something from ESRI about their SuperFlexMappingApp besides how to use their API? How about how they handle a ton of geometry? Or if there really is a ton of geometry or just bitmaps? Do they use Sprites? And when? How do they orchestrate polling the server with rendering?
Well, maybe ESRI is a bad example. It's in their strategic interest NOT to reveal more than how to use their API.
But what about NATO's new Flex-fantastic super program? What about HSA's emergency apps? What an opportunity.
(Please?)
I think Brett's bringing up a good point here - there are tons of very badly performing Actionscript apps out there, and part of this is due to the developers inability to regulate processing at a low level.
Ted's point to this is that while that's true, there are some good reasons for doing it this way also and Adobe's making constant strides in beefing up their player. There are things developers can do now to improve the performance of their apps, and it's not really any more difficult than the things they'd do under other systems (like dynamic time slicing, for example.)
I have to agree with Eric - I'd love to hear how some of the best apps out there are making it work. The first thing that wowed me about Buzzword was how well it performs - I'd give a lot to get under the hood and see what sort of tricks their pulling to make it work
Dear RJ,
I really appreciate your blog!! :)
I've some questions for you.
I'm interested in flex life cycle and the role played by frames and SystemManager.
I've read that a Flex appl has two frames. Then you have a main loop which listen for user event.What does it mean (What is a frame)?
In particular, are there any differences between frame (understood as the unit that contains actionpscript code and used in CREATIVE SUITE) and frames (generally understood as the time slot between two consecutive enterframe events)?
Thanks in advance.
Best regards,
Lorenzo
Hi Lorenzo - "Frames" can be a tricky topic. If you've used Flash, which it sounds like you have, you know that Frames are played through the flash player as the thing seen on the screen. They're animation frames, just like the frames used in cartoons on TV.
Flash gives us the ability to programmatically define something that can occur in a frame rather than drawing it out by hand. We specify what to do in each frame in action script. Flash also gives us the ability to go back to the same frame multiple times so that we can keep all of our code on that frame.
Flex uses two frames. The first frame contains the preloader and has enough information to load the next frame (SystemManager is in the first frame.) The second frame contains the rest of your application, and the player repeatedly loops back to this second frame, re-playing it every time. If you didn't write any actionscript you'd see a static view of frame 2 as the player continuously replays the same thing, over and over and over. Usually we write action script on Frame 2, in the form of MXML and AS flex code, that tells the player what to do on every frame.
I hope that clears it up. It's an excellent topic, and something I should probably write another blog post on soon!
Dear RJ,
Thank you for the prompt reply! ;)
I think this argument is very interesting to understand.
In general, for example in CS4, a frame is considered as a logical unit (deimited by ShowFrame tag) where I can attach as3 code.
This as a simplified version of a swf file, i think:
* Header
* Symbols, classes etc... [FR1]
=== ShowFrame ===
* Symbols, classes etc... [FR2]
=== ShowFrame ===
End
If I have two frame, FR1 and FR2, Flash starts executing FR1 and goes to the next one (FR2). If I don't stop the excution, the Player executes FR1, FR2, FR1, FR2 and so on in a infinite loop.
On the contrary, when you stop the execution, for example in FR2, the flash player loops around FR2. Maybe, there is a sort of frame, like a slot of time, which represents a sort of "pseudo" frame(the period between two consecutive frames), which listens for events.
Isn't true?
How does this model fit into a Flex application?
Can I consider a frame like a logical unit which can do thing? The first one has the job to load (through the SystemManager) the application code and RSl, the second one (through the SystemManager) to visualize application on screen and to allow the user interaction. There is a frame tick running perpetually in my app (at the rate I specified), to which I can register events and on which code is executed.
Thank for your patience.
You're very helpful.
Best regards,
Lorenzo
Lorenzo: Yeah! You've more or less got it down perfectly, and that's a great way of explaining it. One difference: after Frame 1, frame 1 never executes again: it's FR2, FR2, FR2 for ever and ever and ever.
A frame is indeed one unit of time for execution in the player. Each frame, the player executes user code and then renders the output of that code to the screen. It then goes on to the next frame, where it executes more code and renders more to the screen. Each frame is a new period of time for the flash player, though they might be the same physical "frame" in the CS4 sense of the word.
Setting the framerate allows you to control (or try to control) how many of these frames you get per second, but you'll notice that rendering starts to lag if you have too much complicated logic in any single frame.
Ted Patrick did a really amazing post on understanding the frame model - you can find that here:
http://onflash.org/ted/2008/04/flash-player-mental-model-elastic.php
Sean Christmann updated this idea with more information on what's going on at a lower level, specifically important to Flex:
http://www.craftymind.com/2008/04/18/updated-elastic-racetrack-for-flash-9-and-avm2/
In this presentation on the component lifecycle, Brad Umbaugh and I reference this "elastic racetrack" idea of the frame model, and describe both how it works and why this is important to the lifecycle. Ted and Sean's post should explain it all far better than we do, but if you're looking for another resource try this:
http://tv.adobe.com/watch/360flex-conference/diving-deep-with-the-flex-component-lifecycle/
Here are real links for those links:
Ted Patrick's post on the Elastic Racetrack
Sean Christmann's updated Elastic Racetrack post
RJ Owen and Brad Umbaugh's presentation on the Flex 3 Component Lifecycle
I'm happy that it's all clear! ;)
Thank you very much!
P.S. I hope you will write a new post on soon.
Best regards,
Lorenzo
Frame is a misleading concept. Infact, I can say that a frame is a logical unit and, at the same time, a period of time between two consecutive enterframe events.
In the CS4 it's quite simple!!But not in Flex!!
I've reflected on it and I can't understand how the flex framework links the code with a specific frame. In particular, I know that a flex appl has two frames. The first one istantiates the SystemManager and the Preloader, the second one the Application. Ok!
So how does the player link a specific code execution (for example the istantiation of the SystemManager and the Preloader classes) with a frame?
On the contrary, in the CS4 I explicitly link as code with a specific frame.
I hope it's all clear!
Thank you!
Reagards,
Lorenzo
Hey Lorenzo - think of it this way: flash content and flex content both publish .swf files. .swf files contain frame data for display in the flash player.
In the Flash authoring tool (i.e. Flash CS4 or CS3) you have complete control over the timeline, and can attach code to different frames. Best pracitces usually dictate that you place all of the code for a movie clip in a separate layer called "actionscript", and if possible that you keep the actionscript on one frame (depending on what it does of course.)
In Flash Authoring you frequently direct the flash player to move between frames using "gotoAndPlay" or "gotoAndStop" methods in actionscript.
Let's consider a very basic sample application. This application simply detects the mouse's coordinates and displays them on the screen. In flash authoring, we make a 1 frame application. On our stage we have a text field and that's it. You set a listener to the enterFrame event, and on each frame, you simply get the mouse coordinates and set the text field accordingly.
In this case, you're processing the same physical "frame" of code over and over and over and over, but each time the flash player processes it counts as a "logical" frame to the player. The player interacts with the browser and renders data one logical "frame" at a time, even though it's the same physical frame you're using.
In Flex, the Flex compiler creates the frames for you, and you have no direct control over what display objects or actionscript are placed on what frames. Each swf produced by the Flex framework is only two frames long. On the first frame is a few sparse things including the system manager and preloader - EVERYTHING else goes on the second frame. You cannot control which frame your data is placed on directly (you could indirectly by monkeying around with the sysManager or preloader classes.)
Does that clear it up?
Thank you.
It's all clear, but what do you mean by "but each time the flash player processes it counts as a "logical" frame to the player"? In particular what does "logical frame" stand for?
It stands for "each frame is a new period of time for the flash player", I think. A sort of abstract or pseudo frame.
Then, setting the framerate allows you to control (or try) how many frames ("logical frames") you get per second.
In Flash Authoring this concept is less difficult than in flex.
Could you say if my reasoning is correct?
When the Flash authoring compiles a .swf file, it exports all code (classes) in the first frame of the swf. So you have to wait before using this frame.
A little trick is to instruct Flash to export classes at frame N.
I think the same situation is in flex. The flex Framework export all application code at frame 2.
Flex is a two frames application.
The first frame contains light code so it can be used soon. In this one SystemManager is created. SystemManager istantiates the preloader and then stops (pauses) the execution (preventing anything from Frame 2 from running until later). So in this situation FlashPlayer loops around frame 1 (is it the logical frame 1?) and when the application is fully loaded, the SystemManager goes to the frame 2. Here, the SystemManager istantiates the main application and removes the preloader object. Next, the application goes into its component life cycle (invalidation/validation mechanism, components too) and finally dispatches an applicationComplete event.
After, the flash player loops around frame 2 (is it the logical frame 2?) allowing the user interaction.
I saw your video presentation
- http://tv.adobe.com/watch/360flex-conference/diving-deep-with-the-flex-component-lifecycle/
Very good one!!
In this presentation you talk about "frame in as3" and "marshall slice". I think this is the logical frame you talked about in your previous post blog. So could I consider it as a sort of frame tick that performs a scheduled screen update?
Thank you for your patience! :)
You are very helpful!!
Lorenzo
Post a Comment