Comments on: Replacing ENTER_FRAME – Testing Tickers

By: jacksondunstan

jacksondunstan — Mon, 30 Nov 2009 22:46:44 +0000

Thanks for the clarification. I assumed that what you were seeing was roughly what I was seeing: inconsistent framerates on the same computer, browser, and player. On my 2.2 Ghz Intel Core 2 Duo with 2 GB RAM on Mac OS X 10.6 (Firefox 3.5) I was getting anywhere between 18 and 20 average FPS on both tests. I would let one of them run for 30 seconds, stop it, run the other for 30 seconds, stop it, and repeat. Sometimes the average would be 18, sometimes 20, and sometimes in between them. Neither test seemed more likely than the other to perform slowly or quickly. I even refreshed the page and tried it a few more times, but there was no change. From this it seemed likely that the random number generation was causing essentially random results, even if only slightly differentiating the two tests. Hence my suggestion to use a random number generator with a constant seed. I don't think any JavaScript injection would be necessary since you just need to make sure the seed is the same between them. From there it's essentially random and repeatable between both tests.

Since you pointed out differences between platforms and players, I just tested again on my 3.0 Ghz Intel Core 2 Duo with 4 GB RAM running on Windows XP (Internet Explorer 8). Here I get much closer results than on the Mac. Both seem to run at 17.8 FPS and only vary by +- 0.01 FPS. I should note that I never see any “spikes” in either performance or memory usage on any platform with any test. At least this shows that the FrameLoop ticker is no worse than the ENTER_FRAME ticker…

By: Simone

Simone — Mon, 30 Nov 2009 19:27:40 +0000

Hi Alec,

I just forked your repo and added another test case where I use a Timer object that ticks every 1000/defaultFrameRate milliseconds (to emulate the enter frame speed) and I didn't notice a big difference between the three approaches.

here is the repo
http://github.com/vizio360/TickerStrategies

By: alecmce

alecmce — Mon, 30 Nov 2009 16:47:14 +0000

Thanks for the feedback, Shaun.

Yes, I wondered whether the weight of the test might be an issue, but I wanted to ensure that it would run at slower-than-31fps in order to test performance. If I don't find the point at which the racetrack comes under strain I'm not sure how to differentiate the performances. However, perhaps I shouldn't put it under so much strain. Do you have a suggestion? I will try with 2,500 sprites and see how they run.

By: alecmce

alecmce — Mon, 30 Nov 2009 16:20:40 +0000

As ever Jackson, thank you for your comment! I am extremely grateful that you read my work carefully and point out contentious points.

When I mentioned that I get "different results" I was really referring to the different results I get on different platforms, different players, and so on. The gist of the results (that the Frame Loop appears to use marginally less memory but doesn't appear to have a performance benefit) seems consistent. However I want to be scientific about it so put in the proviso that I do get inconsistent results and I need to be more rigorous.

The random number generation issue is an interesting one. Your repeatable random seed generator is terrific for the sorts of scenarios that you offer such as gaming where you need to ensure that each time you repeat a scenario 'random events' happen in a consistent way. In this case, I can guarantee that events happen in roughly the same sort of way (and that the 'roughness of that is actually pretty smooth) by sheer weight of randomness. There are 5,000 sprites, to which a random acceleration vector is applied (2 random numbers) every frame. That's 300,000 random numbers being applied to the model every frame. Over the course of the animation, the aggregate velocities of the elements will approach a normal distribution around zero-velocity. (I'm going to let that point dangle and point to http://en.wikipedia.org/wiki/Central_limit_theorem).

As you say however, I could implement your (or another - as well as Mersenne Twister, also consider http://en.wikipedia.org/wiki/Rule_30) repeatable random seed by creating a random seed in JavaScript and injecting that seed into the two Movies to guarantee that they run in parallel each time a user runs them. It would be a pleasing addition to the test, though I very-much doubt it would change the results. I will add it, if I get the time.

By: shaun

shaun — Mon, 30 Nov 2009 14:55:10 +0000

I also wonder if the graphic intensity is throwing off the test results – my CPUs get maxxed out, so perhaps any gain from the frame loop is being negated by the elastic racetrack.

By: jacksondunstan

jacksondunstan — Mon, 30 Nov 2009 06:11:59 +0000

I think the different results you're seeing may be a result of the heavy use of Math.random(). This is, by nature, introducing a random element into your tests and making them unrepeatable and unverifiable. You might want to try a repeatable random seeded with a constant value in both tests. This allows you to process (virtually) unpredictable numbers and still have some repeatability. Frankly though, I'm not sure why random numbers would be necessary at all. This becomes clearer if you test out other performance paths (like you suggested above), such as those with no graphical elements at all.

Thanks for these articles, tests, and the open code. I'm interested to see if all this discussion leads to any measurable performance increase. If it did, I think a lot of AS3 developers would switch their main ENTER_FRAME loops. I know I sure would!