stats with attached test case: monitor: exits(30002), timeouts(0), type mismatch(0), triggered(30002), global mismatch(0), flushed(0)

We need the real fix here, the old code is all bogus. This patch tries to detect if we don't need to worry about a tree call being self recursive. If we do have to worry, it will either disallow recursion or adjust |InterpState::sor| to prevent popping illegal frames. With this, the test case is now much faster: monitor: exits(191), timeouts(0), type mismatch(0), triggered(191), global mismatch(0), flushed(0)

This version should be a little more optimal. It always emits the start-of-rp-stack barrier if the inner tree is recursive. Then the unwinding process can fast-check for whether there are no more stack frames, which lets it give back a LOOP_EXIT to outer trees faster. The downside is both of these patches are a 2-3% regression on 3d-cube (3ms on buildmonkey-right, overall 0.6% perf loss). The reason is we're tracing more - the code before was quite wrong in two ways: * A loop that called into a recursive function would never compile. * All the protection against self-recursive trees was bogus. Perhaps with the magic combination of this patch, tail call tracing, and merge traces - 3d-cube will get somewhere better. But this patch fixes a potentially nasty bug in recursion, and enables tracing more control flow (the attached test case is 3X faster). So I don't really want to tune too much to SunSpider here, but I'll investigate some more optimizations tomorrow.

Comment on attachment 414474 [details] [diff] [review] better fix I don't know why this patch had a self+ on it in the first place, but it doesn't work anyway.

The code to allow recursion and loops to interact is totally broken - it just doesn't compile. This patch fixes that by giving trees one of four states: - Recursion disallowed (only loops may be compiled) - Recursion expected (only recursion may be compiled) - Recursion unwinds (up recursion, no down yet) - Recursion (up and down recursion) "expected" may change to either of the latter two states (the only reason there are two is so we can choose to not enter solely up-recursive traces). A loop trace can never be recursive and vice versa. Before this wasn't properly enforced, and a loop trace could compile into recursion. Now this is not possible. In addition, this patch makes it safe for any tree to perform a tree call on a recursive trace, or even perform actual recursive tree calls. InterpState::sor is adjusted to avoid pulling frames out from underneath child trees.

With this patch, the test case runs in 16ms instead of 70ms, and gets 160 exits instead of 30,000. I haven't benchmarked SS yet, I suspect it will be a very small loss because of 3d-cube.

SS results: http://pastebin.mozilla.org/690990 Mostly noise, 3d-cube gets slightly slower as expected but other results make up for it.

attemptTreeCall now avoids potential frame slurping which could prevent an outer tree from compiling.

This one reintroduces a small heuristic to keep us from tracing patterns in early-boyer that end up not doing well.

This one is rebased and has some newer, less restrictive heuristics from JM. It appears to be a 1% win on SS and v8.