One possible idea to add to the new debugging APIs is to include an API that would allow developers to be able to get code-coverage results for their code, without having to specifically do source-to-source instrumentation themselves. At the very minimum, I would like to see function and line coverage, and branch coverage is highly desired in my eyes. I could probably live without getting full counts of everything, but I've found that having counts can help divine code paths in some circumstances.

I remember sfink messing with the call/return hooks, and the discussion concluding that the existing debugger call/return hooks couldn't be used for profiling because they revealed JSStackFrames, and making those correctly for every call in JM would kill performance --- even though the profiler didn't need them. The solution was to introduce a separate set of call/return hooks that were only useful for profiling, and just provided the function name, not the top frame. Or something like that. I think this is the js::Probes API. It would be awesome to have those available to JS via a Debug-like API, but perhaps it should be something separate from Debug, since (just as mentioned above) Debug provides details that make life hard for optimizers. "Optimization is lying without getting caught."

Function name isn't necessarily good enough; anonymous functions abound a lot. A way to tie the function name to a specific location (e.g., <file>:<line>:<col>) is better. Alternatively, inferring/creating debug names for anonymous functions might work.

(In reply to comment #2) > Function name isn't necessarily good enough; anonymous functions abound a lot. > A way to tie the function name to a specific location (e.g., > <file>:<line>:<col>) is better. Alternatively, inferring/creating debug names > for anonymous functions might work. Look at js/src/jsprobes.h; it gets you filename and line number as well.

The jsprobes stuff gets surfaced through JS_SetFunctionCallback in jsdbgapi.h. That will give you a callback on every function call and return. You can get the script filename and line number and the function name if available. (The line number is the line of the function definition, not the call.) This will not give anything more granular than functions, though -- you can't use it for line or branch coverage. In bug 637393, I have an uncommitted patch that collects per-op counts, and even divides them up between TM/JM/interpreter. This would give you lines and branches. I was planning on using that mechanism to create a simple code coverage tool once I make it through a couple of things that are on my plate right now. I have not done any work on exposing the information to JS, though. I'd love to talk about what you'd use it for and how best to expose it. (And I'd like even better to hand off as much as possible to someone else!)

(In reply to comment #4) > In bug 637393, I have an uncommitted patch that collects per-op counts, and > even divides them up between TM/JM/interpreter. This would give you lines and > branches. I was planning on using that mechanism to create a simple code > coverage tool once I make it through a couple of things that are on my plate > right now. That looks excellent! > I have not done any work on exposing the information to JS, though. I'd love to > talk about what you'd use it for and how best to expose it. (And I'd like even > better to hand off as much as possible to someone else!) What a coincidence, I too was hoping to hand off as much as possible to someone else! The real use case I want to go for is making integrated code coverage like this: <http://ltp.sourceforge.net/coverage/lcov/output/index.html> (although I hate the UI it gives on so many levels, but I digress), across both the C++ and JavaScript code in Mozilla during test suites. Basically, that means I need at a minimum: * A way to enable profiling for all JavaScript execution henceforth * A list of all functions, run counts (at least 1/0 counts, although more is preferable if feasible), and the location in source code of those functions * A list of all executable lines (i.e., statements) in the code, and run counts for them. * Branch coverage per line I would prefer: * A mechanism to still get profiling even on Mozilla segfaulting mid-test * A way to combine data with previous test runs * An easy way to identify which expression branch taken/not taken corresponds to * Easy indication of which target of a switch was taken Being able to call from JS or dumping to an LCOV format falls under a "nice to have" for me, but that is well within the realm of what I am willing to code on top of profiling APIs. In addition, getting finer-grained coverage (i.e., expression instead of statement/line coverage) might be nice to have, but I do not look forward to making into making a usable front-end for this information. Just trying to account for branch information has been giving me a headache (the [+ -] format in LCOV doesn't make a whole lot of sense... hmm, maybe if I made :hover magic?) In terms of a concrete API proposal, I don't have anything more than a vague idea that it would hang off the jsd2 stuff.

A command-line code-coverage tool based on sfink's -D patch: https://github.com/davidflanagan/CoverMonkey

(In reply to Joshua Cranmer [:jcranmer] from comment #0) > One possible idea to add to the new debugging APIs is to include an API that > would allow developers to be able to get code-coverage results for their > code, without having to specifically do source-to-source instrumentation > themselves. This is being addressed in Bug 1176880. (In reply to Joshua Cranmer [:jcranmer] from comment #5) > The real use case I want to go for is making integrated code coverage like > this: <http://ltp.sourceforge.net/coverage/lcov/output/index.html> (although > I hate the UI it gives on so many levels, but I digress), across both the > C++ and JavaScript code in Mozilla during test suites. What a coincidence, I just added this Bug 1191289, and plan to make it available within Firefox with Bug 1204554.