This bug is to track progress on my GSoC Project to integrate the Firefox devtools into Thunderbird either directly or via remote connection. I propose to modify Thunderbird to allow using all current remote features of the Mozilla Developer Tools. This includes the web console, debugger, style editor and network monitor. The key difference between my proposal and the past solutions is that no extra UI is needed within Thunderbird which might end up breaking because Firefox-specific features are being used. Instead, I will ensure the existing components are fully free of Firefox specific code and write the wrapper layer needed to make use of the backend components.

I was just about to look into how to do this myself when I saw a review for bug 870081 pop up. This will make things much easier for creating a mail actor.

It would be nice to have bug 805526 fixed so we can also have a remote inspector

So, here is my first update and a firm plan on how I would like to go about things. I have spent the last days reading all the toolkit devtools code to find out what possibilities we have and what needs to be done. The devtools team really deserves something shiny for their architecture and thanks to the existence of b2g their code is written to work with more than one application. The dates are oriented at the GSoC dates and rather optimistic. I believe I will be able to achieve some of these milestones earlier. Milestone 1: June 3rd - June 30th * Add UI to Thunderbird to start the remote debugger (Tools menu) * Initialize the debugger server correctly, allow remote connections * Refactor chrome debugger and browser actor (this was my plan, but is now fixed by bug 870081) * Make chrome debugger work for the main window * Make sure it also works for secondary windows * Add a "tab" actor per displayed email to be able to access content documents Milestone 2: July 1st - July 29th * Make WebConsoleActor/StyleEditorActor code in toolkit independent of window * Hook up the web console to Thunderbird's console via WebConsoleActor * Hook up the style editor to Thunderbird via StyleEditorActor -- Midterms -- Milestone 3: August 2nd - August 16th * Make the profiler work for chrome code * Make sure the profiler works for content code (if needed?) * Hook up the network monitor Milestone 4 (bonus): August 2nd - September 9th * Investigate the possibility to add imap connections to the network monitor * Find out if the gcli actor makes sense for Thunderbird and is remoteable - Alternatively, consider adding the gcli to Thunderbird itself. * Check if Inspector is ready for remote support yet * Integrate Scratchpad Milestone 5: September 10th - September 23rd * Clean up UI * Complete writing tests

First success with the debugger on bug 881235! That patch will allow chrome debugging on the main process. Things are going much better than I even anticipated. Next up I will hook up any content windows to the "remote tabs" section of the debugger, this should be a matter of enumerating the windows, searching for content windows and yielding an actor grip for each of them.

bug 875104 will be actually using the tabListChanged notification. Note to self: check if the Thunderbird tab list implementation still works afterwards.

Some other random notes from discussion today: * As a side note, see this project: https://wiki.mozilla.org/Sepsis * The code for the debugger is landed in mail/components/debugger, but it would be even better if this were in mailnews so that Seamonkey can also make use of it * Going one step further, it would be nice to (also) have an extension that works on all xulrunner type applications. Creating the extension from the source directory would make things easier. * There is no need to create Makefile.in's with no directives other than the includes Also, I've blogged about the matter here: http://kewisch.wordpress.com/2013/06/13/the-thunderbird-remote-debugger-is-alive/

Milestone 1: June 3rd - June 30th ================================= > * Add UI to Thunderbird to start the remote debugger (Tools menu) > * Initialize the debugger server correctly, allow remote connections > * Make chrome debugger work for the main window Fixed by bug 881235 > * Refactor chrome debugger and browser actor As mentioned, fixed by the devtools team in bug 870081 > * Make sure it also works for secondary windows I've tested this and it generally works. If you open the window, set the breakpoint, then its triggered as expected. If you close the window and open it again, the breakpoint is not triggered again. I've filed bug 883342 for this, it also fails for Firefox. > * Add a "tab" actor per displayed email to be able to access content > documents Will be fixed by bug 883311. I've also fixed two very trivial bugs in the devtools code. I will monitor bug 883342 and possibly find a solution for this, but aside from that I feel Milestone 1 is completed. Since there is still some time before the next milestone is scheduled, I will take the time to figure out how to best proceed w.r.t Seamonkey and xulrunner. It would be nice to be able to have it work with those kinds of apps too.

bug 825039 will solve Milestone 4's integrate scratchpad, it will be possible to execute scratchpad in the context of the current debugger, which will mean Thunderbird.

Calendar uses a lot of XBL, it would be nice to have bug 777674. Looks like its still a little bit away though.

Not adding this as a dependency, but just another interesting use case: bug 809561.

Just in case you didn't read all dependent bugs: code will likely move into an extension in bug 884805.

The network monitor just works out of the box!

\o/ Fantastic work, Philipp. This is excellent blog fodder, btw.

Milestone 2: July 1st - July 29th ================================= > * Make WebConsoleActor/StyleEditorActor code in toolkit independent of window Patch is up in bug 880511. > * Hook up the web console to Thunderbird's console via WebConsoleActor > * Hook up the style editor to Thunderbird via StyleEditorActor Both actors added as part of bug 897476. Also, in Devtools bug 872897 I fixed a problem with imported style-sheets which are quite common in Lightning. > -- Midterms -- Coming right up :) > Milestone 3: August 2nd - August 16th > * Make the profiler work for chrome code Part of bug 897476. > * Make sure the profiler works for content code (if needed?) I don't have a good content tab to test this on, but given hooking up the profiler was just a matter of adding the actor, I assume this works. > * Hook up the network monitor Works out of the box, yay!

(In reply to Mike Conley (:mconley) from comment #16) > \o/ Fantastic work, Philipp. > > This is excellent blog fodder, btw. Definitely, I will write a blog post on these soon :-)

Ok, let me tell you a little about the post-midterm milestones: > Milestone 4 (bonus): August 2nd - September 9th > * Investigate the possibility to add imap connections to the network monitor Filed bug 897564. This could be pretty tricky and might exceed the scope of a GSoC project. I haven't confirmed this with anyone, but it looks to me like this needs to be done on a per-channel basis, implementing nsITraceableChannel on the mock imap channel, somehow ignoring the cached entries and using the observer service to notify when imap requests are being made. The alternative would be a general mechanism that allows tracking any network connections, this goes down pretty deep into gecko. > * Find out if the gcli actor makes sense for Thunderbird and is remoteable > - Alternatively, consider adding the gcli to Thunderbird itself. Yes, gcli is remoteable, but it contains _a lot_ of Firefox specific code and dependencies. I've filed bug 898278. > * Check if Inspector is ready for remote support yet bug 805526, not ready yet, but dcamp is working on it. > * Integrate Scratchpad It will be possible to use scratchpad remotely, likely without any extra Thunderbird work, when bug 895180 is fixed. > Milestone 5: September 10th - September 23rd > * Clean up UI > * Complete writing tests I'm going to use this milestone for completing the extension packaging work. The idea is to take the code currently in dbg-messenger-overlay.js and package it into a module. Then I can just call these methods from either the builtin overlay or the extension's bootstrap.js. I will also refactor the toolkit DebuggerServer's addBrowserActors() to something like addCommonActors() which contains all actors that are supposed to work regardless of the app and then just call that from our init code. This way, if an actor is added in the future it will automatically work with Thunderbird too. Given there is not much code on the Thunderbird side, the number of tests won't really be extensive, but I guess I could add a test that starts the debugger server, makes sure its initialized and that there are no console error messages.

Fixing bug 899006 will allow us to slim down the code in mail/ so that any new actor that is added by the devtools team will automatically be available in Thunderbird.

Hello Folks, sorry I haven't given you an update lately. Here are some details: I'm working on getting the changes in bug 899006 through, this has been delayed a bit. As mentioned previously, this will make it exceedingly easy to manage the devtools from Thunderbird and makes this project futureproof. Regarding showing IMAP connections, mconley and I have agreed that this has fallen out of scope. To do it right, I would need to implement a general way to make network connections within the Mozilla platform traceable, which requires an experienced Mozilla network team member to do correctly or at least could be a gsoc project on its own. The devtools team has set the respective bug (see above) to P3, so maybe this is something that could be done as joint effort. I will definitely stay on this even after GSoC. Next up is gcli. I've already mentioned that its possible, but I haven't given any details yet. With the code I have now, its possible to connect gcli to a remote Thunderbird instance and send commands. I've been able to work around all visible error messages but am now at a point where there are no error messages but sending commands doesn't bring a response yet. I will continue to debug this. Scratchpad is going forward too, there is a patch from bbenvie that adds a scratchpad tab to the debugger toolbox and uses the remote connection. Its an early WiP patch, but it looks very promising. I have postponed the extension packaging until bug 899006 is fixed since it will greatly influence how the code looks like. 3 of 4 Milestones have already been reached and the remaining stuff doesn't look so complicated, therefore I am confident that I can reach all milestones in time for the finals.

Scratchpad works great with the WIP patch in bug 895180!

w00t!

This is the greatest. :)

Ok, I think I am set for the pencils down date. Here's the status: Patches for bug 880930, bug 884805, bug 897476 and bug 915444 are in review. This covers everything from my side, execpt for the gcli integration. During creating the gcli patch I have been blocked by another bug in the gcli code where I couldn't continue. I don't have a bug for this, but I was promised it would be fixed soon. Aside from that, we have decided gcli is no longer a target for the end of the SoC. There is really a lot of code in browser/ that should be in toolkit/. I have started to move some of this code, but its not easy to separate everything. As gcli is not part of the toolbox but has its own connection mechanisim, I think its more of a secondary goal. As you can see in the above screenshots, inspector and scratchpad also work nicely. There are still some quirks in the inspector support, but I believe these will be fixed by the devtools team anyway. I will create another blog post to wrap things up on Wednesday.

Mike, if there is anything else you think I should cover, please let me know!

Here is the wrapup blog post: http://kewisch.wordpress.com/2013/09/22/thunderbird-developer-tools-wrapup/ Marking this bug as FIXED, I will continue my work after the Summer of Code to make sure nothing breaks and new features in the Firefox Developer Tools are also available in Thunderbird. Thanks to everyone who supported me during the time!

Very, very, very cool. Sorry if I missed it, but are the new install steps? You mention an extension in the wrapup post, but didn't give the link or updated install steps from your first post. Also, what versions are required? Thanks!

I just tried the install instructions from your first blog post. All the steps went fine, but after I click connect Firefox never makes it past the "Connect to remote device, Connecting…" page. I ran wireshark on the loopback and here is the result. I am seeing this with both the 9/24 and 9/26 nightlies on OSX 10.8.5. 124:{ "from": "root", "applicationType": "mail", "testConnectionPrefix": "conn2.", "traits": { "sources": true } }40:{ "to": "root", "type": "listTabs" }760:{ "error": "unknownError", "message": "error occurred while processing 'listTabs: TypeError: tabList.getList is not a function\nStack: RootActor.prototype.onListTabs@resource://gre/modules/devtools/dbg-server.jsm -> resource://gre/modules/devtools/server/main.js -> resource://gre/modules/devtools/server/actors/root.js:233\nDSC_onPacket@resource://gre/modules/devtools/dbg-server.jsm -> resource://gre/modules/devtools/server/main.js:1018\n@resource://gre/modules/devtools/dbg-server.jsm -> resource://gre/modules/devtools/server/main.js -> resource://gre/modules/devtools/server/transport.js:198\n@resource://gre/modules/devtools/dbg-server.jsm -> resource://gre/modules/devtools/server/main.js -> resource://gre/modules/devtools/DevToolsUtils.js:72\n" }

Just in case it was caused by the bump of Firefox nightly to 27 I also tried the 9/14 nightlies (26.0a1 2013-09-14). I'm seeing the same problem. Thx.

Please see the dependent bugs. Not everything has been checked in yet, the error you describe is caused by an API change in the developer tools protocol. So at the current moment the steps are: 1) build thunderbird 2) apply remaining patches 3) use latest ffx nightly. This will of course get easier, I will do another blog post when its just a matter of using the latest nightly versions. You don't need the extension for Thunderbird, its just useful for other Apps like XULRunner or possibly Seamonkey (although I think they were doing something similar as I am for Thunderbird). I have some end user documentation here: https://wiki.mozilla.org/User:Kewisch/Thunderbird_Developer_Tools_Documentation It will be moved to MDN when we figure out a good place to put it.

Success! I applied bug 897476 and bug 915444 and built tbird and I am now able to inspect/debug. Thank you!

What's the current status of this work? Why hasn't this made it into the release version of tbird?

(In reply to Dave Huseby [:huseby] from comment #37) > What's the current status of this work? Why hasn't this made it into the > release version of tbird? Did you read the blog posts referenced. This bug is closed because it is complete. The only Thunderbird UI change was the allow remote debugging entry on the tools menu. The rest of the debugging process occurs from Firefox.

Pure debugging is available on Thunderbird 24 already, the full suite was added in betas shortly after and is therefore also part of Thunderbird 31. Please see the dependent bugs for the real work. For a guide on how to make it work see: https://developer.mozilla.org/en-US/docs/Tools/Remote_Debugging/Thunderbird

Philipp, dude , thank you for that work, no really I mean thank you.