This is intended to be a meta-bug for major architectural changes that I think are needed in layout. I intend to file dependent bugs later, with more detailed descriptions. (Some of these bullet points are clearer than others, and I could expand almost all of them to at least a few paragraphs.) However, for now, I'll just paste the list of thoughts that I have (which I've been adding to for a few weeks now since I originally wrote it) into the bug. Some of these things are cleanups that need to happen for our sanity, and some of them are more along the lines of prerequisites for implementing new concepts from CSS3 or other sources. Prerequisites: * Rewrite block-within-inline handling ("{ib}") to happen during reflow. [ for FC and reflow cleanups ] (bug 142585) * Switch to XBL form controls [ for reflow cleanups ] * Finish moving form control data out of layout (text controls!). Clean up central objects: * Design the loading process to look like the way it should work. (e.g., waiting for stylesheets should delay frame construction, image loads should start from content / stylesheets, old page should go away when new one has something to display) (bug 84582 / etc. / etc.) + Global object proxying to allow loading documents to be behind the scenes. + Eliminate timers all over the place and replace with a well-thought-out system. * Eliminate redundancy (e.g., pres shell / pres context). * Per-medium singletons, allowing sharing/prototypes/fastload of some style/layout objects? (bug 117568) Clean up style system: * Less bloaty CSS stylesheet backend (code size and object size) (bug 125246 / etc.) * Solve the dynamic reresolution problems (ReResolveStyleContext) (style tree must be isomorphic to content tree, not rendering tree) * Clean up user/UA stylesheets (bug 179006) Clean up frame construction / frame tree: * Make everything use 'display' -- no construction by tag. Use things like -moz-listbox instead. * Implement in terms of 'display-role' / 'display-model'. + to allow code consolidation and refactoring (splitting of monsters like nsBlockFrame) in reflow code + Make marginally extensible (arrays of factory functions, registered and placed in hashtable?, perhaps with more of the initialization work in the frame classes rather than in the constructon code). Clean up reflow: * Separate intrinsic width (maximum width) and minimum width (max element size) calculations from reflow (See bugs marked [reflow-refactor] in SW, although with numerous caveats relating to the way setting 'width' on a block still allows its descendants to influence table sizing) * Implement in terms of 'display-role' / 'display-model'. (see FC above) * Make the recursion of reflow move along the content tree rather than the frame tree (http://bugzilla.mozilla.org/show_bug.cgi?id=172031#c16). (Should we still use real recursion to do reflow? Do we want interruptable reflow?) [ This may depend on CSS WG decisions about 'overflow'. ] Clean up rendering: * Fix units/scaling mess. (bug 177805 / bug 153080 / bug 63336) * Allow anything to have a view, and make it not break correct layering (e.g., for floats). Other: * Fix the event state manager not to use frames (bug 130620, etc.)

I have big reservations about the currently proposed split for the display-role/ display-model properties (as I mentioned in Mandelieu). I think we have a great opportunity here to study the problem and give feedback to the working group about what a better, more logical, and less redundant split might be.

> Make the recursion of reflow move along the content tree rather than the > frame tree (http://bugzilla.mozilla.org/show_bug.cgi?id=172031#c16). I doubt this is a good idea although I could be convinced. > (Should we still use real recursion to do reflow? Do we want > interruptable reflow?) Yes and yes. We can and should leverge our incremental reflow architecture to get interruptible reflow. Building some sort of explicit reflow stack will not make interruptible reflow easier IMHO. > Allow anything to have a view, and make it not break correct layering > (e.g., for floats). We're pretty close to this already. I don't know exactly what the float layering issues are but I do know that every float should have a view and I'm confident any layering issues can be resolved in the view manager without a vast amount of work.

Some additional ideas from bzbarsky: [19:31:45] <dbaron> bz_gone: I want to know what your list of things to fix is. [19:32:21] <bz_gone> dbaron: hmm... at the top of my list right now are sane apis for notification of changes to CSS (sheets and rules) [19:32:35] <bz_gone> dbaron: batching of sheet changes (eg for alternate switches) [19:32:46] <bz_gone> dbaron: async loading of sheets [19:33:06] <bz_gone> dbaron: the fact that frame construction is unreadable and fragile [19:33:43] <bz_gone> dbaron: Image loading from content and style system instead of layout [19:33:45] <dbaron> bz_gone: In the grand scheme of things, I don't find frame construction code very fragile, actually... [19:34:01] <bz_gone> dbaron: it's mostly fragile due to code duplication.... [19:34:12] <bz_gone> dbaron: so you have to make the same change in multiple places if you change things [19:34:27] <dbaron> bz_gone: ok, sure. [19:34:35] <bz_gone> dbaron: then there's the whole mess with the MIME service, stream converters, etc [19:34:53] <bz_gone> dbaron: I have a bunch of problems with that [19:35:16] <bz_gone> dbaron: ranging from our lack of support for args to helper apps [19:35:38] <bz_gone> dbaron: to arch issues with detecting .ps.gz files from file:// and ftp:// as postscript [19:36:24] <bz_gone> dbaron: and I'm _still_ hoping to end up with one codepath for saving files instead of the current 3.... [19:36:48] <bz_gone> dbaron: that's that major arch stuff, I think.... [19:37:03] <bz_gone> dbaron: oh, and nsCSSDeclaration.... ;) [19:37:20] <bz_gone> dbaron: though that's more on your hit list [19:37:27] <dbaron> bz_gone: mind if I paste this into my bug? [19:37:42] <bz_gone> dbaron: not at all. [19:38:26] <bz_gone> dbaron: oh, and async loading of sheets means ReResolveStyleContext needs to be as good as possible for those cases when we _do_ start constructing frames before the sheet loads... [19:39:24] <dbaron> bz_gone: in most cases we should delay frame construction [19:40:01] <bz_gone> right [19:40:15] <bz_gone> but in some cases we'll still end up doing it... (if the sheet is very slow) [19:40:32] <bz_gone> oh, and on the topic of stylesheet notifications, nsIDocumentObserver in general is weird [19:40:47] <bz_gone> eg the content sink has to know about frame construction in the current scheme... [19:41:20] <bz_gone> so making nsIDocumentObserver be a little more useful is on the list too

Getting gecko to properly support pagination (page-break-inside, etc) is a prerequisite to properly supporting the 'projection' media. It would be nice to also support multiple page widths, which I understand we have zero support for right now. jkeiser said some work was already progressing in this direction?

See bug 15608 comment 26 (bzbarsky) on dynamic changes and subtrees.

PresShell / PresContext combining is bug 154199. As to moving data out of text controls, that can only be done by sacrificing performance or making huge changes in the plaintext editor. Specifically, it would have to simply operate on the text chunk in the textarea, and the textarea frame would have to directly display that text chunk with -moz-pre-wrap. This would actually improve performance probably, since there's much less construction / tear-down cost. I have one more item: * take the splitting stuff out of layout, or modularize it at least This is the source of huge and increasing complexity in layout. A new printing architecture currently under discussion (use clipping during paint instead of splitting during layout to decide what parts of an object show up on a page) could get rid of this. This will cause some harm the CSS3 multiple-page thing Hixie pointed out, though that would be a huge deal to implement anyhow (multiple sizes for blocks??). I'd appreciate if we could wrangle that out in the appropriate bug rather than here, though. One of my favorites on this list is: > * Make everything use 'display' -- no construction by tag. Use things like > -moz-listbox instead. My opinion is fairly similar to your next points. If we are going the route of display construction only, I'd say: * move all special frame construction logic out of nsCSSFrameConstructor and into singleton display-type-constructor objects * allow these display-type-constructor objects to be registered with nsCSSFrameConstructor as a map from display type to constructor object, for extensibility reasons (mathml and svg anyone?). I have one more idea to throw out here, a bit more radical. It stems from the brainstorm, "what would happen if all our layout objects were the same object--a 'tile' containing x/y/width/height and a few other items of layout state?" We move the reflow and painting algorithms out into singleton objects--one per display type. Reflow becomes a matter of destroying and recreating, or possibly just modifying, the tile associated with the content. This also makes it easier to plug new display types in, and makes things fairly fast (no virtuality necessary). In order to do this, we would need to first do two things which are both good on their own: * move style contexts to content (or at least keep them around when their frame goes away and make them accessible when there *is* no frame) * move *all* state that needs to persist longer than the frames, out of frames and into content (or somewhere else besides frames, at least). dbaron has a few of these specifically in the top of this bug (form controls specifically)

> "what would happen if all our layout objects were the same object--a > 'tile' containing x/y/width/height and a few other items of layout state?" I don't really understand how this is different from what we have today, except that it would require an extra pointer to the frame-type-specific object.

> what would happen if all our layout objects were the same object--a > 'tile' containing x/y/width/height and a few other items of layout state? I found out yesterday that this is exactly what XUL does. Each box holds a pointer to a BoxLayout object (there are a few different layout objects, eg SprocketLayout, GridLayout, etc) and asks it to handle the layout when needed. The win is that a single type of box object can be used for many different purposes, since all the reflow logic is encapsulated inthe layout object (which is a stateless singleton that just handles the reflow).

Note bug 139912, which the display frame construction stuff would certainly fix.

An additional thought: nsObjectFrame could be redesigned so that it delegates *everything* to another frame -- either (for alternate contents) a frame construted for the object by display type or a frame (nsImageFrame, nsFrameFrame) for the object.

We need to simplify the way we handle incremental reflows -- reflow commands could probably be eliminated in favor of dirty bits and a parameter that says "blow everything away since an ancestor was dirty". (This is certainly the way I'd plan to handle min-width and max-width calculation.)

That's odd. kairo@kairo.at: Your last change wiped out BugsThisDependsOn, Alias,Keywords,Summary(!)

restoring bug fields after kairo@kairo.at's strange commit

A few more thoughts on incremental reflows (I think I wrote something like this in another bug, but maybe not): There are only a few cases for which we need to optimize: 1. incremental loading of pages 1a. specified style changes to very small parts of pages 2. resizing of the window 3. changes to 'top', 'left', 'right', and 'bottom' Pretty much everything else doesn't matter. In particular, any changes to the specified style for an element should mean *everything* inside that element gets recomputed. The optimization to make in (2) is not to recalculate intrinsic widths (min/max widths), which would be simpler to do in a world where min/max width computation were separate from reflow. (See bugs marked with [reflow-refactor] in the status whiteboard for other things that such a refactoring would fix.) It would be nice if the optimization we'd need to make for (3) meant we could just move things and recompute overflow areas. I think this was the original intent of those properties. I also thought this when I commented in bug 157681. However, I'm afraid that's not actually the case, and the rules in the current CSS2.1 draft are correct as far as WinIE compatibility goes. The optimizations necessary for (1) are a bit more complicated. I listed (1a) as a subcategory of (1) because in our current implementation all the (relatively minimal) optimizations needed for (1a) follow from those necessary for (1). In other words, the optimizations necessary for making incremental page loading work reasonably mean that changes to arbitrary style information for small parts of the page don't take time that scales with the size of the page (in any significant way, at least -- perhaps some tiny component). I suspect that this would be true in any good design to handle the things necessary for (1). The optimizations necessary for (1) should maintain the invariants that: A. The page always ends up displayed the same way, no matter what the units of incremental layout were during its load B. The result after any incremental load of part of a page should be the same as if that part were the whole page (excluding unloaded images, etc.) Otherwise we confuse authors and encourage hacks to prevent incremental reflow. Incremental loading of content requires recomputation of preferred widths and of sizes for almost any content that has descendants modified. More on this later, perhaps...

How about 4. Inserting and deleting content during editing operations ?

Right, except it's likely to be 1b.

Another thing that would be nice, that isn't really an architecture issue, but more an implementation issue, is to not let the reflow-code hold strong references to nsIContent objects. Once bug 215981 is fixed it is possible to traverse an entire tree without ever calling addref or release on a node. Of course this is only possible as long as you know that the tree won't change during the traversal, but that shouldn't be possible to happen during reflow, right?

This is already happening in bug 190735. For example, almost all calls to nsIFrame::Getcontent() no longer add a reference.

I ask the contributors here to add some further thoughts for the users of your works when considering future changes to the layout engine. I speak as a technical communicator who must translate your decisions into conceptual material that is at minimum comprehensible, but that ideally is also conceptually clean. I'm speaking from the experience of constructing a book on Mozilla; gratuitous adverts for that book may be found elsewhere. In particular I'm speaking for non-embedded(XPFE) learner programmers, who occassionally are exposed to: frame and frame-like artifacts of the layout engine, artifacts related to the Model-View-Controller pattern, and the extensions to that pattern (builders, renderers, etc) that are required for a full implementation. It is my job, or part therof, to make a serious attempt at easing the burden of learning these concepts. One way to do that is to encourage you to make those concepts non-burdensome in the first place. In the current design (and I accept that exploiters of the layout engine may be more responsible than designers of that engine), some aspects of layout are quite confusing to beginners/learners. One sentence of truisms: no programmer successfully enters the Mozilla diaspora without learning; and Mozilla technology benefits the most from well-informed programmers. The first confusing aspect is that of (my terminology) extrinsic versus intrinsic layout elements. The property pattern used to specify layout properties (in CSS) is a flat hierarchy, and it is non-obvious which of these properties are of interest to XUL/CSS/JS programmers, and which are implementation details handily exposed. A big ask to you all is to (A) somehow taxonomically group exposed parts of the layout engine into "those bits XPFE programmers might conceivably use" and "those bits that should stay buried". For learners, we once had some grouping of this nature with pseudo-classes, which look at intrinsic aspects of the layout (eg :first-letter digs into a block/box constructed from more than one layout primitive), but various -moz enhancements now also grant access to intrinsic-like aspects of the layout, so the line is blurred. Could there be some grouping of those intrinsic elements? On top of all that, learners need an entry-level concept to cling to, and that concept is currently the "frame". So another big ask is to (B) constrain your re-architecting so that the concept of frame retains some approximately correct meaning for learners. Regardless of whether you believe that frames/MVC are discrete concepts or not, and regardless of whether they are discrete from the nitty-gritty of implementation, the pragmatic fact is that they are early rocks that learners will cling to when attempting to access the technology. Anything that gives that early clinging action validity in future problem solving is good. For example, should a frame become factored into two or more pieces, those pieces taken as a lump and used crudely should still have some currency as a frame (or as some new noun, but frame is the lingua franca and I vote it is retained). Once programmers gain experience they can go from confident/part-right to confident/accurate. Without a frame concept, they must go from uncertain/unknown to confident/accurate directly. That latter transition is common for gifted thinkers, but the former is easier for the majority. Please retain the concept of a frame, even if it is retained in a somewhat abstract form. A second confusing aspect is that of value-added uses of the layout (kick me if I'm OT). In particular, layout builders, views and controllers are exposed to XPFE programmers very patchily. It is not obvious that every significant XML tag (say) has MVC status, since only <menupopup>, <listbox>, <tree> and <template> expose anything to the DOM/XPCOM of the internals, and only <tree> makes an attempt at exposing a reasonable part of the pipework. No one is saying the current layout engine has shortcomings, it is just that this limited exposure of MVC plumbing poses significant problems to learners. It means that the big, ugly tags (<tree>, <template>) are loaded with learning hurdles separate and in addition to their base conceptual use. Trees are confidence defeating, because they are conflated with MVC concepts not yet seen elsewhere in the XUL widget (or frame) set. No one would advocate padding out the layout engine so that the M,V,C,builder, renderer, etc of each tag is exposed via the DOM/XPCOM. For consistency, however, some uniformity is required if the functionality of the design is to make rational sense. A big ask to you all is to (C) make some rules about this exposure that will apply to ALL DOM/XPCOMed tags, and then reduce away most of the complexity everywhere except where necessary. If that means contructing layout APIs that encourage uniform design for specific XML rendering systems (HTML, XUL), then so be it. The first corollory to this last point is that there should be a demo frame (and suitable tag) that learners can use as a test-bed before they attack the <tree> tag or templates. That frame should be exercisable so that the programmer can control the layout *process* in code, thus revealing the dynamics of the engine, and preparing the way for more complex uses. Such a beginner's sample is common to most development environments, and in Mozilla would go a long way to helping the user understand MVC, renderers and builders. That is big ask (D). You may well point me towards <iframe> or similar, but of course, the layout process is conflated with document management concepts in that case. <iframe> is not a simple learning testbed. The second corollary is that of interface-object relationships. To an XPFE JS programmer, it is not currently clear (or at least not as clear as it might be) what the relationships are between various concepts. It is a quagmire for the learner. On the one hand, there is the theory of MVC. This theory does not map 1-1 (or neatly) to the architectural concepts of builders, renderers or (sigh, [command] controllers). Then there is the matter of underlying, possily XPCOM'ed, objects. Neither theory or architecture map neatly to those objects. Just listen to anyone frothing at the mouth trying to differentiate the content builder from the template or tree builder, or trying to understand why a tree-and-template combination uses the tree builder not the template builder, when a box-and-template combination uses the template builder (from memory, which just shows my point). Again, it is unfair to land this problem strictly on layout, but layout provides the primitives that these higher systems (except for the controllers, sigh) are at least partially built out of. Great as it currently is, "controller" as jargon should be bought back by layout. The Command pattern underlying the controllers in the DOM should provide an alternate property name that better reflects command jargon (eg "commandFacades"). If MVC is to be a useful crutch for learners, then controllers should appear in the layout jargon/implementation. That is big ask (E). Here is a simple example of the cognitive disharmony that can result from mis-matched theory vs implementation. Look at the filesystem directory XPCOM objects, which I hope you are unfamiliar with. To explain this little system, one must explain the theory (providers are registered with queried directory services), and then match it to the technology (@mozilla.org/file/directory_service;1 and friends). All is well, until, dismayingly, this XPCOM object also contains a provider, one that is used directly and not via a service it is to be registerd with. This exception is digestable once pointed out but not easily discoverable. It is also at the expense of the initially touted model, which is now either useless or distractingly special-cased, depending on your point of view. Alas, the layout system combined with the various semi-exposed objects is far more special-cased than the filesystem services. In summary, this note is a request that you appreciate that one measure of a successful re-architecture is the maximum ply of the implied learning path. Laying out a tag like <DIV> or <box> is a one-ply (one step) learning path, well supported by Mozilla. <UL> is a two-ply learning path, because both <UL> and <OL> must be comprehended before virtuosity is achieved. The more complex parts of the layout process: lists, trees, and templates, have very high plys currently, and are obstacles to the gaining of that sense of virtuosity. And yet trees and templates are almost mandatory tech for XPFE folk. No doubt the layout engine should be able to silently and seamlessly solve the display of a given XML document using nothing but content and CSS hints, but in practical terms programmer will interact with it more than that. My biggest ask, therefore, is (E) that XPFE (application, JS) programmers be put abreast of embedded C/C++ programmers. In the long term they will be more numerous and on average less gifted than the embedders, since most software is application software. Perhaps they will never be as numerous as Web developers, but they are more likely to aid this community. Please add their learning needs to your considerations. regards, Nigel.

As far as I can tell, comment 19 is completely off-topic for this bug and now makes it so much harder to read (since it's so tall) that I'm considering refiling this entire bug as a separate bug.

And I don't see how any of the decisions in this bug relate to how users of the XUL toolkit understand it. I think most of comment 19 is about other parts of Mozilla.

This is the best existing bug with keyword=arch for this feedback. My remarks bear directly on architectural issues. Comment 8 shows that some consideration of related matters (XUL) is germaine. I have followed that lead . The re-design of the layout engine might assist in relieving these matters. I'll post comment 19 to n.p.m.layout and n.p.m,xpfe , in case it is too cross-module in scope for here. But it remains the case that changes to layout will likely impact these matters one way or the other. Being advised in advance is only sensible.

More thoughts in my "cleaning up layout" post on mozilla-layout a few months ago: http://groups.google.com/groups?selm=20030917230123.GA4593%40darby.dbaron.org

I've run across this bug more than once. Is there a timetable (granted nothing follows a timetable very well) for implementing all these changes? Is it worth waiting on bugs like bug 217527, hoping this bug takes care of it. Or assume this bug is years away? From what I understand (not to much of the fancyspeak), it looks like a massive undertaking, but will be a great improvement to Gecko.

Using bugzilla for things like this isn't going to work.

Verifying as seeming-unworkable-with-bugzilla. Perhaps some future version of bugzilla will allow read-only access to everyone except a group maintained by the bug reporter.

Another wacky idea (since I still don't have another place for this): Perhaps we could do line breaking at frame construction time -- maintain the invariant that a text frame can never have a break in the middle of it. Doing this would reduce the number of data structures needed. The text frame would still need to store whether it was breakable at the ends, whether the last character(s) should be trimmed when breaking (for whitespace or &shy;), perhaps the width of the trimmed characters, etc. However, it would allow what I described in the post in comment 23 without introducing (multiple) additional levels of data structures.

That sounds like a pretty large expansion in footprint. Given that we already have this complex, but efficient, setup, I'd rather try to make that as rational as possible.