I've been playing with the directory viewer, and trying to work out why it is so slow. dougt mentioned to me on IRC that he has a hack in his branch to replace the dir listing with html output, and that it was way faster. Note that I've thought about it a bit more, and I think my description on IRC this afternoon wasn't totally correct (mainly because I forgot to shift-reload the output file between runs) First, I'll attach some profiling data I got with jprof by: - setting JPROF_FLAGS="JP_DEFER JP_START=2 JP_PERIOD=0.010" - starting the browser - typing gopher://vm1.mcgill.ca in the urlbar - sending the start signal - loading gopher://vm1.mcgill.ca - sending the stop signal - quiting and running jprof I did this twice - once with the sortActive="true" sortDirection="ascending" line in directory.xul present, and once with it deleted. These results appears to be repeatable, +/- a bit of noise. I can't make any sense of the "flat" data output. vm1.mcgill.ca is literally down the road from me, and I'm on an ADSL connection, so network latency shouldn't matter. I didn't use localhost because of bug 68199 (ftp://localhost doesn't work). That wasn't fixed on Saturday by dougt's checkin, but he did mention to me on IRC that localhost is really slow (after I send him a debug log - see the bug), and that that would probably be fixed by his branch. I didn't test that on the weekend, but loading 500 entries over localhost is currently slow, even using TestPrototols, so its not the dirviewer. The following are some semi-random thoughts: The 2 second delay allows me to put the moz window back into focus before profiling starts. When stopping profiling, I just move the mouse into the xterm I used to start the profiling, and type in the cmd without bringing the xterm window over the moz window. This is important - I mispelled JP_START once, and got totally different results, presumably because the xterm was over mozilla, and then I brought moz to the front, causing a whole lot of stuff to happen. I used gopher initially, because there's no connection state information like ftp to mess with the reults. (Bug 49334 has gopher patches). See below for why I think ftp isn't seeing a speedup with this removed. Note that file is fast, and unaffected by this, probably because all the rdf data comes at once. OTOH, the profiles include the time taken to actually load the viewer. I don't know if this makes a difference. Firstly - this is a debug build with no optimisation flags. I've also got patches to the dir viewer (so that gopher works), but waterson reviewed them reasonably thoroughly. I don't think I can revert this without breaking gopher (and so then I can't reproduce this) although I'll see if I can break it enough so that loading a url (wihtout following anything) still works. Has anyone else done any profiling of the dir viewer? I'd like some comments on the stuff below before I try this. There are about 25 entries in this particular gopher directory. nsHTTPIndex::FireTimer is called way too many times - 243 - in the sorted case. The timer "only" fires 65 times with no sorting. Could there be some sort of recursion going on with the sortservice's binary search causing an assert on the httpindex, which causes a timer, which causes the assert, etc? Alternatly, the way the timer refiring occurs may mean that if for some reason we're slow, the timer fires more often, so its a symptom not a cause. Earlier this evening, the gopher dir listing loaded in about 16 secs (wall clock time, rough estimate) with sorting, 6 without. Clicking on the column header to cause a sort (although its descending rather than acending - shouldn't make a difference) takes 2 seconds, including painting black over the content area, and then redisplaying everything. So there are 8 seconds going somewhere. The sorted version loading in three very distinct stages, with new data being inserted into the proper places (and the side grippy stuff seems to vanish in between updates, but thats another bug, and it happens with ftp sometimes), and thats what pointed me to this. (Thats earlier today - now it loads in two. Net conjestion?) ftp doesn't appear to have such a large difference (it appears to be about 0.5-1 second faster, but I think thats just noise - i'm using a wristwatch for timing). My _guesses_ for this are: - ftp dirs are initially sorted, as received from the server (is this always correct? If so we can just set the initial sort order to natural, or whatever its called). Gopher dirs aren't (and its a very minor bug in certain cases that we do sorting at all) So when we get some more data, its always at the end of what we have, and all the entries are adjacent to each other, rather than being inserted between existing entries. Is there some sort of XUL template/tree optimisation somewhere for either adjacent entries being added at once, or entries being added at the end, that ftp/no-sort-gopher triggers? The adding-at-the-end thing may be explained by the fact that we don't have to repaint everything below, but it doesn't explain why this is still the case if there are other directories below one which is being expanded (ie if its the bottom of the subtree, not of the tree) - inserting into the middle of the tree individually causes stuff to happen multiple times, instead of once. - the fireTimer stuff from above is triggering less often when adding to the end for some reason. I haven't profiled ftp at all to check this, and can't make enough sense of the profiled data I have to make it worthwhile for me to do this. Does anything else in mozilla dynamically add lots of data to an rdf tree in chunks/on a timer, so code can be compared? The above is largly guesswork, combined with playing arround with the dirviewer for the last couple of months. Its 1:15am, so I'm sure I left stuff out. Can someone else see something useful in the profiled data?

Adding perf keyword. (Note that each of those files are 400-500K)

Is this just going to be faster without any extra effort once <outliner> lands?

If you add gopher://vm1.mcgill.ca/ to the bookmarks menu, then manage bookmarks shows the same problem - sorted is slow, unsorted isn't. The bookmarks allows tristate ordering - maybe we should just do that?

I think that part of the deal here is that nsHTTPIndex::fireTimer's existence is really just a workaround for the fact that we don't have good prioiritization in the event system (bug 50104). If the event system could prioritize things reasonably, we should be able to assert this stuff into RDF as it arrives without having to resort to the timer hack. Dougt/danm, am I remembering correctly?

reassigning to xpapps to convert to outliner

any idea when we would see something like a outliner replacement of the tree widget? Post mozilla 0.91?

*** Bug 73634 has been marked as a duplicate of this bug. ***

Adding bug 73509, convert directory viewer to use rdf outliner.

nav triage team: Not a beta stopper, marking nsbeta1- and future

bryner, should this be yours, since you're rewriting the file picker to use the outliner?

no it shouldn't, what I'm working on only affects the filepicker, not the directory viewer.

over to bbaetz

My patch in bug 78148 will mean that all protocols can use the html-based directory listing. I may convert this to outliner at some point.

*** Bug 73509 has been marked as a duplicate of this bug. ***

Nominating 0.9.2 (from the selfish Netscape perspective of "we released NS6 with a XUL-based directory viewer and it'd look like a step backwards to let the next release out the door without one..."). Feel free to ignore ;) Also, cleaning up summary, platform, os (I hope you don't mind, bbaetz).

/me shrugs Feel free to take this. I doubt I'm going to get to it soon. Anyone who does, do look at my patch in bug 78148. It _seems_ faster with that patch, but it could just be my imagination.

bryner: do you have any time for this? if not, i guess punt it back to bbaetz...

I don't think I can get to this very soon either. But someone has to own this bug. -> Future.

As a user, I'm quite happy with the HTML directory listings in the current trunk builds. The XUL listings were nifty, but I'll take instant page load over nifty every time. HTML directory listings also have the advantage of visited-link highlighting; that wasn't in the XUL version I had, and it was irritating to be without it in some contexts.

*** Bug 83926 has been marked as a duplicate of this bug. ***

-> bbaetz

Dirviewer triage. I'm going to start with rdf-outliner, and then see if I need to do better later.

outliner -> 0.9.7

bradley, are you working on this ?

I haven't started on it yet, but I plan to do this (and all the xpapps bugs assigned to me for 0.9.7) this milestone. I've had a lot of school sowkr, and no time, recently, unfortunately.

According to posts on npm.xpfe, this will Just Happen sometime soon. I may end up using the new outliner features, but given time constraints: ->0.9.9

OK, moving all XUL dirviewer bugs to post 1.0. I don't want to be doing this, but there are only so many hours in the day. These will be taken earlier if someone submits a patch, or I find time.

So nobody is doing this currently? If I have some time next week, I'll try to make something. (I've done already very simple outliner for file:// -stuff only.)

Nope, noone is doing this currently. I'm away from basically now until the end of Jan, so it won;t be done by me before then. I believe that the <outliner> tag will be renamed to <tree> somethime in the 0.9.8 milestone, though.

I'm doing this currently. Also wondering if there should be some kind of sidebar tab for files.

Unfortunately this is taking some time.

Smaug, I suggest to discuss your approach.

I just can't get the moz-icons to work fast enough. Maybe there is some images/icons-related thing in outliner that I do not know and that I cannot find. Should we make first an outliner without the moz-icon-stuff? Maybe someone of the makers of the outliner should fix this bug? (Sorry, I have to give up.)

taking

ok, a good candidate for WONTFIX, since we're going to have html view for directory viewer

Well, the idea was to have both, as some pepole do prefer the xul interface. Future, rather than wontfix, probably, or do the minimal requried for <tree> to vanish. I'm not goign to have time for this, unfortunately.

future then

nsbeta1- per Nav triage team

*** Bug 130515 has been marked as a duplicate of this bug. ***

*** Bug 133596 has been marked as a duplicate of this bug. ***

me

fixed by 110156 tree conversion work

vrfy'd fixed using 2002.04.09 comm bits on linux rh7.2, win2k and mac 10.1.3 --tested with a local file listing (file:///).