Max Kanat-Alexander Assignee
	Description • 16 years ago

reed reported that on the staging site for upgrading bugzilla.mozilla.org, the bugs_fulltext populating code was running for hours and hours and was only 14% done. I think there's something wrong with their staging server for it to be *that* slow, but we could special-case MySQL and use GROUP_CONCAT to do this all server-side.

	Max Kanat-Alexander Assignee
	Comment 1 • 16 years ago

This is somewhat of a hack, but it works.

	David Lawrence [:dkl]
	Comment 2 • 16 years ago

We encountered an issue today with this patch on our specific installation. When migrating a 2.18 database to 3.2 using checksetup.pl the patch fails because when it runs, the longdescs.comment_id columns does not yet exist. If I add the column manually and then run then it works better.

	Max Kanat-Alexander Assignee
	Comment 3 • 16 years ago

Okay. That could be fixed. Does this get you a noticeable performance improvement?

	David Lawrence [:dkl]
	Comment 4 • 16 years ago

yes, about 1/10 of the time required by the old method :)

	David Lawrence [:dkl]
	Comment 5 • 16 years ago

(In reply to comment #4)
> yes, about 1/10 of the time required by the old method :)
> 

I mean just for the fulltext conversion part. The rest still takes a while (dropping/adding indexes, email prefs, moving attachment data, etc.)

	Max Kanat-Alexander Assignee
	Comment 6 • 16 years ago

Okay! That's enough of a benefit for me!

We can create a sql_group_concat in specific DB drivers that can support it, and then call $dbh->can('sql_group_concat') when we need it.

	Max Kanat-Alexander Assignee
	Comment 7 • 16 years ago

This version can actually go upstream. It does still contain one hack for MySQL, but that's OK for now. I'm OK with hacks for clear performance gains, and this is the only place we use GROUP_CONCAT right now.

	David Lawrence [:dkl]
	Comment 8 • 16 years ago

Running now on our internal snapshot database. Will let you know how it goes. Code wise looks good to me.

	David Lawrence [:dkl]
	Comment 9 • 16 years ago

Comment on attachment 331863 [details] [diff] [review]
v2 (non-hack)

Works good. Converted 283155 bugs in 19mins.

r=dkl

	Max Kanat-Alexander Assignee
	Comment 10 • 16 years ago

tip:

Checking in Bugzilla/DB/Mysql.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/DB/Mysql.pm,v  <--  Mysql.pm
new revision: 1.65; previous revision: 1.64
done
Checking in Bugzilla/Install/DB.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Install/DB.pm,v  <--  DB.pm
new revision: 1.54; previous revision: 1.53
done

3.2:

Checking in Bugzilla/DB/Mysql.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/DB/Mysql.pm,v  <--  Mysql.pm
new revision: 1.60.2.3; previous revision: 1.60.2.2
done
Checking in Bugzilla/Install/DB.pm;
/cvsroot/mozilla/webtools/bugzilla/Bugzilla/Install/DB.pm,v  <--  DB.pm
new revision: 1.51.2.1; previous revision: 1.51
done