Ed Morley [:emorley] Assignee
	Description • 11 years ago

mozharness adds a log prefix of the form:

{
21:26:35     INFO - <normal line>
}

ie datetime then log level, where log level can be {DEBUG, INFO, WARNING, ERROR, CRITICAL, FATAL}.

For cases where TBPL falls back to searching bugzilla for the full failure line from the log, this prefix means we don't match the intermittent failure bugs, since the datetime varies. In addition, some of the error regexes contain start of line anchors (eg: preg_match("/^BaseException:/", $line)), which now won't match (and in many cases, removing the start anchor would cause false positives).

We should:
a) Remove the prefix before matching against the regexes that decide if a line is a failure.
b) Remove the prefix when making the full-line bzapi search.
c) Remove the prefix when displaying the annotated summary panel (to reduce the amount of line wrapping).

However we still want the prefix in the brief/full log views.

	Ed Morley [:emorley] Assignee
	Comment 1 • 11 years ago

Make it clearer that we are dealing with line numbers, not the line contents themselves.

	Ed Morley [:emorley] Assignee
	Comment 2 • 11 years ago

We're only iterating over known failure lines (ie matched lines) as it is - so this matchLine() call is redundant, since it will return true for all lines.

	Ed Morley [:emorley] Assignee
	Comment 3 • 11 years ago

This removes the prefix from the log excerpt (the summary of log failures), which
is used to generate the bug suggestions and is shown in the annotated summary
panel / in the top summary section of the brief log view. The main log content
is unaffected.

	Ed Morley [:emorley] Assignee
	Comment 4 • 11 years ago

(Since it's a common case, and we're about to increase complexity in the following parts).

	Ed Morley [:emorley] Assignee
	Comment 5 • 11 years ago

This solves comment 0 part [a].

	Ed Morley [:emorley] Assignee
	Comment 6 • 11 years ago

Match against mozharness error lines earlier, since they're a common case (and will only become more common as more things move over to using mozharness). Saves us the preg_replace() call.

	Ed Morley [:emorley] Assignee
	Comment 7 • 11 years ago

After testing locally, there are now a bunch of search terms that return 20+ unhelpful bugs in the annotated summary. In the future we may switch to using regex here (and cover all the exit codes in one go), but for now this gets rid of the worst.

	Ed Morley [:emorley] Assignee
	Comment 8 • 11 years ago

Until now, there were no full-line-fallback search terms that would return unhelpful
results. However, with the removal of the mozharness log prefix that is no
longer the case. We now sanity check the full line ($trimmedLine). This means
that $searchTerm can now end up not being defined, so we're more now cautious
before searching too.

This patch is needed to make the newly added 'TypeError: content is null'
blacklist entry actually work.

	Ed Morley [:emorley] Assignee
	Comment 9 • 11 years ago

This patch is needed to make the newly added 'TypeError: content is null'
blacklist entry actually work, since some of the Javascript type errors have a
bunch of preceeding whitespace.

	Ed Morley [:emorley] Assignee
	Comment 10 • 11 years ago

This log line refers to the search term mentioned prior to it, so should be
nested under it.

ie log output will now be:
{
* Search term: "test_cpows.xul"
  Removing cached bug suggestions
  BzAPI query: 1268ms
  Bugs returned: 1
}

	Ed Morley [:emorley] Assignee
	Comment 11 • 11 years ago

Thank you for the reviews :-)

https://hg.mozilla.org/webtools/tbpl/rev/befcc9cf9182
https://hg.mozilla.org/webtools/tbpl/rev/f4f5f13b93c3
https://hg.mozilla.org/webtools/tbpl/rev/d5897e55e093
https://hg.mozilla.org/webtools/tbpl/rev/ba77c54d74a8
https://hg.mozilla.org/webtools/tbpl/rev/0d46be1d463a
https://hg.mozilla.org/webtools/tbpl/rev/0ba0a2f33d5e
https://hg.mozilla.org/webtools/tbpl/rev/543e945cd1d5
https://hg.mozilla.org/webtools/tbpl/rev/505b7bf5c2c9
https://hg.mozilla.org/webtools/tbpl/rev/fc63892a419c
https://hg.mozilla.org/webtools/tbpl/rev/bf28eeeea8f3

	Ed Morley [:emorley] Assignee
	Comment 12 • 11 years ago

Some of the bugs blocking this are fixed, others not. However can't really wait any longer since the switch to mozharness for Talos means there are now many many more intermittent failures that have stopped generating bug suggestions due to the mozharness log line prefix.

As such, I've just pushed this out, and we'll have to deal with the annotated summary spam in the meantime.

\o/

	Ed Morley [:emorley] Assignee
	Comment 13 • 11 years ago

Dependant bugs moved to bug 910614.