There are two issues in Sentry with around 500 events between them all from a pentest 12 hours ago.

Report 1: https://sentry.prod.mozaws.net/operations/socorro-new-prod/issues/9797494/

RequestError: TransportError(400, 'SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[ubhNykYLS2q_4fbpbnZDGw][socorro202040][0]: RemoteTransportException[[American Eagle][inet[/172.31.43.5:9300]][indices:data/read/search[phase/query]]]; nested: SearchParseException[[socorro202040][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"filtered": {"query": {"match_all": {}}, "filter": {"bool": {"must": [{"bool": {"must": [{"range": {"processed_crash.date...
  File "django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "crashstats/signature/views.py", line 68, in inner
    return view(request, *args, **kwargs)
  File "crashstats/signature/views.py", line 427, in signature_summary
    search_results = api.get(**params)
  File "crashstats/supersearch/models.py", line 200, in get
    return super(SuperSearch, self).get(**kwargs)
  File "crashstats/crashstats/models.py", line 420, in get
    return self._get(expect_json=expect_json, **kwargs)
  File "crashstats/crashstats/models.py", line 467, in _get
    return self.fetch(
  File "crashstats/crashstats/models.py", line 287, in inner
    result = method(*args, **kwargs)
  File "crashstats/crashstats/models.py", line 373, in fetch
    result = implementation_method(**params)
  File "socorro/external/es/supersearch.py", line 406, in get
    results = search.execute()
  File "elasticsearch_dsl/search.py", line 589, in execute
    es.search(
  File "elasticsearch/client/utils.py", line 69, in _wrapped
    return func(*args, params=params, **kwargs)
  File "elasticsearch/client/__init__.py", line 530, in search
    _, data = self.transport.perform_request('GET', _make_path(index,
  File "elasticsearch/transport.py", line 307, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "elasticsearch/connection/http_requests.py", line 78, in perform_request
    self._raise_error(response.status_code, raw_data)
  File "elasticsearch/connection/base.py", line 105, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)

Report 2: https://sentry.prod.mozaws.net/operations/socorro-new-prod/issues/9797554/

RequestError: TransportError(400, 'SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[ubhNykYLS2q_4fbpbnZDGw][socorro202040][0]: RemoteTransportException[[American Eagle][inet[/172.31.43.5:9300]][indices:data/read/search[phase/query]]]; nested: SearchParseException[[socorro202040][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"filtered": {"query": {"match_all": {}}, "filter": {"bool": {"must": [{"bool": {"must": [{"range": {"processed_crash.date...
  File "django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "ratelimit/decorators.py", line 24, in _wrapped
    return fn(request, *args, **kw)
  File "crashstats/supersearch/views.py", line 211, in search_results
    search_results = api.get(**params)
  File "crashstats/supersearch/models.py", line 200, in get
    return super(SuperSearch, self).get(**kwargs)
  File "crashstats/crashstats/models.py", line 420, in get
    return self._get(expect_json=expect_json, **kwargs)
  File "crashstats/crashstats/models.py", line 467, in _get
    return self.fetch(
  File "crashstats/crashstats/models.py", line 287, in inner
    result = method(*args, **kwargs)
  File "crashstats/crashstats/models.py", line 373, in fetch
    result = implementation_method(**params)
  File "socorro/external/es/supersearch.py", line 406, in get
    results = search.execute()
  File "elasticsearch_dsl/search.py", line 589, in execute
    es.search(
  File "elasticsearch/client/utils.py", line 69, in _wrapped
    return func(*args, params=params, **kwargs)
  File "elasticsearch/client/__init__.py", line 530, in search
    _, data = self.transport.perform_request('GET', _make_path(index,
  File "elasticsearch/transport.py", line 307, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "elasticsearch/connection/http_requests.py", line 78, in perform_request
    self._raise_error(response.status_code, raw_data)
  File "elasticsearch/connection/base.py", line 105, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)

I can't see the whole parse error message, so I'm guessing based on what I can see. In the specific events I looked at, the pentest was filtering on jit_category_return_code with values like this:

• akzldka
• wrtqvaxoaelie3zm2

Seems like we should be sanitizing that column value better.

Next step is to reproduce this locally so that we can see the whole error message. That'll tell us more.

The sentry error links are dead. Here's a new one:

https://sentry.prod.mozaws.net/operations/socorro-new-prod/issues/17696207/

Tossing this in my queue to look into soon.

willkg merged PR #6027: "bug 1671636: handle malformed search filter values" in 7702c85.

I deployed this to prod just now in bug #1759160. Marking as FIXED.