In autophone.py when a new job is dispatched, the revision hash is looked up from Treeherder and is ultimately stored in the jobs.sqlite database. If the treeherder_url is not specified, the revision_hash is set to None. When a build is downloaded, the revision_hash is looked up from Treeherder and stored in the build's metadata.json file. Again, if treeherder_url was not specified, the revision_hash is set to None. If you run a test with no treeherder_url set, the job and the build's metadata.json will both have revision_hash None. If you later run a test with the same build but now with the treeherder_url set, autophone.py's new_job will look up and find the revision_hash and store it with the new job and will submit the pending job notification to Treeherder. Later when worker.py get's the job from the database and the build from the build cache, it will use the value of revision_hash from the build's metadata.json file which will be the cached value of None. The result is that the submit_running and submit_completed notifications for the job will not be sent to Treeherder. Originally the metadata.json's revision_has was added first, followed by the revision_hash as stored in the jobs database as part of the patch for bug 1153992. The failure was in keeping them in sync. A solution would be to stop using the metadata.json value of revision_hash and to always use the value from the job which was the value found by autophone.py's new_job originally. This is only an issue during testing and can easily be worked around by clearing the builds cache directory when switching from not reporting to Treeherder to reporting to Treeherder.

Treeherder no longer uses revision_hash, so this should no longer occur.