Open
Bug 1331399
Opened 8 years ago
Updated 4 years ago
The way we store job information on the front end is inefficient, wasting 10s of MB of memory (or more)
Categories
(Tree Management :: Treeherder: Frontend, defect, P1)
Tree Management
Treeherder: Frontend
Tracking
(Not tracked)
NEW
People
(Reporter: wlach, Unassigned)
References
(Blocks 1 open bug)
Details
Attachments
(2 files)
If you do a heap profile of treeherder in Chrome, you'll notice that a very large chunk of the data stored in Treeherder (especially on a branch with a lot of jobs like mozilla-inbound) in something called ThJobModel -- upwards of 64M in my most my recent testing. This is basically a literal translation of what we get from the jobs model, data structures like this:
{
"build_architecture": "armv7",
"build_os": "android",
"build_platform": "android-4-0-armv7-api15",
"build_platform_id": 153,
"build_system_type": "buildbot",
"end_timestamp": 0,
"failure_classification_id": 1,
"id": 69320706,
"job_coalesced_to_guid": null,
"job_group_description": "",
"job_group_id": 2,
"job_group_name": "unknown",
"job_group_symbol": "?",
"job_guid": "13669171854e32b8db0f5c8d8c38b4e73848c882",
"job_type_description": "Build from sources",
"job_type_id": 2,
"job_type_name": "Build",
"job_type_symbol": "B",
"last_modified": "2017-01-16T16:01:06.712719",
"machine_name": "unknown",
"machine_platform_architecture": "armv7",
"machine_platform_os": "android",
"option_collection_hash": "102210fe594ee9b33d82058545b1ed14f4c8206e",
"platform": "android-4-0-armv7-api15",
"platform_option": "opt",
"push_id": 160580,
"reason": "unknown",
"ref_data_name": "Android armv7 API 15+ try build",
"result": "unknown",
"result_set_id": 160580,
"running_eta": 723,
"signature": "b8d8be8202eb1c0461f60a309615dbea280889c4",
"start_timestamp": 1484582367,
"state": "running",
"submit_timestamp": 1484582364,
"tier": 1,
"who": "unknown"
}
We could save a considerable amount of memory by factoring out some of the elements that are common between jobs into some kind of immutable data structure and then referencing that from the job objects themselves. E.g. in the above example, we could have a "build platform" object that looks like this and then just assign it to the buildPlatform:
{
"architecture": "armv7",
"os": "android",
"platform": "android-4-0-armv7-api15",
"id": 153
}
The same would be true for "machine_platform", "job group" and "job type"
To solve this bug, we'd probably want to create some kind of angular service/singleton to store the various permutations of machine/build/job type/job group information and then have a query function to fetch existing instances of this data (or store a new one) based on what we find in the jobs. Then when loading the jobs, we'd replace (remove) the job instance's string properties with direct references to this data structure.
Updated•7 years ago
|
Updated•7 years ago
|
Component: Treeherder → Treeherder: Frontend
Comment 1•5 years ago
|
||
This bug may be ancient and from the Angular days, but it's still relevant. I think this would help us quite a bit.
Updated•5 years ago
|
Priority: P2 → P1
Comment 2•5 years ago
|
||
Comment 3•5 years ago
|
||
Comment 4•4 years ago
|
||
De-prioritizing Treeherder work in new reorg. So removing myself.
Assignee: cdawson → nobody
You need to log in
before you can comment on or make changes to this bug.
Description
•