Closed Bug 1314957 Opened 8 years ago Closed 8 years ago

Spark clusters should use centralized metastore

Categories

(Cloud Services Graveyard :: Metrics: Pipeline, defect, P1)

Product:
Cloud Services Graveyard
Component:
Metrics: Pipeline
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: rvitillo, Assigned: whd)

References

Details

(Whiteboard: [SvcOps])

User Story

Now that we have a centralized Hive metastore we should configure our Spark clusters to use it so that both Spark & Presto see the same tables.
No description provided.
hey Blake, I think you are on this, please add effort.
Assignee: nobody → bimsland
Priority: -- → P1
Points: --- → 2
Priority: P1 → P2
Blocks: 1269781
Whiteboard: [SvcOps]
Assignee: bimsland → whd
Priority: P2 → P1
https://github.com/mozilla/emr-bootstrap-spark/pull/70 https://github.com/mozilla/emr-bootstrap-spark/pull/72
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Well, this doesn't appear to work with EMR < 5 (spark 2.0), so we're either going to have to migrate everything to later EMR releases or use different configurations to make this happen.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Depends on: 1344020
Reapplied now that EMR 4 is dead: https://github.com/mozilla/emr-bootstrap-spark/pull/75
Status: REOPENED → RESOLVED
Closed: 8 years ago → 8 years ago
Resolution: --- → FIXED
Product: Cloud Services → Cloud Services Graveyard
You need to log in before you can comment on or make changes to this bug.