Closed Bug 1303763 Opened 8 years ago Closed 8 years ago

Resolve differences between Vagrant/prod DB schemas

Categories

(Tree Management :: Treeherder, defect, P1)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: emorley, Assigned: emorley)

References

(Depends on 1 open bug)

Details

Attachments

(3 files, 14 obsolete files)

(deleted), text/plain
Details
(deleted), text/plain
Details
(deleted), patch
Details | Diff | Splinter Review
As part of working on bug 1279169, I've discovered there are several differences between the DB schemas used in stage vs prod vs the local vagrant environment. Differences between environments include: * Missing indexes * Duplicate indexes * Different charsets/collations * Different default values / NULL vs NOT NULL * Different character field lengths * Missing/superfluous tables * Missing constraints I suspect the cause of these are a mixture of: * Different server options (eg default collation) * Manual changes run against one environment but not another * Automatic changes (ie: using migrations files) that timed out in one environment but not another * The original stage/prod schema being from the in-repo SQL rather than having been generated by the initial Django migration I've dumped the schema from all environments (including that generated by a fresh `vagrant up`), and have tweaked them slightly to make the diffs cleaner (eg by removing the randomly generated index names / timestamps / auto-increment current value etc). Please can we all have a quick look through these to find individual issues, and file dependant bugs to resolve?
Attached file Schema: Vagrant (obsolete) (deleted) —
Attached file Schema: SCL3 Stage (obsolete) (deleted) —
Attached file Schema: Heroku Prototype (obsolete) (deleted) —
Attached file Schema: Heroku Stage (obsolete) (deleted) —
The Heroku stage schema should match SCL3 prod, since it was not long ago copied from it. However there are already differences (due to default collation). -- I meant to mention - for all environments, I only dumped the following tables (to get a mix of old vs new): * treeherder(_stage)? * stylo * mozilla_inbound(_jobs_1)?
Attached file Schema: SCL3 Prod (obsolete) (deleted) —
Attachment #8792529 - Attachment mime type: text/x-sql → text/plain
Attachment #8792528 - Attachment mime type: text/x-sql → text/plain
Depends on: 1303765
Attached patch Diff: Vagrant -> SCL3 Prod (obsolete) (deleted) — Splinter Review
Attached patch Diff: SCL3 Stage -> SCL3 Prod (obsolete) (deleted) — Splinter Review
Attached patch Diff: Heroku Stage -> SCL3 Prod (obsolete) (deleted) — Splinter Review
Depends on: 1303767
Depends on: 1304047
Depends on: 1304062
Depends on: 1304078
Depends on: 1304083
Depends on: 1304088
Depends on: 1304096
Depends on: 1304327
Depends on: 1304332
No longer depends on: 1304327
Depends on: 1304335
Depends on: 1304338
Depends on: 1304339
Blocks: 1279169
No longer depends on: 1279169
Assignee: nobody → emorley
Depends on: 1320357
Attached file Schema: Vagrant (obsolete) (deleted) —
Attachment #8792528 - Attachment is obsolete: true
Attached file Schema: Heroku Prod (obsolete) (deleted) —
Updated schema dumps. Taken when prod was on 67f83a2, so I reset Vagrant to that revision too. I've only included the main `treeherder` DB this time, since the others are going away soon with the datasource->ORM migration.
Attachment #8792529 - Attachment is obsolete: true
Attachment #8792530 - Attachment is obsolete: true
Attachment #8792534 - Attachment is obsolete: true
Attachment #8792535 - Attachment is obsolete: true
Attached patch Diff: Vagrant -> Heroku Prod (obsolete) (deleted) — Splinter Review
Still quite a lot of differences, including in tables that have only recently been added using the ORM :-( The "let's manually add things that are hard to do via migrations" seems prone to introducing discrepancies.
Attachment #8792543 - Attachment is obsolete: true
Attachment #8792544 - Attachment is obsolete: true
Attachment #8792545 - Attachment is obsolete: true
Narrowing scope to be about Vagrant vs prod - I think we should just reset stage to a recent prod snapshot once the datasource->ORM migration work is complete.
Summary: Resolve differences between Vagrant/stage/prod DB schemas → Resolve differences between Vagrant/prod DB schemas
Blocks: 1302869
Depends on: 1320913
Depends on: 1320914
Depends on: 1320916
Depends on: 1320918
Depends on: 1320920
Depends on: 1320926
Depends on: 1320928
Depends on: 1320930
Depends on: 1320936
Depends on: 1320937
Depends on: 1320946
Depends on: 1320947
Depends on: 1320952
Attached file Schema: Vagrant (obsolete) (deleted) —
Attachment #8814459 - Attachment is obsolete: true
Attached file Schema: Prod (obsolete) (deleted) —
Attachment #8814460 - Attachment is obsolete: true
Attachment #8831915 - Attachment description: Schema: Heroku Prod → Schema: Prod
Attached patch Diff: Vagrant -> Prod (obsolete) (deleted) — Splinter Review
Attachment #8814461 - Attachment is obsolete: true
No longer blocks: 1302869
Depends on: 1302869
Depends on: 1335464
Depends on: 1335975
Attached file Schema: Vagrant (deleted) —
Attachment #8831914 - Attachment is obsolete: true
Attached file Schema: Prod (deleted) —
Attachment #8831915 - Attachment is obsolete: true
Attached patch Diff: Vagrant -> Prod (deleted) — Splinter Review
Updated schema dumps/diff after the fixes in the 29 dep bugs. Aside from the datetime vs datetime(6) differences on the failure_line table (which we've decided to ignore, bug 1304062 comment 6), the schema on prod now matches that in the Vagrant environment. Please let's try to keep it that way! ;-)
Attachment #8831916 - Attachment is obsolete: true
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Depends on: 1343630
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: