Closed Bug 721572 Opened 13 years ago Closed 13 years ago

Abstract our file storage

Categories

(addons.mozilla.org Graveyard :: Code Quality, defect, P1)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: clouserw, Assigned: ashort)

References

Details

(Whiteboard: [fat])

Right now our files are all stored on an NFS server and are accessed by concatenating a bunch of file paths and just opening the files. As we open more data centers around the world that's not going to scale so we should be abstracting that file access which will give us the ability to replace NFS with something else (likely S3). Fortunately, Django has a file abstraction layer built in (https://docs.djangoproject.com/en/dev/ref/files/storage/). This bug is to convert our current system to use the abstraction layer. This bug is not about using S3 - we'll continue using NFS, just doing it through the abstraction layer. Starting in one spot (add-on icons? user profile pics?) and doing this in stages is fine, but the end goal is to replace all file access (aside from temporary file stuff) with this abstraction layer. Get reviews early and ask questions if things come up!
Depends on: 724659
Target Milestone: 6.4.0 → 6.4.1
This is partially done. Will do add-ons next week.
Target Milestone: 6.4.1 → 6.4.2
Target Milestone: 6.4.2 → 6.4.3
Target Milestone: 6.4.3 → 6.4.4
Target Milestone: 6.4.4 → 6.4.5
Target Milestone: 6.4.5 → 6.4.6
Target Milestone: 6.4.6 → 6.4.7
Target Milestone: 6.4.7 → 6.4.9
Target Milestone: 6.4.9 → 6.5.2
Here's where I left off: - The utility to use abstract storage everywhere in the site landed and is ready to use: https://github.com/mozilla/zamboni/commit/ed22b221 The next step is to apply it everywhere (or in steps). I started that work here: https://github.com/kumar303/zamboni/compare/master...all-file-storage that branch has some merge conflicts now because it was before the mkt fork. Shouldn't be too hard to fix though. This is a nose plugin I used to find the parts of the code to fix (using failing tests) https://github.com/kumar303/zamboni/commit/645ff782555b6a85d328ce4d2b865afc7132dcce It also has some shell commands in the comments I used to find code that needed changing.
Let's talk about this for next week. IT is getting worried.
Target Milestone: 6.5.2 → 6.5.1
Assignee: kumar.mcmillan → ashort
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Blocks: 749338
Product: addons.mozilla.org → addons.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.