We have a journaling system that keeps track of application errors and some other things and gives us a good way of diagnosing and keeping track of problems in the application-layer. However, we don't want to or need to keep that data forever. Further, that data could be talking about other data we purge when it's older than 6 months. For those reasons, we should add some code to purge journal data that's older than 6 months.

Changing this to a P1. It should get done asap.

Weird. I thought I added a comment about changing the journal. I think instead of purging all the data that's older than 6 months, we instead should change the journal record so that each record has an expiration date. Data we're tossing in the journal is all different and some can expire sooner than other bits. Thus the new system would require the logger to specify the expiration for the entry. Then we create a purge management command that purges everything that's expired. I think that's a couple of days of work, so tentatively marking this as 2 points.

I still want to do this, but can't this quarter. Pushing it off.

Yeah, the data in the journal is all different, but I'm thinking the journal shouldn't be holding any data that's not ephemeral. e.g. if we lose it all, we should be fine. Given that, I'm going to go with the much-easier-to-implement "delete everything older than 6 months" method. Grabbing this now since the table is getting pretty big and slow.

Making this 1 point. It involves the following work: 1. create a maangement command that deletes everything older than 6 months and prints to stdout how much it deleted 2. write a test to make sure the management command is working properly 3. add a cronjob to run the management command every sunday

I'm tweaking item 1 to generalize it into a big project-wide "purge_data" command which will touch all the apps that have a purge function registered.

In a PR: https://github.com/mozilla/fjord/pull/647

Landed in master in https://github.com/mozilla/fjord/commit/912dcde198416f33425281542c0216b0a9b53460 I'll push it out tomorrow.

Pushing this to prod just now. I'll make sure to check it on Monday. Pretty sure it'll be fine, so I'm going to optimistically mark it FIXED now.