Open Bug 1598639 Opened 5 years ago Updated 5 years ago

Investigate: innodb_io_capacity vs PIOPS

Categories

(Tree Management :: Treeherder: Infrastructure, task, P3)

Tracking

(Not tracked)

People

(Reporter: ekyle, Unassigned)

References

(Blocks 1 open bug)

Details

In an effort to solve RDS slowness IOPS was switched to PIOPS 7000, this may be much higher than we need.

https://github.com/mozilla-platform-ops/devservices-aws/pull/113/files#diff-aac6aeeaf82f5228af02042b7e7825d2R149

Plus, innodb_io_capacity, innodb_io_capacity_max also increased.

https://github.com/mozilla-platform-ops/devservices-aws/commit/69ab6c7d210e845ee735347cfdcbb42e0f86f5ba#diff-aac6aeeaf82f5228af02042b7e7825d2R68

It would be nice to know how these numbers relate; they probably do not measure the same thing.

  1. Set innodb_io_capacity_max to innodb_io_capacity; reduce innodb_io_capacity to the point we see IO being constrained. As a side effect, our transactions should be getting longer
  2. Adjust PIOPS downward until we see its constraining effect on IO.

This should provide us with a rough ratio, R, we can use to set PIOPS in production: Set PIOPS >= R * innodb_io_capacity_max so we have spare capacity for other processes on the machine (like the ability to login).

Blocks: 1599095
No longer depends on: 1597136
Priority: -- → P3
You need to log in before you can comment on or make changes to this bug.