Inasmuch as it's relevant to this context, a measure is the standard, system, or unit by which something is measured, e.g.,
Philosophers have long pondered the true measure of a man.
The standard measure of distance is the meter.
or an unstated or undefined amount, e.g.,
This recipe calls for a measure of egg for texture.
She failed to show any measure of sympathy.
A measurement is the datum obtained by measuring, e.g.,
His measurements are off by an order of magnitude.
Either word can serve as the act of measuring.
His measure(ment) of the frequency failed to account for the red-shift in the observations.
For your example,
The measure of this algorithm's performance is its execution time. Therefore, our testing suite gathers measurements of the execution under different runtime conditions. Our testing suite itself adds overhead that will not be present in a deployment environment, however, and this must not be forgotten when we analyze its measure(s/ment/ments) of the underlying algorithm's performance.