Take midi into account for penalty computation