[Utilities] allow penalty relaxation for ScalarNonlinear constraints

[Robbybp]

I'd like to use JuMP.relax_with_penalty! on a model with nonlinear constraints. If this implementation looks reasonable, I can clean this up and add tests. Here's an example:

using JuMP
model = Model()
@variable(model, x[1:3] >= 0)
@objective(model, Min, sum(x .^ 2))
@constraint(model, eq1, x[1] * x[2]^1.5 == x[3])
@constraint(model, ineq1, x[1] + x[2] + x[3] >= 4)
@constraint(model, ineq2, x[1]*x[2]*x[3] >= 1)
@constraint(model, ineq3, x[1] * exp(x[2]) <= 20)
@constraint(model, ineq4, 0 <= log(x[2] * x[3]) <= 10)
relax_with_penalty!(model)
println(model)

┌ Warning: Skipping PenaltyRelaxation for ConstraintIndex{MathOptInterface.VariableIndex,MathOptInterface.GreaterThan{Float64}}
└ @ MathOptInterface.Utilities ~/.julia/dev/MathOptInterface/src/Utilities/penalty_relaxation.jl:367
Min x[1]² + x[2]² + x[3]² + _[4] + _[5] + _[6] + _[7] + _[8] + _[9] + _[10]
Subject to
 eq1 : ((x[1] * (x[2] ^ 1.5)) - x[3]) + (_[4] - _[5]) = 0
 ineq2 : (((x[1]*x[2]) * x[3]) - 1.0) + _[6] ≥ 0
 ineq3 : ((x[1] * exp(x[2])) - 20.0) - _[7] ≤ 0
 ineq4 : log(x[2]*x[3]) + (_[8] - _[9]) ∈ [0, 10]
 ineq1 : x[1] + x[2] + x[3] + _[10] ≥ 4
 x[1] ≥ 0
 x[2] ≥ 0
 x[3] ≥ 0
 _[4] ≥ 0
 _[5] ≥ 0
 _[6] ≥ 0
 _[7] ≥ 0
 _[8] ≥ 0
 _[9] ≥ 0
 _[10] ≥ 0

Limitations

• Doesn't support ScalarNonlinearFunction objectives. We get a confusing warning in this case.
• Doesn't support VariableIndex objectives. We probably get the same confusing warning here.

I think both of these can be supported without too much effort.

[odow]

Nice nice nice. I can take over and add tests, etc.

[odow]

So one issue: we currently modify each constraint in sequence, and for every constraint, we modify the objective. It might be better to cache the penalties and modify the objective only at the end.

[odow]

Doesn't support ScalarNonlinearFunction objectives. We get a confusing warning in this case.
Doesn't support VariableIndex objectives. We probably get the same confusing warning here.

Done

[odow]

Okay, I've now done a bunch of things:

• Add support for ScalarNonlinearFunction constraints and objective
• Add support for VariableIndex objective function
• Make JuMP.relax_with_penalty! significantly faster because we now modify the objective once instead of per constraint

Before

julia> function main(N)
           model = Model()
           @variable(model, x[1:N] >= 0)
           @objective(model, Min, sum(x))
           @constraint(model, [i in 1:N], x[i] == i)
           @time relax_with_penalty!(model)
       end
main (generic function with 1 method)

julia> main(1_000);
┌ Warning: Skipping PenaltyRelaxation for ConstraintIndex{MathOptInterface.VariableIndex,MathOptInterface.GreaterThan{Float64}}
└ @ MathOptInterface.Utilities ~/.julia/packages/MathOptInterface/tVdNJ/src/Utilities/penalty_relaxation.jl:296
  0.005167 seconds (26.51 k allocations: 1.539 MiB)

julia> main(100_000);
┌ Warning: Skipping PenaltyRelaxation for ConstraintIndex{MathOptInterface.VariableIndex,MathOptInterface.GreaterThan{Float64}}
└ @ MathOptInterface.Utilities ~/.julia/packages/MathOptInterface/tVdNJ/src/Utilities/penalty_relaxation.jl:296
 10.636809 seconds (2.60 M allocations: 165.800 MiB, 0.82% gc time)

After

julia> main(1_000);
┌ Warning: Skipping PenaltyRelaxation for ConstraintIndex{MathOptInterface.VariableIndex,MathOptInterface.GreaterThan{Float64}}
└ @ MathOptInterface.Utilities ~/git/jump-dev/MathOptInterface/src/Utilities/penalty_relaxation.jl:380
  0.003695 seconds (21.54 k allocations: 1.515 MiB)

julia> main(100_000);
┌ Warning: Skipping PenaltyRelaxation for ConstraintIndex{MathOptInterface.VariableIndex,MathOptInterface.GreaterThan{Float64}}
└ @ MathOptInterface.Utilities ~/git/jump-dev/MathOptInterface/src/Utilities/penalty_relaxation.jl:380
  0.309691 seconds (2.10 M allocations: 155.829 MiB)

This should also make MathOptIIS much faster as well.