How to use `DirectLinearSolver`

[bmxam]

Hi,

I'm trying to use the DirectLinearSolver instead of the default IterativeLinearSolver but I must be missing something because the below MWE, adapted from the doc, fails. How can I fix this examples?

using ForwardDiff
using ImplicitDifferentiation
using LinearAlgebra
using Optim

function forward_optim(x)
    f(y) = sum(abs2, y .^ 2 .- x)
    y0 = ones(eltype(x), size(x))
    result = optimize(f, y0)
    y = Optim.minimizer(result)
    z = nothing
    return y, z
end

function conditions_optim(x, y, _z)
    ∇₂f = 4 .* (y .^ 2 .- x) .* y
    return ∇₂f
end

x = [4.0, 9.0]

implicit_iterative = ImplicitFunction(
    forward_optim,
    conditions_optim;
    linear_solver = IterativeLinearSolver(),
)
@show ForwardDiff.jacobian(first ∘ implicit_iterative, x) # OK

implicit_direct =
    ImplicitFunction(forward_optim, conditions_optim; linear_solver = DirectLinearSolver())
@show ForwardDiff.jacobian(first ∘ implicit_direct, x) # KO
# ERROR: LoadError: MethodError: no method matching (::ImplicitDifferentiation.DirectLinearSolver)(::ImplicitDifferentiation.JVP{…}, ::Nothing, ::Vector{…}, ::Vector{…})

[gdalle]

Hi, thanks for opening the issue!
The direct linear solver relies on matrix factorization, and thus it only works when the linear map is represented explicitly as a matrix. Here's the change you need:

implicit_direct =
    ImplicitFunction(forward_optim, conditions_optim;
    representation = MatrixRepresentation(),
    linear_solver = DirectLinearSolver()
)

I agree that this needs to be documented, would you like to submit a PR since you detected the inconsistency? Otherwise I can do it

[bmxam]

Thanks Guillaume, I'll give a try. I'm ok to submit a PR, but feel free to do it if I'm not fast enough ^^. I'll probably do it on monday.

[bmxam]

There are actually several places where this could be documented:

• in the FAQ (something like "select your linear solver"
• in the API reference (under DirectLinearSolver)
• in the advanced use cases, but I don't think that's the best option because this part is more "application focused"
  What do you prefer?

[gdalle]

The first two (both of them) seem good?

[bmxam]

I think this issue can now be closed (with #202), but I'd prefer that a maintainer does it.

[gdalle]

Yes indeed.
Note that you can automate the closing of an issue by adding text like "Fixes #1234" to your PR description.

[gdalle]

Thanks for the contribution!