Complex entries need to be conjugated after cost_and_gradient

[fliingelephant]

Cases

Failed to pass these two tests if change Float64 to ComplexF64.

TensorInference.jl/test/belief.jl

Lines 46 to 79 in d914583

	@testset "belief propagation" begin
	n = 5
	chi = 3
	mps_uai = TensorInference.random_tensor_train_uai(Float64, n, chi)
	bp = BeliefPropgation(mps_uai)
	@test TensorInference.initial_state(bp) isa TensorInference.BPState
	state, info = belief_propagate(bp)
	@test info.converged
	@test info.iterations < 20
	mars = marginals(state)
	tnet = TensorNetworkModel(mps_uai)
	mars_tnet = marginals(tnet)
	for v in 1:TensorInference.num_variables(bp)
	@test mars[[v]] ≈ mars_tnet[[v]] atol=1e-6
	end
	end
	@testset "belief propagation on circle" begin
	n = 10
	chi = 3
	mps_uai = TensorInference.random_tensor_train_uai(Float64, n, chi; periodic=true)
	bp = BeliefPropgation(mps_uai)
	@test TensorInference.initial_state(bp) isa TensorInference.BPState
	state, info = belief_propagate(bp; max_iter=100, tol=1e-6)
	@test info.converged
	@test info.iterations < 100
	contraction_res = TensorInference.contraction_results(state)
	tnet = TensorNetworkModel(mps_uai)
	mars = marginals(state)
	mars_tnet = marginals(tnet)
	for v in 1:TensorInference.num_variables(bp)
	@test mars[[v]] ≈ mars_tnet[[v]] atol=1e-4
	end
	end

Also there is some problem in using BP to approximately contracting MPS yuqingrong/IsoPEPS.jl#25

Possible bugs

It seems that it is because cost_and_gradient is used to calculate a tensor network without certain tensors:

TensorInference.jl/src/belief.jl

Line 82 in d914583

cost, gradient = cost_and_gradient(code, (t, vectors_in...))

One needs to do conj instead of direct usage:

using OMEinsum
A = rand(ComplexF64, 4, 4)
B = rand(ComplexF64, 4, 4)
C = rand(ComplexF64, 4, 4)
code = ein"(ij, jk), ki->"
cost, grad = cost_and_gradient(code, (A, B, C))

@assert !isapprox(ein"ij, ij->"(grad[1], A)[], cost[])
@assert isapprox(ein"ij, ij->"(conj(grad[1]), A)[], cost[])

Fix

After adding conj in the BP part and marginal function:

TensorInference.jl/src/mar.jl

Line 80 in d914583

cost, grads = cost_and_gradient(tn.code, (adapt_tensors(tn; usecuda, rescale)...,))

the first test aforementioned is passed, but the second one ("belief propagation on circle") still fail. There might be some other issues.

[GiggleLiu]

Nice catch, it makes sense. Could you please submit a PR to fix this issue? Thank you!

[fliingelephant]

Fixed in #100

Also note there will be additional gauge degrees of freedom in BP approximately contracting complex TNs.