Handle unparsable multiaddrs

[Stebalien]

Due to the fact that binary multiaddrs don't include protocol definitions, we can't parse them unless we know all the relevant protocols.

However, we can parse a prefix. Therefore, I'd like to propose a special, string-only "unknown" protocol that takes a single multibase encoded argument. That is: `/ip4/1.2.3.4/tcp/123/unknown/bxyz". This would only exist in the string format and would allow us to keep and use multiaddrs we don't fully understand.

This should fix some of the problems described in #6.

[alanshaw]

A few things I'd like to explore:

1. How can we retain the original string? In terms of printing these addrs out, it's more useful to be able to print the representation that we recieved, not a /unknown/whatever even if it cannot be used. The use case is an old js-ipfs-api requesting swarm.addrs to be printed out on a webpage. It might not understand how to parse QUIC addrs but it's preferable to print the QUIC addr than an "unknown" addr
2. How can we distinguish between a multiaddr that is valid or not? If we can pass it any string protocol - how do we determine if it's valid (at least in our version of the codec table)? What would valid mean now? What do we do when passing a string/buffer to the multiaddr constructor?
3. If we create a "string only" multiaddr - what happens when we request the buffer version - throw error?

[alanshaw]

@jacobheun it might be useful to have your opinion on this?

[olizilla]

The requirement that a multiaddr can be represented as a buffer is tricky. Every user of multiaddr has a stale copy of the buffer-to-protocol mapping. It's trivial to syntacitically validate a multiaddr string, but hard to semantically validate that protocols are valid if the definition of valid is their existance in the mapping table.

We will keep running into situations where peers and apps and api clients are running with different versions of multiaddr, with different mapping tables, and older ones unable to parse newer ones.

Having a string only format for multiaddr that doesn't guarantee it can be converted to a buffer would at least tick the points 1 to 4 on the problem statement https://github.com/multiformats/multiaddr#introduction

[Stebalien]

I think we're talking past each other a bit here. My primary concern is relay dialing (which should have been in the issue description...). That is, I need to be able to use a multiaddr prefix (encoded) without the rest.

For example: /ip4/1.2.3.4/tcp/1234/p2p/QmRelay/p2p-circuit/ip4/tcp/1234/special_protocol/p2p/QmTarget.

QmRelay and QmTarget may understand special_protocol while I may not.

Basically, I'm worried about creating unnecessary network partitions because we're being too strict in parsing our inputs.

We still need to solve the UX issues.

How can we retain the original string?

We usually don't and can't know the original string (e.g., binary peer address records from the DHT).

How can we distinguish between a multiaddr that is valid or not? If we can pass it any string protocol - how do we determine if it's valid (at least in our version of the codec table)? What would valid mean now? What do we do when passing a string/buffer to the multiaddr constructor?

This proposal tries to cover the use-case where I don't really care about the rest of the multiaddr. Ideally, I'd be able to validate it, but even if I could, I'm just going to pass it off to someone else anyways.

If we create a "string only" multiaddr - what happens when we request the buffer version - throw error?

By string-only, I meant that /unknown/... would only appear in the string representation. When converting back to the binary representation, we'd parse $x/unknown/$y back into ParseMultiaddr($x) + CastMultiaddrBytes(DecodeMultibase($y)). That is, everything after unknown would just be the part of the multiaddr we failed to parse (binary), multibase-encoded.

Now, there is the case where a user gives me /ip4/.../...../someproto and I don't know what someproto is. This proposal doesn't handle that case but I guess we could:

1. Allocate a special "string" path multicodec.
2. Use that multicodec and encode the rest of the string as the argument.

That is, if we ran across $x/someproto/$y, we'd encode this to BinaryMultiaddr($x) + StringMultiaddrCodec + Length + $y. When parsing such a multiaddr, implementations would do their best to lower it to a the correct binary format.

However, really, "upgrade your client" may also be a valid solution here.

[alanshaw]

Thanks for your thoughts!

That is, if we ran across $x/someproto/$y, we'd encode this to BinaryMultiaddr($x) + StringMultiaddrCodec + Length + $y. When parsing such a multiaddr, implementations would do their best to lower it to a the correct binary format.

Do we need a StringMultiaddrCodec - could we just use the unknown codec?

That aside, I'm interested to see if/how this would work in code - I'll see if I can work up a PR to js-multiaddr and report back :D

[Stebalien]

Do we need a StringMultiaddrCodec - could we just use the unknown codec?

So, there wouldn't really be an unknown "codec". Basically, we're trying to handle two inverse cases:

1. We have a string and need to turn it into binary. In this case, we need a protocol that only exists in binary form. When converting back to a string, the protocol would disappear.
2. We have a binary address and need to turn it into a string. In this case, we need a protocol that only exists in string form. When encoding to the binary representation, it would disappear as we already know the binary protocol code.

Basically, we need a StringInBinary protocol (StringMultiaddrCodec) and a BinaryInString protocol (the "unknown" protocol).

That aside, I'm interested to see if/how this would work in code - I'll see if I can work up a PR to js-multiaddr and report back :D

The second half (binary in strings) has a PR here: multiformats/go-multiaddr#74.

However, I'm still worried about multiple binary representations of multiaddrs. That's why I'm less sure we want StringInBinary.