No you don't, sort of don't. When you look at the Bugout [1] library, you see that they're using the infrastructure of WebTorrent to fix that.
I emailed the author and he said he's working on a form of infrastructure in which the infrastructure takes care of this problem.
So yea, you don't need a signaling server then. I'm pretty sure the extended version of this idea also includes STUN servers. And I think you'd always need to pay for TURN servers as the NAT is then simply disabling P2P all-together [2].
[2] Context for people who are like what are all these terms?
In WebRTC when you have 2 computers behind different corporate networks trying to connect they need to go through a router. A router performs Network Address Translation (NAT), the internal IP (usually 192.168.178.xxx, definitely not always) needs to become the public IP. The problem is that when both networks are too complicated in its structure, then WebRTC cannot work in undoing this layer of translation.
In order to circumvent this, they tunnel your data through a server that has a public facing IP. They call this a Traversal Using Relays around NAT (TURN) server. IMO, you could just see it as an intermediary that shoves data from A to B and vice versa. However, IMO this defeats the point of P2P as you need a 3rd-party helping you to get the data to the other side.
Note: other relevant ideas are signaling servers (i.e. knowing each other's reachable address through an intermediary server)
and STUN servers (a server that tells you your own public facing IP address).
I would argue that this is still "a way to establish a connection between the two browsers" and therefore does not contradict my comment, but this does seem very interesting and I will look into it further.
Man, my thinking got lazy these days. I read an argument a while back on HN saying that WebRTC is not purely P2P, I was basically insta-responding to that.
Yes, and it's tricky to make a smooth UX out of it in a web browser setting but the technology is absolutely there.
IIRC we managed to make it server-less (in the true sense) apart from the need of a Namecoin gateway.
Typically you would use a signalling server coupled with STUN/TURN to discover peers and establish WebRTC connections/routes but it's not required. You could take a look at Rymd above, IIRC the principles are described in the paper. Here's also a minimal PoC that I found by a quick duck:
Shameless plug: As an alternative to solutions like WebTorrent, if you do need to rely on a signaling server but don't want to send your signaling data (ICE candidates containing IP addresses in this case) unencrypted over a cloud service like Firebase, it's possible to use an end-to-end encrypted protocol for this use case: https://saltyrtc.org/ The good thing about this protocol is that you don't need to trust the server.
(Of course it would be nice if we could just send an UDP broadcast or announce the device via mDNS/Avahi, but browsers do not allow direct network socket access, due to reasons™.)