Get added to urllib3.

[Lukasa]

Let's start this discussion right now. I'd like to have hyper eventually be integrated into urllib3, giving all the happy HTTP/2.0 love to users of urllib3 (and indirectly, users of Requests). What do we think hyper needs to make this happen?

I don't expect this to happen quickly, and certainly not until hyper is more mature, so let's have a really honest discussion about what we need.

Potentially interested parties: @shazow, @sigmavirus24, @t-8ch, @kennethreitz.

[shazow]

We talked about hyper having a layer which acts as a drop-in replacement for httplib. That would be easiest to transition with.

If I recall the scope for hyper right now is to only support HTTP/2.0+? If there are plans for having HTTP/1.0+ support, then I'd be happy to drop httplib altogether and use whatever native interface hyper has.

We'll probably need an easy way to differentiate between an HTTP/2.0+ vs HTTP/1.1- server, ideally as early in the connection process as possible. Maybe some utility helper for this could live inside hyper?

Also, I assume if we're moving forward, then we're going to vendor this into urllib3.

[Lukasa]

The drop-in replacement layer is on the table, and is the interface I would expect urllib3 to use.

You're right about the scope stuff. At this time, writing a complaint HTTP/1.1 parser is a little bit too much work, and I'll introduce some alarming bugs into urllib3 that I doubt you want. I'm open to considering it, but it's very long term.

What do you have in mind when it comes to differentiating? The obvious concern is about connection pooling, yeah?

As for vendoring: that was what I expected. =) For all the reasons Requests vendors urllib3, urllib3 would want to vendor hyper.

[shazow]

The drop-in replacement layer is on the table, and is the interface I would expect urllib3 to use.

Would be neat if this compatibility layer automagically chose whether to use hyper or httplib, depending on the server type. That would truly be the minimal amount of work needed for urllib3 to migrate.

You're right about the scope stuff. At this time, writing a complaint HTTP/1.1 parser is a little bit too much work, and I'll introduce some alarming bugs into urllib3 that I doubt you want. I'm open to considering it, but it's very long term.

I'm willing to take the risk if you decide to delve into it. :)

What do you have in mind when it comes to differentiating? The obvious concern is about connection pooling, yeah?

I mean more if it was up to urllib3 to decide whether to use hyper or httplib for any given connection/request, it'd be nice if there were some helpers which streamlined that decision.

[Lukasa]

Ah, yes, that's the compatibility layer plan. It'll use TLS next-protocol-negotiation to make the decision. The biggest difficulty here is trying to do this cleanly while maintaining http.client's "connect as late as possible" policy.

[shazow]

Yea, not sure how much work/complexity this kind of abstraction would involve. If you'd like a more incremental approach, I'd be open to having a separate (or configurable) non-default PoolManager dedicated for HTTP/2.0+ to start.

[Lukasa]

I don't think it'll be too bad. Maybe.

[Lukasa]

So the consensus here is that the obvious block is #3.