Robin Tarsiger
2017-08-20 21:48:43 UTC
Good day, tor-dev;
As some of you know, I've been working on a patchset for Tor to allow
it to participate in Pluggable Transports 2.0 configuration, primarily
the new JSON Parameter Block SOCKS method (at least that's what I've
been calling it in the absence of a more official name), almost but
not quite as described in section 3.3.4 of PT2 draft 2 [1]---i.e.,
basically #21816 [2].
[1] https://www.pluggabletransports.info/assets/PTSpecV2Draft2.pdf
[2] https://trac.torproject.org/projects/tor/ticket/21816
I'm attaching a draft patchset which adds this functionality, with the
intent of getting feedback and making remaining cleanups or other
modifications necessary to get it merged into Tor. I have successfully
completed circuits through an obfs4 bridge using both obfs4proxy (PT1)
and a version of shapeshifter-dispatcher (PT2) using a patched Tor. I've
tried to follow the local style, but the preferred implementation
strategies aren't always clear, and of course I'd appreciate any reports
of other problems.
A forked Git repository is also available on Bitbucket [5][6], which
will be updated as I make remaining changes.
[5] https://bitbucket.org/DasyatidPrime/tor-rtt2017-21816.git (Git)
[6] https://bitbucket.org/DasyatidPrime/tor-rtt2017-21816/src (Web)
More implementation details are below if you're interested in this;
thanks for your attention. I'll try to be around on IRC more during
the week, so feel free to ping me there as well.
-RTT
... Details:
Not visible above, related to the target functionality:
- I'm assuming the SOCKS method includes a response with an
analogous structure to RFC 1929; PT2 draft 2 doesn't specify
one. I've cleared this with blanu, and that's intended for the
next draft of the PT2 specification.
- Similarly, the length prefix is in fact big-endian; the example in
PT2 draft 2 is wrong, though the text is correct.
- We're looking into getting an IANA assignment for the method
number. Technically this probably meets the requirements for the
private use block, but I feel like interop might be easier later
on, and it could simplify code paths in places to have a registered
number (mainly if it's possible to decouple the method negotiation
from the configuration-version plumbing). I believe this is still
in limbo.
- For the PT2 side, I've been testing against my branch of
shapeshifter-dispatcher [3] compiled with my branch of
shapeshifter-ipc [4] since there were some breaking changes to
Shapeshifter upstream which were otherwise preventing it from
interoperating with Tor. I'm planning to help merge fixes into
Shapeshifter upstream as I can; it's my current understanding that
there aren't any other PT2 managed transport implementations to
test against.
[3]
https://github.com/OperatorFoundation/shapeshifter-dispatcher/tree/rtt2017
[4] https://github.com/OperatorFoundation/shapeshifter-ipc/tree/rtt2017
Issues on my radar currently (comments appreciated):
- We probably want unit tests for the (limited) JSON encoding
functions, and for the factored-out RFC 1929 encoding functions.
Anything else that looks feasibly testable here?
- It's not clear to me whether negotiating a PT2 configuration
version still allows PT1-style RFC 1929 parameter encoding so that
managed transports can support the new configuration version and
the new SOCKS method separately. I've assumed it might be
possible, so far, but that's not being tested against anything.
- I currently restrict parameters to ASCII to avoid either writing
a JSON encoder that can spit out invalid JSON or writing a JSON
encoder that has to validate incoming UTF-8. The impression I've
gotten is that this is probably okay, but if there are counter-
examples, I can put in UTF-8 passthrough.
- The commit sequence isn't the cleanest. How high a priority is it
to reorder/combine patch hunks to make a cleaner one?
- We still need a 'changes' file. (What would be an appropriate
heading for this? Is this a minor feature, for instance?)
A few other questions:
- Is there an effective way of doing automated testing of the SOCKS
state machine currently in Tor? I didn't see anything obvious in
the test directory. This seems like the most fragile part,
especially since both the original and modified versions are not
very explicit in their state machine nature and are split between
multiple files.
- Can there ever be more than one managed_proxy_t to a transport
name? More generally, is there a relational diagram of the main
Tor data structures somewhere? A lot of the way the plumbing for
state and configuration information is set up feels kind of
fragile.
As some of you know, I've been working on a patchset for Tor to allow
it to participate in Pluggable Transports 2.0 configuration, primarily
the new JSON Parameter Block SOCKS method (at least that's what I've
been calling it in the absence of a more official name), almost but
not quite as described in section 3.3.4 of PT2 draft 2 [1]---i.e.,
basically #21816 [2].
[1] https://www.pluggabletransports.info/assets/PTSpecV2Draft2.pdf
[2] https://trac.torproject.org/projects/tor/ticket/21816
I'm attaching a draft patchset which adds this functionality, with the
intent of getting feedback and making remaining cleanups or other
modifications necessary to get it merged into Tor. I have successfully
completed circuits through an obfs4 bridge using both obfs4proxy (PT1)
and a version of shapeshifter-dispatcher (PT2) using a patched Tor. I've
tried to follow the local style, but the preferred implementation
strategies aren't always clear, and of course I'd appreciate any reports
of other problems.
A forked Git repository is also available on Bitbucket [5][6], which
will be updated as I make remaining changes.
[5] https://bitbucket.org/DasyatidPrime/tor-rtt2017-21816.git (Git)
[6] https://bitbucket.org/DasyatidPrime/tor-rtt2017-21816/src (Web)
More implementation details are below if you're interested in this;
thanks for your attention. I'll try to be around on IRC more during
the week, so feel free to ping me there as well.
-RTT
... Details:
Not visible above, related to the target functionality:
- I'm assuming the SOCKS method includes a response with an
analogous structure to RFC 1929; PT2 draft 2 doesn't specify
one. I've cleared this with blanu, and that's intended for the
next draft of the PT2 specification.
- Similarly, the length prefix is in fact big-endian; the example in
PT2 draft 2 is wrong, though the text is correct.
- We're looking into getting an IANA assignment for the method
number. Technically this probably meets the requirements for the
private use block, but I feel like interop might be easier later
on, and it could simplify code paths in places to have a registered
number (mainly if it's possible to decouple the method negotiation
from the configuration-version plumbing). I believe this is still
in limbo.
- For the PT2 side, I've been testing against my branch of
shapeshifter-dispatcher [3] compiled with my branch of
shapeshifter-ipc [4] since there were some breaking changes to
Shapeshifter upstream which were otherwise preventing it from
interoperating with Tor. I'm planning to help merge fixes into
Shapeshifter upstream as I can; it's my current understanding that
there aren't any other PT2 managed transport implementations to
test against.
[3]
https://github.com/OperatorFoundation/shapeshifter-dispatcher/tree/rtt2017
[4] https://github.com/OperatorFoundation/shapeshifter-ipc/tree/rtt2017
Issues on my radar currently (comments appreciated):
- We probably want unit tests for the (limited) JSON encoding
functions, and for the factored-out RFC 1929 encoding functions.
Anything else that looks feasibly testable here?
- It's not clear to me whether negotiating a PT2 configuration
version still allows PT1-style RFC 1929 parameter encoding so that
managed transports can support the new configuration version and
the new SOCKS method separately. I've assumed it might be
possible, so far, but that's not being tested against anything.
- I currently restrict parameters to ASCII to avoid either writing
a JSON encoder that can spit out invalid JSON or writing a JSON
encoder that has to validate incoming UTF-8. The impression I've
gotten is that this is probably okay, but if there are counter-
examples, I can put in UTF-8 passthrough.
- The commit sequence isn't the cleanest. How high a priority is it
to reorder/combine patch hunks to make a cleaner one?
- We still need a 'changes' file. (What would be an appropriate
heading for this? Is this a minor feature, for instance?)
A few other questions:
- Is there an effective way of doing automated testing of the SOCKS
state machine currently in Tor? I didn't see anything obvious in
the test directory. This seems like the most fragile part,
especially since both the original and modified versions are not
very explicit in their state machine nature and are split between
multiple files.
- Can there ever be more than one managed_proxy_t to a transport
name? More generally, is there a relational diagram of the main
Tor data structures somewhere? A lot of the way the plumbing for
state and configuration information is set up feels kind of
fragile.