[tor-dev] Proposal: only parse .torrc files in torrc.d directory

[iry]

Dear Tor Developers,

I have been testing and using the torrc.d feature for a while, and
here is a potential improvement we may make.

%include /etc/torrc.d/

Every file in the directory will be treated and parsed as a valid Tor
configuration file. However, sometime, this may not be what users and
developers want.

For example, users may use /etc/torrc.d/50_user.torrc as the place to
put their own torrc configurations. But sometimes, when they use a
text editor to edit it, the text editor will leave a
/etc/torrc.d/50_user.torrc~ file which will also be treated as a valid
torrc file.

Another example that also happens very frequently is, when dpkg does
an update on /etc/torrc.d/30_distribution.torrc, users' previous
configuration can be saved as
/etc/torrc.d/30_distribution.torrc.dpkg-old which will also be parsed
by Tor.

In best case users will just be frustrated because Tor does not work
as expected and in worst case this could be dangerous. This could be a
severe problem especially because of the following reasons:
1. filename.torrc~ filename.torrc.dpkg-old has higher priority than
filename.torrc when Tor does the parsing.
2. In most cases, this will happen without being noticed by the normal
suer.

Therefore, may I propose to let Tor parse only the files whose name
ends with .torrc ? Or maybe even only parse number_filename.torrc for
better consistency and for more clear priority order?

Thank you very much! Looking forward to hearing your insights!

Best Regards,
iry

[teor]

Post by iry
In best case users will just be frustrated because Tor does not work
as expected and in worst case this could be dangerous. This could be a
1. filename.torrc~ filename.torrc.dpkg-old has higher priority than
filename.torrc when Tor does the parsing.
2. In most cases, this will happen without being noticed by the normal
suer.
Therefore, may I propose to let Tor parse only the files whose name
ends with .torrc ?

Yes, this is standard behaviour among many tools.

Post by iry
Or maybe even only parse number_filename.torrc for
better consistency and for more clear priority order?

No, this is counter-intuitive. It will confuse many users.
It is not how most other tools work.

T

[iry]

Post by teor

Post by iry
Therefore, may I propose to let Tor parse only the files whose
name ends with .torrc ?

Yes, this is standard behaviour among many tools.

Post by iry
Or maybe even only parse number_filename.torrc for better
consistency and for more clear priority order?

No, this is counter-intuitive. It will confuse many users. It is
not how most other tools work.

Thank you so much for the feedback, teor!

I have opened the ticket #25140 for this. Shall I try to make a patch
on this? Or is it preferred to leave it to developers from TPO?

Thank you very much!

Best Regards,
iry

[teor]

Post by iry

Post by teor

Post by iry
Therefore, may I propose to let Tor parse only the files whose
name ends with .torrc ?

Yes, this is standard behaviour among many tools.

To be more precise, most tools accept files ending in ".conf".
We might want tor to accept ".conf" for consistency.

I suggest we also accept files called "torrc", or ending in ".torrc".
This should probably also include files called literally ".torrc".

Since this is a breaking change, we might want to check if any
distribution has adopted another naming scheme, and avoid
breaking that naming scheme.

Post by iry

Post by teor

Post by iry
Or maybe even only parse number_filename.torrc for better
consistency and for more clear priority order?

No, this is counter-intuitive. It will confuse many users. It is
not how most other tools work.

Thank you so much for the feedback, teor!
I have opened the ticket #25140 for this. Shall I try to make a patch
on this? Or is it preferred to leave it to developers from TPO?

We accept patches from anyone.

If you'd like your patch to be reviewed quickly:
* put the ticket in 0.3.3, because it's a bugfix
* change the status to "needs review" once you have a patch

Thanks!

T

[iry]

To be more precise, most tools accept files ending in ".conf". We
might want tor to accept ".conf" for consistency.
I suggest we also accept files called "torrc", or ending in
".torrc". This should probably also include files called literally
".torrc".

Thank you for your suggestions, teor! I have updated it in the ticket.

Since this is a breaking change, we might want to check if any
distribution has adopted another naming scheme, and avoid breaking
that naming schem

I agree. I will wait for a little bit before implementing it so that
we may get more input on the issue. :)

If you'd like your patch to be reviewed quickly: * put the ticket
in 0.3.3, because it's a bugfix * change the status to "needs
review" once you have a patch

Thank you so much for your instructions!

Best Regards,
iry

[teor]

Post by iry

Post by teor
Since this is a breaking change, we might want to check if any
distribution has adopted another naming scheme, and avoid breaking
that naming schem

I agree. I will wait for a little bit before implementing it so that
we may get more input on the issue. :)

Expecting packagers to follow every thread and reply here is unrealistic.

Would you mind doing the search yourself, like Patrick asked?
It seems to have dropped off your TODO list.

Could you please look at existing .d folders of any other
projects tell me what you think? Perhaps discuss this with Tor Project.

https://forums.whonix.org/t/torrc-d-is-comming/4041/13

From my quick search, it appears that the Debian feature request is still open,
and no other distro is using torrc.d yet. But you should check, too.

T

[iry]

Hi teor!

Could you please look at existing .d folders of any other projects
tell me what you think? Perhaps discuss this with Tor Project.
[...] From my quick search, it appears that the Debian feature
request is still open, and no other distro is using torrc.d yet.
But you should check, too.

I went through all Tor packages listed here:
https://trac.torproject.org/projects/tor/wiki/doc/packages and no
distros shipped a torrc with %include line enabled.

I know Whonix will not use torrc.d before next stable version. I also
did a grep -r -i "%include" on Tails source code and I do not think
Tails has enabled it by default.

nickm suggested proposed to create a new syntax to take care of the

%include /etc/torrc.d/*.conf

Here is my thoughts on this:

1. I agree that "[a]nybody who currently has a working setup will have
it fail if we start requiring a suffix that they didn't know to
provide", which is not good for compatibility. But, letting people
still use or will be able to use a setting that is not recommended
anymore seems also not to be a very good idea? Considering the
potential danger of parsing all the files, shall we go a little bit
aggressive? I would rather break people's current potentially
dangerous settings. What do you think?

2. Since no distros I know has enabled this feature by default, I
guess there are only a very small number of users has enabled this
feature. Will an info in the release note saying "%include
/etc/torrc.d/ will only pase files suffixed with .torrc files" be
enough to inform them? Maybe we can even document the manual migration
somewhere.

3. %include /etc/torrc.d/*.conf syntax is very flexible so that Tor
does not have to decide which extension names should be parsed.

4. %include /etc/torrc.d/*.conf syntax explicitly says which extension
name will be used rather than the implicit document.

5. But is it a good idea to make the syntax that flexible? For
example, anon-connection-wizard will generate a torrc files in torrc.d
directory, I (and maybe many other developers) prefer writing to a
file that I can guarantee it will be parsed in most case. If I write
to 40_anon-connection-wizard.conf but some people set to pase .torrc
or anything else only, it will be not be very good? (I do not want
anon-connection-wizard to touch /etc/tor/torrc)

Finally, do you think it is a good idea to switch to the ticket for
further discussion to avoid cross posting and high volume on @tor-dev?

Thank you very much!

Best Regards,
iry

[teor]

Post by iry
…
https://trac.torproject.org/projects/tor/wiki/doc/packages and no
distros shipped a torrc with %include line enabled.
I know Whonix will not use torrc.d before next stable version. I also
did a grep -r -i "%include" on Tails source code and I do not think
Tails has enabled it by default.
nickm suggested proposed to create a new syntax to take care of the

%include /etc/torrc.d/*.conf

1. I agree that "[a]nybody who currently has a working setup will have
it fail if we start requiring a suffix that they didn't know to
provide", which is not good for compatibility.

nickm is right: we released this feature in a stable release.
Let's not break it.

Post by iry
But, letting people
still use or will be able to use a setting that is not recommended
anymore seems also not to be a very good idea? Considering the
potential danger of parsing all the files, shall we go a little bit
aggressive? I would rather break people's current potentially
dangerous settings. What do you think?

I would rather not break existing configs in stable releases.

Instead, I think we can:
* document the recommend usage
* warn on potentially unexpected files

Tor already warns on most duplicate options, so the issue should be
obvious in most cases.

Does Tor log each config file name when it is loaded?
Logging the file name would help users to debug issues like this.

Post by iry
2. Since no distros I know has enabled this feature by default, I
guess there are only a very small number of users has enabled this
feature. Will an info in the release note saying "%include
/etc/torrc.d/ will only pase files suffixed with .torrc files" be
enough to inform them? Maybe we can even document the manual migration
somewhere.

I think these are good things to do.

Post by iry
3. %include /etc/torrc.d/*.conf syntax is very flexible so that Tor
does not have to decide which extension names should be parsed.
4. %include /etc/torrc.d/*.conf syntax explicitly says which extension
name will be used rather than the implicit document.

This seems like a good design.

Post by iry
5. But is it a good idea to make the syntax that flexible? For
example, anon-connection-wizard will generate a torrc files in torrc.d
directory, I (and maybe many other developers) prefer writing to a
file that I can guarantee it will be parsed in most case. If I write
to 40_anon-connection-wizard.conf but some people set to pase .torrc
or anything else only, it will be not be very good? (I do not want
anon-connection-wizard to touch /etc/tor/torrc)

We should recommend a default extension for packagers.

But our standard advice to users in situations like this is:
"Don't do that then"

In particular, our advice is:
"If you modify your torrc, your tor might not work like you expect"

Post by iry
Finally, do you think it is a good idea to switch to the ticket for

I don't mind.

In my experience, mailing lists are good for discussion, trac tickets are
good for tasks, and gitlab/oniongit are good for code review.

Please let us know on this list if you move the discussion elsewhere.

T

[iry]

Thank you so much for the immediate feedback, teor!

Could you please help me to review the requirements (TODO list) below:

* %include /etc/torrc.d means parsing all the files in the directory
* implement %include /etc/torrc.d/*.extension syntax
* let Tor log each config file it loads
* warn on potentially unexpected files
* document *.conf as the recommend usage
* document how to manually migrate from * to *.conf
* recommend /etc/torrc.d/*.conf as "a default extension for packagers"

Post by teor
In my experience, mailing lists are good for discussion, trac
tickets are good for tasks, and gitlab/oniongit are good for code
review.

Got it!

Thank you very much!

Best Regards,
iry