[tor-dev] Dir auths using 2x bandwidth in last week

[Roger Dingledine]

https://atlas.torproject.org/#details/9695DFC35FFEB861329B9F1AB04C46397020CE31
https://atlas.torproject.org/#details/F2044413DAC2E02E3D6BCF4735A19BCA1DE97281
https://atlas.torproject.org/#details/BD6A829255CB08E66FBE7D3748363586E46B3810
https://atlas.torproject.org/#details/74A910646BCEEFBCD2E874FC1DC997430F968145
https://atlas.torproject.org/#details/7EA6EAD6FD83083C538F44038BBFA077587DD755
all show a big increase in sent bytes starting at the end of July.

It isn't growth in Tor users, since those have stayed relatively flat
in the last two weeks.

And the new rate seems to be the new normal -- it's showing no signs of
going back to the old rate.

I would assume it's outgoing directory stuff, since that's most of what
dir auths do.

Any guesses?

--Roger

[teor]

Post by Roger Dingledine
https://atlas.torproject.org/#details/9695DFC35FFEB861329B9F1AB04C46397020CE31
https://atlas.torproject.org/#details/F2044413DAC2E02E3D6BCF4735A19BCA1DE97281
https://atlas.torproject.org/#details/BD6A829255CB08E66FBE7D3748363586E46B3810
https://atlas.torproject.org/#details/74A910646BCEEFBCD2E874FC1DC997430F968145
https://atlas.torproject.org/#details/7EA6EAD6FD83083C538F44038BBFA077587DD755
all show a big increase in sent bytes starting at the end of July.
It isn't growth in Tor users, since those have stayed relatively flat
in the last two weeks.
And the new rate seems to be the new normal -- it's showing no signs of
going back to the old rate.
I would assume it's outgoing directory stuff, since that's most of what
dir auths do.
Any guesses?

In July, Tor 0.3.0 became the most common relay version in the network,
growing at quite a rapid rate:

https://metrics.torproject.org/versions.html

There doesn't seem to be any corresponding Tor Browser release in that
timeframe:

3 July: https://blog.torproject.org/blog/tor-browser-702-released
8 August: https://blog.torproject.org/blog/tor-browser-704-released

(Our estimates suggest that half the load on directory authorities is
from relays, and half is from clients.)

I wonder if the new guard selection algorithm, or some other relay
change, is causing relays to download more descriptors from more
directory authorities?

T
--
Tim Wilson-Brown (teor)

teor2345 at gmail dot com
PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B
ricochet:ekmygaiu4rzgsk6n
------------------------------------------------------------------------

[David Fifield]

Post by teor

Post by Roger Dingledine
https://atlas.torproject.org/#details/9695DFC35FFEB861329B9F1AB04C46397020CE31
https://atlas.torproject.org/#details/F2044413DAC2E02E3D6BCF4735A19BCA1DE97281
https://atlas.torproject.org/#details/BD6A829255CB08E66FBE7D3748363586E46B3810
https://atlas.torproject.org/#details/74A910646BCEEFBCD2E874FC1DC997430F968145
https://atlas.torproject.org/#details/7EA6EAD6FD83083C538F44038BBFA077587DD755
all show a big increase in sent bytes starting at the end of July.
It isn't growth in Tor users, since those have stayed relatively flat
in the last two weeks.
And the new rate seems to be the new normal -- it's showing no signs of
going back to the old rate.
I would assume it's outgoing directory stuff, since that's most of what
dir auths do.
Any guesses?

In July, Tor 0.3.0 became the most common relay version in the network,
https://metrics.torproject.org/versions.html
There doesn't seem to be any corresponding Tor Browser release in that
3 July: https://blog.torproject.org/blog/tor-browser-702-released
8 August: https://blog.torproject.org/blog/tor-browser-704-released

2 July is when deb.torproject.org switched to 0.3.0.

nusenu noted it here:
https://twitter.com/nusenu_/status/884128686764687361

I added a line already to
https://trac.torproject.org/projects/tor/wiki/doc/MetricsTimeline

[teor]

Post by David Fifield

Post by teor

Post by Roger Dingledine
https://atlas.torproject.org/#details/9695DFC35FFEB861329B9F1AB04C46397020CE31
https://atlas.torproject.org/#details/F2044413DAC2E02E3D6BCF4735A19BCA1DE97281
https://atlas.torproject.org/#details/BD6A829255CB08E66FBE7D3748363586E46B3810
https://atlas.torproject.org/#details/74A910646BCEEFBCD2E874FC1DC997430F968145
https://atlas.torproject.org/#details/7EA6EAD6FD83083C538F44038BBFA077587DD755
all show a big increase in sent bytes starting at the end of July.
It isn't growth in Tor users, since those have stayed relatively flat
in the last two weeks.
And the new rate seems to be the new normal -- it's showing no signs of
going back to the old rate.
I would assume it's outgoing directory stuff, since that's most of what
dir auths do.
Any guesses?

In July, Tor 0.3.0 became the most common relay version in the network,
https://metrics.torproject.org/versions.html
There doesn't seem to be any corresponding Tor Browser release in that
3 July: https://blog.torproject.org/blog/tor-browser-702-released
8 August: https://blog.torproject.org/blog/tor-browser-704-released

2 July is when deb.torproject.org switched to 0.3.0.

We merged a fallback directory mirror change to 0.2.8 and later in
mid-May, but I think that's too far back. (The actual releases were
up to a month later.)

https://trac.torproject.org/projects/tor/ticket/21564

If we want to reduce the load on authorities, we can backport this
change, which makes clients try fallbacks before authorities:

https://trac.torproject.org/projects/tor/ticket/17750

It's been in master (0.3.2) for almost 2 months now.

T
--
Tim Wilson-Brown (teor)

teor2345 at gmail dot com
PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B
ricochet:ekmygaiu4rzgsk6n
------------------------------------------------------------------------

[teor]

Post by teor

Post by Roger Dingledine
https://atlas.torproject.org/#details/9695DFC35FFEB861329B9F1AB04C46397020CE31
https://atlas.torproject.org/#details/F2044413DAC2E02E3D6BCF4735A19BCA1DE97281
https://atlas.torproject.org/#details/BD6A829255CB08E66FBE7D3748363586E46B3810
https://atlas.torproject.org/#details/74A910646BCEEFBCD2E874FC1DC997430F968145
https://atlas.torproject.org/#details/7EA6EAD6FD83083C538F44038BBFA077587DD755
all show a big increase in sent bytes starting at the end of July.
It isn't growth in Tor users, since those have stayed relatively flat
in the last two weeks.
And the new rate seems to be the new normal -- it's showing no signs of
going back to the old rate.
I would assume it's outgoing directory stuff, since that's most of what
dir auths do.
Any guesses?

In July, we went from 1% of fallback directory mirrors being down, to about
10% of fallback directory mirrors being down.

See "Fallback Directories Running, Past 90 Days" on:
https://consensus-health.torproject.org/graphs.html
(Needs JS, SVG)

This means that 10% of client requests go straight to directory
authorities. For the other 90%, the first fallback and first directory
authority request happen simultaneously, and the authority often wins
(and gets to supply the client's first consensus).

Deploying #17750 in 0.3.2 (or backporting it to earlier releases)

Post by teor
If we want to reduce the load on authorities, we can backport this
https://trac.torproject.org/projects/tor/ticket/17750
It's been in master (0.3.2) for almost 2 months now.

T
--
Tim Wilson-Brown (teor)

teor2345 at gmail dot com
PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B
ricochet:ekmygaiu4rzgsk6n
------------------------------------------------------------------------

[Roger Dingledine]

Post by Roger Dingledine
https://atlas.torproject.org/#details/9695DFC35FFEB861329B9F1AB04C46397020CE31
https://atlas.torproject.org/#details/F2044413DAC2E02E3D6BCF4735A19BCA1DE97281
https://atlas.torproject.org/#details/BD6A829255CB08E66FBE7D3748363586E46B3810
https://atlas.torproject.org/#details/74A910646BCEEFBCD2E874FC1DC997430F968145
https://atlas.torproject.org/#details/7EA6EAD6FD83083C538F44038BBFA077587DD755
all show a big increase in sent bytes starting at the end of July.
It isn't growth in Tor users, since those have stayed relatively flat
in the last two weeks.
And the new rate seems to be the new normal -- it's showing no signs of
going back to the old rate.
I would assume it's outgoing directory stuff, since that's most of what
dir auths do.
Any guesses?

Well, whatever it was, it settled out -- back to normal -- once August
ended.

https://atlas.torproject.org/#details/9695DFC35FFEB861329B9F1AB04C46397020CE31

--Roger

[David Fifield]

Post by Roger Dingledine

Post by Roger Dingledine
https://atlas.torproject.org/#details/9695DFC35FFEB861329B9F1AB04C46397020CE31
https://atlas.torproject.org/#details/F2044413DAC2E02E3D6BCF4735A19BCA1DE97281
https://atlas.torproject.org/#details/BD6A829255CB08E66FBE7D3748363586E46B3810
https://atlas.torproject.org/#details/74A910646BCEEFBCD2E874FC1DC997430F968145
https://atlas.torproject.org/#details/7EA6EAD6FD83083C538F44038BBFA077587DD755
all show a big increase in sent bytes starting at the end of July.
It isn't growth in Tor users, since those have stayed relatively flat
in the last two weeks.
And the new rate seems to be the new normal -- it's showing no signs of
going back to the old rate.
I would assume it's outgoing directory stuff, since that's most of what
dir auths do.
Any guesses?

Well, whatever it was, it settled out -- back to normal -- once August
ended.
https://atlas.torproject.org/#details/9695DFC35FFEB861329B9F1AB04C46397020CE31

The start and end of the doubling of dirauth bandwidth coincide with a
temporary doubling or tripling of reported users from the Netherlands:

https://people.torproject.org/~dcf/metrics-country.html?start=2017-07-01&end=2017-09-30&country=nl

There is a somewhat similar pattern in the Seychelles, Lithuania, and
Romania, all of which had unexplained increases at the same time as the
Netherlands.

https://lists.torproject.org/pipermail/metrics-team/2017-August/000428.html
https://people.torproject.org/~dcf/metrics-country.html?start=2017-07-01&end=2017-09-30&country=sc
https://people.torproject.org/~dcf/metrics-country.html?start=2017-07-01&end=2017-09-30&country=lt
https://people.torproject.org/~dcf/metrics-country.html?start=2017-07-01&end=2017-09-30&country=ro