-
Notifications
You must be signed in to change notification settings - Fork 3
/
CHANGES
executable file
·821 lines (649 loc) · 42.3 KB
/
CHANGES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
Portions Copyright (c) 2006-2007, Joseph B. Kowalski
Portions Copyright (c) 2007 Kasimir Gabert
See LICENSE for licensing information
TorStatus - Tor Network Status Changes
===============================================================================
Version 4 Changes {{{
===============================================================================
--Version 4.0--
* (ticket 56) The DNSEL database no longer is left in memory
* (ticket 58) Read-write history now works through extra-info/digest
* (ticket 8) A logo was created for TorStatus
* (ticket 53) Graphs have been moved to a Perl based framework
* (ticket 64) The interface was greatly modified-includes JS now, but works
equally well without it
* (ticket 65) tns_update.pl now just needs to be run
* (ticket 66) The HSDir flag was added
* (ticket 68) tns_update will no longer break if there are multiple router
names that are the same as the nickname for the opinion source
* (ticket 43 49) Up and down arrows were fixed to work with more browsers
* (ticket 52) A search option is now at the top for ease of use
* (ticket 47) Names of countries are now used intead of just the country code
===============================================================================
}}}
===============================================================================
===============================================================================
Version 3 Changes {{{
===============================================================================
--Version 3.6--
* Will no longer crash with an error if the database is not populated
* An interface has been added to the WHOIS data
* The guard image is now a shield
* (ticket 40) It will now work with modern versions of Tor
* A Debian agent daemon has been created
--Version 3.5--
* Complete interface redesign
* Integrated operating system detection in by default
* Bandwidth change to have a progress bar
* Legend created
* WHOIS query integrated for IP addresses
* Background colors for different router status
* Overall softer colors
* Added notifications for Asc or Desc sort orders
--Version 3.4.2--
* (ticket 12) Fixed error with server IP detection for non-Squid servers
* (ticket 12) Added config option for Squid server
--Version 3.4.1-- (changes made by Kasimir Gabert)
* Fixed bottom headers for new domain
* Updated IP detection so it works through Squid
* Added known mirrors bar
TODO:
* Move the two changes into configuration items
* Centralize the known mirrors and make it dynamic
--Version 3.4-- (end of changes by Joseph Kowalski)
Rename of the DNSBL server to DNSEL (DNS Exit List) server. This was by
recommendation of Roger Dingledine of the Tor project. All filenames, variable
names, etc, have been adjusted to reflect the rename.
Added basic hit counting functionality to the DNSEL server. Now, every five
minutes, the DNSEL server will write a record to the "DNSEL_LOG" table with a
timestamp, a TotalResponses counter, and counters of how many "NOERROR",
"SERVFAIL", "NXDOMAIN", and "NOTIMP" responses it has sent out since it was
started. This procedure runs in it's own thread, and so should not have any
measurable effect on performance.
Improved the code that detects whether a user is accessing the site via a Tor
hidden service. Previously, I checked to see if the user was coming from
127.0.0.1. Now, I check to see if the "HOST" header provided by the client
upon access exists in the "$Hidden_Service_URL" from the config file. If so,
we tell them they are accessing the site as a hidden service. If not, we
proceed on to the normal checks to see if they are accessing through the Tor
network.
Modified the DNSEL server to return SERVFAIL instead of NXDOMAIN for requests
which are outside of the zone the DNSEL server is authoritative for.
Modified the DNSEL server to return NXDOMAIN instead of NOTIMP for requests
other than type 'A' and 'ANY' when the request is in the authoritative zone.
Modified the documentation / default variables for the DNSEL server to use the
"ip-port.torhosts.example.com" domain, rather than "tor.dnsel.example.com".
This encourages users to setup servers that are consistent with the
"torel-design.txt" document maintained by the Tor project.
--Version 3.3--
Complete rewrite of the DNSBL server in Java. This provided a performance
increase on the order of approximately 5x over the old Perl version of the
server. On a Pentium 2.4 GHz machine with 2 GBs of RAM, I was able to
consistently obtain load test results of 200 queries/sec or more. These tests
were conducted across the internet, and not on the local machine.
Addition of the "dnsbl_server.php" page. This page provides usage instructions
for the DNSBL server, and informs users of it's domain name. Note that the
link to "dnsbl_server.php" from the main page will only display if you
set the "$DNSBL_Domain" variable in the "config.php" file, so you can leave
this variable set to 'null' if you are not running a DNSBL server, or do not
wish to advertise it on the web site.
Addition of the "$Hidden_Service_URL" variable to the "config.php" file. If
you are also making the TorNetworkStatus site available through a Tor Hidden
Service, you can enter the URL here, and a link will be added to the main page.
If you are not making the site available as a Hidden Service, or do not wish
to advertise it on the web site, you can leave this variable set to 'null'.
Removal of the "tor_exit_status" page. This functionality is already provided
by the main page (Tells a user if they are accessing the site through the Tor
network or not), and the user already has two options to do custom exit
queries if desired: The "tor_exit_query" page, and the DNSBL server. Both are
linked to from the main page.
Modifications to the "tns_update.php" file and the database create script to
accommodate the re-write of the DNSBL server.
--Version 3.2--
Removed the first type of query that the DNSBL server could perform, which
was to simply lookup if a provided IP address was an active Tor server or not.
This was causing confusion amongst some users, and causing people to focus on
whether or not an IP is a Tor server at all, which is exactly what we are
trying to get away from by providing an answer to the more granular question
"Is this IP address a Tor server that is capable of exiting to my destination
IP and port?", which is precisely what the second (and now only) query type
does. This had the added benefit or making the server a little faster, since
a bit of logic was removed. Please see the "README_DNSBL" file in the source
distribution for full setup and usage information.
--Version 3.1--
Nearly a complete redesign of the DNSBL server added in the last version to
improve performance. Major changes:
1) Previously, there was a Perl script which performed the actual listening
for DNS requests, and when one was received, the script passed on the details
of the request to a PHP script, which is where the main processing happened.
The PHP script would then return a value to the Perl script indicating success
or failure of the request. This had the horrible performance penalty of
needing to invoke PHP every time a new request was received. Now, the entire
process is handled within the same Perl script -- All of the PHP code to
process the DNSBL request has been ported over to Perl -- So there is no
calling an external executable again and again. Additionally, Perl runs faster
than PHP with all other factors being equal.
2) With the old PHP script, since it had to be invoked anew from within the
Perl script with each request, this also meant that a new database connection
had to be setup with each request. This also caused a performance penalty.
Now, since everything is handled from within a single script, a single
database connection can be maintained the entire time the script is running.
3) Previously, when the DNSBL script did it's database queries to pull the
IP addresses and exit policies of Tor routers, it did it the same way that the
main application does, which means joining two large tables together on the
router fingerprint key. While this works fine for the main application and
it's very large queries, it did not suit the performance needs of a DNS
server. Now, I have added two new in-memory database tables specifically for
the DNSBL functionality. These two tables are populated during the update
cycle, and contain only the necessary information (prejoined) required by the
DNSBL server. The fact that they are in memory and are a lot smaller than the
other tables should help the DNSBL server to return queries much quicker.
These two tables act just like the set of NetworkStatus and Descriptor tables
in that only one is ever active at a given time. The update cycle will always
be updating the non-active table in the background while requests can continue
being served by the active table. When the update is done successfully, a flag
in the Status table is toggled to tell the application to begin using the
other table as the active one, and the process repeats -- No downtime for
updates.
Fixed a bug that would cause the DNSBL server to only return "NXDOMAIN"
responses if the database connection was lost until the script was restarted,
even if the database connection came back up later. Now, the server will
return "SERVFAIL" responses while the database server is down, but it will
resume correct operation once the database server comes back up.
Various other minor bug fixes, mainly relating to the "Tor Exit Status" and
"Tor Exit Query" functionality that was added in the last release.
--Version 3.0--
Complete redesign of the way that the application determines if you are
accessing the site through the Tor network or not, or if the provided IP
belongs to a Tor server capable of exiting to the specified IP and port (See
section on "Tor Exit Query" page, below). Previously, we would simply check to
see if the IP address that the user was coming from was a Tor server with the
'Exit' flag set to true, and tell them they were coming through the Tor
network if so. Now, we check if the user is coming from the IP address of a
Tor server, regardless of it's 'Exit' flag status. If so, we pull the Exit
Policy for the matching Tor server(s), look at what IP and Port the client is
accessing us on, and parse the exit policy to determine if the Tor server(s)
would allow or deny such an access. We then give the user information on each
Tor server that matches their exit IP, letting them know if it would allow
access to us or not considering it's exit policy. Finally, we also tell the
user if it appears that they are coming to the site via a Tor Hidden Service
or from the local machine directly. This is determined by checking if the user
is coming to the site from the '127.0.0.1' IP address. Note that the
application now handles situations where there are multiple Tor servers with
IP's that match where the user is coming from (Or the IP provided, in the case
of the "Tor Exit Query" page, see below). In these cases, it will cycle through
each applicable Tor server to provide Exit Policy match information.
Addition of "Tor Exit Status" page. This page provides the same information
about whether a user is accessing the site through the Tor network as is
available on the main page, but without all the other information. This may be
easier for someone who just wants a simple answer for "Is my Tor installation
working?", and should serve as a good entry point for such situations. This
page also contains a link to the main Tor documentation page.
Addition of "Tor Exit Query" page. This page allows a user to enter in an IP
to check if it matches that of one or more Tor servers, and, if it does,
checks to see if those Tor servers are capable of exiting to the destination
IP address and port that the user provides.
Addition of a DNSBL server. This allows a standard, non-web interface method
for clients to query whether or not an IP address belongs to an active Tor
server, and whether or not an IP address belongs to an active Tor server that
would allow exiting to a specified destination IP address and port. Please
see the "README_DNSBL" file in the source distribution for full setup and
usage information.
===============================================================================
}}}
===============================================================================
===============================================================================
Version 2 Changes {{{
===============================================================================
--Version 2.0--
Small performance enhancement to background update process. When clearing old
data from the descriptor and network status tables, we now use the "truncate
table" command rather than the "delete from" command. This is faster because
"truncate table" deletes all records from the table at once, while "delete
from" deletes records one row at a time.
Enhancement of "Require Flags" custom query option so that the user now has
three options available for each flag: "Off", "Yes", and "No". "Off" means
the status of the flag is not a factor in the search. "Yes" means a router
must have flag set in order to show up in result set. "No" means that a router
must not have flag set in order to show up in result set. Previously, a user
only had the equivalent of the "Off" and "Yes" options. Also, when a flag is
set to "Off", it's column header in the result list will have a blue
background, "Yes" will have a green background, and "No" will have a red
background. This further increases the level of flexibility with searches.
Addition of "Country Code" field to "Tor Server Providing Network Status
Opinion" section.
Addition of background color to drop-down lists and edit box in advanced
search section.
Minor changes to text in custom query section to facilitate clarity.
--Version 2.1--
Addition of "Aggregate Network Statistic Graphs / Details" page. This page
contains four graphs of network-wide aggregate information: Number of Routers
by Country Code, Number of Routers by Observed Bandwidth, Number of Routers
by Time Running, and an aggregate summary of various criteria including router
flags.
Addition of "EventDNS" and "Hibernating" statistics to newly renamed
"Aggregate Network Statistics Summary" section on main page.
Improvement to the way that the "Aggregate Network Statistics Summary" section
of the main page gets it's data. Previously, each piece of information used
was collected by a separate SQL query. Now, all of this has been joined into
one large query to cut out the overhead of multiple SQL calls.
Change to CSS which affects the highlight color of links which are on a white
background, for increased readability.
--Version 2.2--
Fixed a bug in the code that detects whether or not the user is connecting to
the server through the Tor network. Previously, it would tell a user they were
connecting through the Tor network if their IP address matched the IP address
of any Tor server. Now, it will only tell the user they are connecting through
the Tor network if their IP address matches the IP address of a Tor server AND
that Tor server is an exit node.
Addition of exit node only CSV file download, for those who don't want the
IP's of all Tor servers.
Addition of graph of exit routers by country code to Network Detail page. This
is useful for letting us see where we are most likely to exit the Tor network.
Addition of "BadDirectory" flag information everywhere we make use of flags.
The "BadDirectory" flag was added in Tor version 0.1.2.5-alpha.
Minor cleanups to documentation to enhance readability.
Fixed minor bug in SQL database create script.
--Version 2.3--
Redesign of the "Custom / Advanced Query Options" box on the main page. I am
now using drop-down lists for the sort request, sort order, and address mode
parameters instead of radio buttons, which takes up a lot less screen real-
estate and also makes adding new options for these parameters in the future
more practical.
Slight changes to the "Application Server Details" (Previously "Application
Server Information") box on the main page to enhance readability.
Addition of the "Number of Descriptors In Cache" field to the "Application
Server Details" box.
Movement of the "Application Server Details" box to the bottom of the page.
This has the dual purpose of improving page readability as well as making the
"Approximate Page Generation Time" figure more accurate, since it is now
calculated closer to the bottom of the page. Previously, the PHP code which
generates the "Custom / Advanced Query Options" box still ran after this time
was calculated.
Update of source URL at bottom of page to new domain.
--Version 2.4--
Modified "Aggregate Network Statistic Summary" box to also show "percent of
total network" values for each statistic listed. This allows you to easily see
what percent of Tor routers are exit nodes, what percent are mirroring the
directory, what percent of the total network your current result set makes up,
etc.
Modified "Network Status Opinion Source" box to allow clicking on the nickname
of the router to jump directly to its router detail page. This makes it easy
for a user to get detailed information about the source of the network data
that they are seeing.
Modified code that detects whether a user is accessing the site through the
Tor network or not. Now, if the user is accessing through the Tor network, the
name of their exit node is displayed in addition to the IP address. Also, the
user can click on the name of the exit node to jump directly to its router
detail page. This makes it easy for a user to get detailed information about
the exit node they are currently coming through.
Modified page footer to also include a link to the change history and license
of the application.
Removed the "$MinRouterThreshold" variable from the config file, and all code
that made use of the variable. This really became obsolete when version 1.4 of
the application was released and we started using two sets of tables in the
database for storing router information instead of one. Since that time, a
user will always have their page generated from the active set of router
tables in the database, while the next refresh operation is being performed on
the inactive set. This makes this variable, which was used to see if a
database refresh was in progress and display an error to the user back when we
used a single set of tables, obsolete.
Fixed a bug in the generation of the Comma Separated Value (CSV) list of exit
nodes. A comma was incorrectly being added after the last entry in the file,
which might have caused problems for parsers.
Added "Number of Routers by Platform" graph to network detail page. This lets
us easily see how popular different operating systems are for running Tor
routers.
Modified "Number of Routers by Time Running (Weeks)" graph on network detail
page to be full page width. This makes this graph easier to read since it
usually has a lot of items along the x axis.
--Version 2.5--
Fixed lots of minor HTML and CSS errors. All pages now validate as "CSS
version 2.1" and "HTML 4.01 Transitional".
Modified the handling of all SQL queries which return a result set so that the
result set populates into an associative array rather than a numerically
indexed array. This allows items in the result set to be referred to by column
name rather than a numerical array index. This is a significant step towards
being able to offer the user the ability to do things such as select which
columns they want displayed on the page, etc, as it is much more flexible
than using numerical indexes because your entire result set handling code is
no longer affected by re-arranging the order of items in your query, or adding
and removing items on the fly. It is also generally considered a database
programming best practice, as a lot of confusion and potential for bugs is
eliminated by being able to use easily recognizable column names as opposed to
numerical indexes.
Modified the "Address Display Mode" in the "Custom / Advanced Query Options"
so that users now have a third choice -- "Hostname + IP Address". Previously,
the user could only choose one at a time, and had no way to view both
simultaneously. "IP Address Only" mode is still the default as the "Hostname +
IP Address" mode can result in a much larger amount of data being returned,
but now the option is there for those who want it.
--Version 2.6--
Added the ability for users to select which columns they want displayed in the
main router table. This should be good for ensuring that the page does not
grow larger and larger as new informational items about Tor routers become
available, since we can just maintain reasonable default settings and leave
the obscure stuff off unless a user intentionally turns it on. Also, the SQL
query for obtaining the main router list has been made dynamic so that it will
only request columns from the database if the user will actually be displaying
them.
Added "Fingerprint" as a column display option.
Removed "Address Mode" from "Custom / Advanced Query Options" since it is made
obsolete by the users ability to select which columns they want displayed,
which offers more granularity.
Efficiency improvement in the way that it is determined if a given flag
column for a given router should display the red "x" or green check mark.
Previously, there was an "If" statement that set the appropriate CSS class for
the table data cell based on whether a flag variable for that router was '0'
or '1'. Now, there is a CSS class that ends in '0' and '1', and the variable
value is just dumped directly into the class statement in the appropriate
place. This saves up to '$NumberOfDifferentFlags X $NumberOfRoutersInResultSet'
'If' statements from being executed on each page load, normally many thousand.
Fixed a bug in the generation of CSV files that was causing a comma to be
added after the last column in a row. According to RFC 4180, this is incorrect.
Also, added a newline character (\n) to the end of files.
Added "Comma Separated Value (CSV) List of Current Result Set" option. This
lets a user get a comma separated value (CSV) file of their current search
result set, exactly as displayed on screen (Including sort preferences, column
display preferences, and any other custom query parameters that may be
applied). The first record (row) in the generated CSV file will contain the
column names for reference. This should be very useful for users wishing to
export the data from a custom query into an external data source.
--Version 2.7--
Moved the column display options to their own page, rather than having them on
the main page in the "Custom / Advanced Query Options" box. You can get to the
column display options by following the "Custom / Advanced Display Options"
link from the top of the page and then clicking on the "Column Display
Preferences" link. This should provide a much more comfortable interface for
customizing column display options.
Added the ability for users to re-configure the order that columns display in.
Previously, users could only add and remove columns. This should allow much
more flexibility to users wishing to customize the look of the page.
Updated "Comma Separated Value (CSV) List of Current Result Set" functionality
to support the newly added ability of users to re-configure column display
order. This way, the CSV file of the current result set benefits from the same
level of flexibility as the main page display.
--Version 2.8--
Added the ability for users to display four additional columns on the main
page: "Platform", "Contact", "EventDNS", & "Hibernating". Users can sort by
the new columns and, in the case of the "EventDNS" and "Hibernating" items,
filter query results based on whether they are true or false, as they are
treated as "Flags" by the application. All four of the newly added columns are
set to "Off" by default.
Updated the "Comma Separated Value (CSV) List of Current Result Set" script to
support the addition of the new columns.
Bug fix to the "Comma Separated Value (CSV) List of Current Result Set" script:
Commas (,) and double-quotes (") are now replaced with a dash (-) and a single-
quote ('), respectively, when they occur in a field to be written to the CSV
file, as these characters would cause problems for automatic parsers.
Modified the "Router Detail" page to display information on the "EventDNS" and
"Hibernating" items alongside the other flags, since it has now been
determined that the application will be treating these items like any other
flag. Previously, these two items were displayed in the "General Information"
section of the page, not the "Router Flags" section.
Modified the order in which some items display in the "Aggregate Network
Statistic Summary" section of the main page and the "Aggregate Summary --
Number of Routers Matching Specified Criteria" graph on the "Network Detail"
page to enhance consistency throughout the application.
Minor source code cleanups to enhance consistency between scripts.
--Version 2.9--
Various application-wide code cleanups to fix all known situations that would
generate PHP warnings.
Fixed a rare division by zero bug that could occur during the generation of
the bandwidth history graphs on the router detail page.
Updates to the way "Uptime" information is presented. Previously, the "Router
Detail" page would calculate the real-time uptime of the router (By looking
at the difference between the current time and when the router descriptor was
last published, and adding this value to the published "Uptime" value in the
descriptor, compensating for time zone differences), but the other places that
made use of the "Uptime" value did not, such as the main page, the "Network
Detail" page, and the "Comma Separated Value (CSV) List of Current Result Set"
functionality. These places would simply use the static "Uptime" value from
the latest descriptor, and would not perform any real-time calculations on it.
Now, all places that make use of the "Uptime" value perform the real-time
calculations on it. This eliminates discrepancies between what the "Router
Detail" page shows and what is shown everywhere else. Additionally, code to
gracefully deal with situations where a certain descriptors publish time is
obviously wrong, such as when it's in the future, was added to all places
doing the "Uptime" calculations.
Modified all places that display a "DirPort" value to say "None" rather than
"0" for routers that do not mirror the directory. This is less confusing.
Modified all places that display a "CountryCode" value to say "N/A" rather than
nothing for routers where country code information could not be retrieved.
Again, less confusing.
Made the "Tor Network Status" text at the top and bottom of each page a link
to get back to the main page. This should be more user-friendly than having to
hit "back" to get to the main page when on the "Network Detail" and "Router
Detail" pages. The links also exist on the main page itself, which will just
refresh the page with the user's current settings.
Added two new array variables to the config file to allow the default column
display options to be set on a global scale. These options are what will be
used when the user has not yet performed any customization on their display
preferences. Both the active/inactive columns and their display order can be
specified.
Changed default column display preferences -- Added "Hostname" and removed
"Bad Dir" and "Bad Exit".
Removed support for the "EventDNS" descriptor flag entirely. The logic for this
functionality was faulty, and since all new Tor versions have EventDNS enabled
by default, and the Tor developers plan to remove the old DNS logic entirely
at some time in the future (According to the "dir-spec" document), I decided
to just go ahead and remove this now since it is becoming irrelevant.
Removed "IRIX" as a listed operating system on the "Platform Graph", as it
seems there have been no Tor servers running on this OS for some time now.
Other minor fixes/enhancements.
===============================================================================
}}}
===============================================================================
===============================================================================
Version 1 Changes {{{
===============================================================================
--Version 1.0--
Initial release.
--Version 1.1--
Addition of router detail page.
Addition of real-time-calculated router uptime to router detail page.
Addition of bandwidth graphs to router detail page. This functionality makes
use of the free 'JPGraph' PHP libraries.
Addition of country code data to main and router detail pages. This
functionality makes use of the free 'GeoIP' PHP API and country database.
Separation of database refresh operation from main page render operation. The
database refresh operation has been moved out of the web root into it's own
script, 'tns_update.php'. This greatly increases page render speed in
situations where the local-cache is expired.
Addition of 'tns_agent.php' script to handle background scheduled database
refresh operations.
Movement of the SQL database creation script out of the web root.
Addition of ability to click on a column header on main page to sort. Multiple
clicks alternate sort direction.
Addition of code to handle situations where a user attempts to load a page
while a background database refresh is in progress. In these situations, the
user will be notified of the database refresh being in progress, and the page
will automatically reload in 10 seconds, retaining any sort or search criteria
that was initially passed by the user.
Various performance improvements during main page render, some major. Code
cleanups. Implementation of CSS / HTML restructuring to greatly reduce the
size of page.
--Version 1.2--
Application-wide security fixes.
Multiple XSS vulnerabilities fixed.
Possible SQL injection vulnerability fixed.
Code cleanups, further performance enhancements.
--Version 1.3--
Major changes to the way the application gets it's network status information
and router descriptors.
Previously, for network status information, the application would cycle
through the local Tor server's "cached-status" documents of Authority servers
you explicitly allowed in the application config file, using a different one
for each local-cache refresh. This had two problems. First, it meant that you
would get a different number of routers listed, and oftentimes different
properties listed for a specific router depending on which Authority server's
cached-status doc had been used. Second, and this is the bigger of the two
problems, is that this is not the way the Tor network works. The Tor network
leaves it up to each client/server to calculate what it believes to be the
correct view of the network based on the multiple sources of network status
information it has available. This is important for security reasons, and
plays a significant role in making it harder for someone to play some bad
tricks on network users. So, now network status information is obtained by
asking the local Tor server's control port to list all routers it has an
opinion about. This ensures the data being presented has been through the
proper "voting" process done by the Tor software, and is what the local Tor
server believes to be correct.
Previously, for individual router descriptors, the application would do one
of two things. One, the default, was to directly ask whichever Authority
server was used for the last local-cache refresh. This was bad, as it
basically meant that for the duration of the current cache life, every
visitor going to the "router_detail" page was causing a request for a
descriptor to be sent to that Authority server. Not what we want for the
Authority servers, I'm sure. Two, a user could specify that they always wanted
to use a certain Tor server for descriptor requests. This fixed the issue with
hammering the Authority servers, but only if people used the option. Now,
router descriptors are also obtained through the local Tor server's control
port. Also, router descriptors are no longer written to files on the hard
drive. This was unnecessary and slow.
These changes allowed a lot of clean-up to be done in the config file. There
are a lot less variables in there now.
--Version 1.4--
Addition of an option to export the IP Addresses of all current Tor routers to
a CSV (Comma Separated Value) file. This may be of some use for people looking
to get a quick, current list of Tor router IP's, whether to formulate block
lists, or to assist with other Tor network status apps / pages.
Addition of "Bandwidth" and "Uptime" columns to main page. The bandwidth
reading is obtained from the "Bandwidth Observed" parameter of the router
descriptor and is displayed in KB/s. The uptime reading is obtained from the
"Uptime" parameter of the router descriptor and is displayed in days. Both of
these columns are sortable.
Back end changes to make the addition of the "Bandwidth" and "Uptime" columns
to the main page, described above, possible. These changes consisted of making
the scheduled background update process request and parse the descriptors of
every router that the local Tor server has an opinion about, as identified by
it's network status request response, so that the bandwidth and uptime
information (and everything else in the descriptor) could be inserted into the
database at this stage to be available for use on the main page. Previously,
descriptors were only requested and parsed on an as-needed basis when a user
loaded the "router_detail.php" page, and the data parsed from them was not
stored. Now, it's all inserted into the database during the main update
process which opens up the possibilities for all kinds on new stuff. The
downside to doing it this way is that the main update process CAN take
considerably longer (especially when the local Tor server is very busy), which
led to the second major back end change: Using two router (local-cache) tables
instead of one. Basically, only one table is marked as active at any given
time, and all page loads use the active table. When the update process runs,
it runs on the non-active table, and marks it as the active table upon
successful completion. This prevents users from seeing the "Database refresh in
progress" error message, as they will instead just be seeing the old data
until the new data is ready. I was quite serious about not causing a
performance hit to page load times due to the new functionality, and this was
the solution implemented to avoid that.
--Version 1.5--
Major performance enhancements to both the background update process and the
main page load process.
First, for the update process, we are now obtaining the full network status
opinion document and the full recent descriptor document from the control port
of the Tor server. Previously, we were obtaining the full network status
opinion document, and then, as we parsed out this file, we requested server
descriptors of the Tor server's control port individually. This tended to
sometimes cause somewhat of a load on the Tor server and cause the descriptors
to be fed back very slowly, sometimes taking up to a half hour for a full
refresh. Now, since we grab both full documents all at once, it only takes the
Tor server a few seconds to provide them. Second, we are now writing the
parsed network status information and the parsed descriptor information into
two separate database tables (NetworkStatus & Descriptor). This makes for
cleaner design and tables with less columns. And, in keeping with the spirit
of the updates made in Version 1.4, there is two of each of these tables
(NetworkStatus & Descriptor) so that one set can be updated while the other
set can continue to be used by folks loading the page and running queries,
etc., until the new set is ready.
Performance improvements to the load time of the main page were obtained by
switching to the MySQL MyISAM storage engine, instead of the InnoDB engine we
had been using previously. MyISAM is significantly faster than InnoDB, and,
since we currently don't make use of transactions, there is no reason not to
use it. Also, indexes were added to the NetworkStatus and Descriptor tables
on the "Fingerprint" column, since this is what is used for the join of these
two tables during page load. This further increased the query speed.
Some other small performance improvements were implemented, especially on the
"router_detail.php" page.
The update process now has more robust error handling.
Addition of "Count" column to main page.
Addition of green check marks and red x's to indicate whether a flag is "Yes"
or "No". Previously, it would just print "Yes" or "No".
Modifications of font selections and background images to hopefully achieve
two things: Fit more in a smaller space, and make the page easier to read.
--Version 1.6--
The application now performs reverse-DNS lookups during the update process and
stores this information in the database. This enables a user to choose whether
they would like addresses displayed as IP or Hostname. Of course, whichever
display they choose, it is sortable.
Added Hostname information to the router detail page as well.
Added "Last Update Elapsed Time" field to main page. Good way to see if your
system is experiencing problems during the background updates.
Added "Current Descriptor Signature" field to the main page for the Network
Status Source router information section.
Minor CSS changes to improve layout and formatting in certain situations.
--Version 1.7--
Changes to the way variable handling is done. I moved all state maintaining
variables into PHP session, such as all of the "Flags" variables and the
Address Display Mode, SortRequest, and SortOrder variables. Also, wherever
possible because of addition to session, variables were removed from the URL's
in the anchor tags of the generated page. The only ones that were left in
URL's were variables that need to be manipulated by clicking on a link, namely
the SortRequest and SortOrder variables. This makes the URL's in the anchor
tags of the generated page much shorter and cleaner, since a whole lot less is
being passed around via GET. This in turn reduces the generated page size,
sometimes considerably depending on the custom query options in effect.
Another nice benefit to doing things this way is that if a user has custom
options set, completely browses away from the site, and later comes back
within their same browser session, all of their options will be configured as
they were when they left.
Also, variables that needed to remain in URL's have shorter names to further
reduce the amount of data being passed around via GET, and hence the length
of the URL's in the generated page.
Further cleanups to HTML formatting and CSS on both the main page and the
router detail page.
Addition of timer to bottom of main page to indicate how long page generation
took on the server.
--Version 1.8--
Changes / cleanup to CSS. All CSS information is now defined in one external
style sheet which resides in the "web/css" directory, and the different parts
of the code that need the style sheet simply reference it rather than defining
everything at the top of every HTML page output to the browser.
Addition of row highlighting for Mozilla Firefox / Opera. This makes it easier
to visually track which row you are currently moused over. This feature does
not currently work in Internet Explorer since that would require JavaScript,
and I'm trying to keep this app JavaScript free for those who browse with it
disabled.
When sorting by IP Addresses, they are now sorted naturally, rather than as
strings. This should make the IP Address search quite a bit more meaningful.
This change also applies to the CSV (Comma Separated Value) file list of IP's.
--Version 1.9--
Added proper escaping of problem characters to all strings that get populated
from data parsed from router descriptors and network status documents before
database inserts are performed. This should prevent the update process from
dying when these characters are encountered, which would sometimes happen.
Now using the standard "gethostbyaddr" PHP function to do hostname lookups.
Previously, I was doing some custom parsing on the output of the Unix "host"
command, as I thought it would be faster, but, after comparing the two, I see
no speed degradation with using "gethostbyaddr". The advantage of using
"gethostbyaddr" over "host" is that we are not using a platform-specific
command which limits where the application can run without modification.
Addition of Advanced Search functionality, which lets you perform free-text
searches on the following data: Fingerprint, Router Name, Country Code,
Bandwidth, Uptime, Last Descriptor Published, IP Address, Hostname, Onion
Router Port, Directory Server Port, Platform, & Contact. You can apply the
following modifiers when searching the above data: Equals, Contains, Is Less
Than, or Is Greater Than. The Advanced Search works in conjunction with the
other Custom Query options (Sort By, Sort Order, Address Display Mode, and
Require Flags), allowing a great deal of flexibility to customize the result
set as desired.
Addition of indexes where necessary in the database to better facilitate
speedy free-text searches.
Addition of "Hostname" field to "Tor Server Providing Network Status Opinion"
section.
Addition of feature to tell user if they are accessing the page through the
Tor network or not, color coded.
Additional escaping of potentially troublesome HTML characters before
displaying on page for "Contact" field on main page and router detail page.
===============================================================================
}}}
===============================================================================
NOTE: This product includes GeoLite data created by MaxMind, available from
http://www.maxmind.com/.