$Id$ Legend: SPEC!! - Not specified SPEC - Spec not finalized N - nick claims R - arma claims P - phobos claims S - Steven claims E - Matt claims M - Mike claims J - Jeff claims I - ioerror claims W - weasel claims K - Karsten claims - Not done * Top priority . Partially done o Done d Deferrable D Deferred X Abandoned ======================================================================= External constraints: - mid July W - Take the results from instrumenting directory downloads on Tor clients, and analyze/simulate some alternate approaches. Finish proposal for how to improve things, iterate based on feedback, convince us that the anonymity tradeoffs and/or scalability tradeoffs are acceptable. - mid August KS - Design hidden service improvements, evaluate them and consider security properties: write some proposals, get feedback, revise them, etc. ? - nlnet 'user safety contest'. submit torbrowser, others? - end of August I - Auto update o Vidalia learns when Tor thinks it should be updated R - Tor status events should suggest a new version to switch to I - Figure out a good PKI, document the design, assess security issues: "write a proposal" - Vidalia fetches the new one via Tor when possible, but fetches it without Tor "when necessary", whatever that means. - Give an interface for notifying the user, and letting her decide to fetch and decide to swap out the old Tor for the new. - Do the same for Polipo - and for Vidalia itself - end of September NSE - Write first draft of research study for Paul's research problem. This should be at least vaguely related to what was discussed in the end-of-May deliverable. - mid October KS - Finish implementation of hidden service improvements: have a set of patches that you think work. W - Finish implementation of directory overhead changes: have a set of patches that you think work. - mid January KS - Finish testing, debugging, unit testing, etc the hidden service changes. Have it in the development version and in use. W - Finish testing, debugging, unit testing, etc the directory overhead changes. Have it in the development version and in use. ======================================================================= Other things Roger would be excited to see: Nick - Finish buffer stuff in libevent; start using it in Tor. - Tors start believing the contents of NETINFO cells. . Work with Steven and Roger to decide which parts of Paul's project he wants to work on. - respond to Steven's red-team TLS testing (a.k.a, look at a packet dump and compare) Matt - Fit Vidalia in 640x480 again. - When user changes the language in Vidalia, have it change right then. - Vidalia should display/edit PlaintextPorts events/config. . Vidalia's GUI should let you specify an http proxy that it launches for you. Maybe in the general config window next to which Tor it launches for you. - Vidalia should avoid stomping on your custom exit policy lines just because you click on 'save' for a totally different config thing. - How much space do we save in TBB by stripping symbols from Vidalia first? Good idea or crazy idea? ioerror - gmail auto responder so you send us an email and we send you a Tor binary. Probably needs a proposal first. - weather.torproject.org should go live. o Learn from Steven how to build/maintain the Tor Browser Bundle. - Keep advocating new Tor servers and working with orgs like Mozilla to let them like Tor. - Start converting critical wiki pages into real Tor wml pages. E.g., https://wiki.torproject.org/noreply/TheOnionRouter/VerifyingSignatures - Find out what happened to the buildbot and get it back up: http://tor-buildbot.freehaven.net:8010/ - Learn about locking memory pages that have sensitive content. Get that started in Tor. - Translation portal - Vidalia html help files - should we i18nize polipo's error messages too? - Some of our translated wml files are very old -- so old that they are harmful to leave in place. We need some sort of way to notice this and disable them. Steven - Figure out (or give up on) how to run Tor Browser and ordinary Firefox side-by-side. - Enumerate and analyze traces left when running from USB - Write a list of research items Tor would like to see done, for the volunteer page. Pick a few you'd like to work on yourself. - Move proposal 131 or equivalent forward. - Keep bugging us about exploits on the .exit notation. - If relays have 100KB/s but set relaybandwidthrate to 10KB/s, do your interference attacks still work? - Mike's question #3 on https://www.torproject.org/volunteer#Research - Worthwhile shipping TBB with some local html help files that come as bookmarks? - Decide whether TBB should use Torbutton's "lock" feature. http://archives.seul.org/or/cvs/Jun-2008/msg00186.html Andrew - Which bundles include Torbutton? Change the docs/tor-doc-foo pages so they admit that Torbutton is in them too. Change the download page too. - The OS X bundle screenshots are from forever ago -- they don't include Torbutton, they still say it's tor.eff.org, etc. - Should we still be telling you how to use Safari on OS X for Tor, given all the holes that Torbutton-dev solves on Firefox? Karsten o Make a hidden services explanation page with the hidden service diagrams. See img/THS-[1-6].png. These need some text to go along with them though, so people can follow what's going on. - We should consider a single config option TorPrivateNetwork that turns on all the config options for running a private test tor network. having to keep updating all the tools, and the docs, just isn't working. Weasel - Figure out how to make Vidalia and Tor play nicely on Debian, make the necessary modifications, and make some Vidalia debs that pass muster. - Fix bug 393. - Get oftc to switch to Tor dns bulk exitlist. Or tell us why it's not suitable yet. - Take non-Running entries out of the networkstatus consensus. - Move proposal 134 forward. - putting port predictions in state file - if tor hasn't been used in a while it stops fetching consensus documents. Retain that state over restarts. Roger - Finish tor-doc-bridge.wml . Fix FAQ entry on setting up private Tor network - Review Karsten's hidden service diagrams - Roger should visit Internews DC sometime. - Did we actually apply Steven's dkimproxy patch? - Brainstorm about safe but effective ways for vidalia to auto-update its user's bridges via Tor in the background. Mike: - Roger wants to get an email every time there's a blog change, e.g. a comment. That way spam doesn't go undetected for weeks. - Or, maybe just disable linking from blog comments entirely? ======================================================================= Bugs/issues for Tor 0.2.0.x: . we should have an off-by-default way for relays to dump geoip data to a file in their data directory, for measurement purposes. o Basic implementation N - Include probability-of-selection R d let bridges set relaybandwidthrate as low as 5kb R - bridge communities . spec . deploy - man page entries for Alternate*Authority config options Documentation for Tor 0.2.0.x: - Proposals: . 111: Prioritize local traffic over relayed. R - Merge into tor-spec.txt. - 113: mark as closed close. o document the "3/4 and 7/8" business in the clients fetching consensus documents timeline. R - then document the bridge user download timeline. - HOWTO for DNSPort. See tup's wiki page. . Document transport and natdport in a good HOWTO. - Quietly document NT Service options: revise (or create) FAQ entry ======================================================================= For 0.2.1.2-alpha: R d bug: if we launch using bridges, and then stop using bridges, we still have our bridges in our entryguards section, and may use them. R d add an event to report geoip summaries to vidalia for bridge relays, so vidalia can say "recent activity (1-8 users) from sa". R - investigate: it looks like if the bridge authority is unreachable, we're not falling back on querying bridges directly? R - if "no running bridges known", an application request should make us retry all our bridges. R - get matt to make vidalia do a getinfo status/bootstrap-phase to get caught up after it connects. R d Setting DirPort when acting as bridge will give false Warnings For 0.2.1.x: - Proposals to do: o 110: avoid infinite-length circuits R d 128: families of private bridges - 134: handle authority fragmentation. - Proposals to write: R d Do we want to maintain our own set of entryguards that we use as next hop after the bridge? X Add an 'exit-address' line in the descriptor for servers that exit from something that isn't their published address. [I think tordnsel solved this. -RD] - Proposal to supersede 117 by adding IPv6 support for exits and entries. - Internal code support for ipv6: o Clone ipv6 functions (inet_ntop, inet_pton) where they don't exist. - Many address variables need to become tor_addr_t - addr in connection_t - n_addr in extend_info_t - Teach resolving code how to handle ipv6. . Teach exit policies about ipv6 (consider ipv4/ipv6 interaction!) - Generate END_REASON_EXITPOLICY cells and parse them right - Generate new BEGIN cell types and parse them right - 118: Listen on and advertise multiple ports: - Tor should be able to have a pool of outgoing IP addresses that it is able to rotate through. (maybe. Possible overlap with proposal 118.) - config option to publish what ports you listen on, beyond ORPort/DirPort. It should support ranges and bit prefixes (?) too. - Need to figure out the right format for routerinfo_t on this. - Fix voting to handle bug 608 case when multiple servers get Named. d Possibly: revise link protocol to allow big circuit IDs, variable-length cells, proposal-110 stuff, and versioned CREATES? o Eliminate use of v2 networkstatus documents in v3 authority decision-making. N . Draft proposal for GeoIP aggregation (see external constraints *) o Separate Guard flags for "pick this as a new guard" and "keep this as an existing guard". First investigate if we want this. . Figure out how to make good use of the fallback consensus file. Right now many of the addresses in the fallback consensus will be stale, so it will take dozens of minutes to bootstrap from it. This is a bad first Tor experience. But if we check the fallback consensus file *after* we fail to connect to any authorities, then it may still be valuable as a blocking-resistance step. o Write the proposal. - Patch our tor.spec rpm package so it knows where to put the fallback consensus file. d Something for bug 469, to limit connections per IP. . Put bandwidth weights in the networkstatus? So clients get weight their choices even before they have the descriptors; and so authorities can put in more accurate numbers in the future. d Fetch an updated geoip file from the directory authorities. - Tiny designs to write: . Better estimate of clock skew; has anonymity implications. Clients should estimate their skew as median of skew from servers over last N seconds, but for servers this is not so easy, since a server does not choose who it connects to. - Do TLS connection rotation more often than "once a week" in the extra-stable case. (One reason not to do it more often is because the old TLS conn probably has a circuit on it, and we don't really want to build up dozens of TCP connections to all the other extra-stable relays.) - If a relay publishes a new descriptor with a significantly lower uptime or with a new IP address, then we should consider its current "running" interval to have ended even if it hadn't yet failed its third reachability test. the interval ended when the new descriptor appeared, and a new interval began then too. - Use less RAM * - Optimize cell pool allocation. d Support (or just always use) jemalloc (if it helps) - mmap more files. - Look into pulling serverdescs off buffers as they arrive. - Use less bandwidth - Use if-modified-since to download consensuses - Handle multi-core cpus better - Use information from NETINFO cells - Don't extend a circuit over a noncanonical connection with mismatched address. - Learn our outgoing IP address from netinfo cells? - Learn skew from netinfo cells? - Testing - Better unit test coverage - Refactor unit tests into multiple files - Verify that write limits to linked connections work. - Use more mid-level and high-level libevent APIs - For dns? - For http? - For buffers? - Tool improvements: - Get IOCP patch into libevent * - Security improvements - make is-consensus-fresh-enough check way tighter. - If we haven't tried downloading a consensus for ages since we're tired, try getting a new one before we use old descriptors for a circuit. Related to bug 401. - Feature removals and deprecations: - Get rid of the v1 directory stuff (making, serving, and caching) - First verify that the caches won't flip out? - If they will, just stop the caches from caching for now - perhaps replace it with a "this is a tor server" stock webpage. - The v2dir flag isn't used for anything anymore, right? If so, dump it. - Even clients run rep_hist_load_mtbf_data(). Does this waste memory? Dump it? - Unless we start using ftime functions, dump them. - can we deprecate 'getinfo network-status'? - can we deprecate the FastFirstHopPK config option? - Can we deprecate controllers that don't use both features? Nice to have for 0.2.1.x: - Proposals to write - steven's plan for replacing check.torproject.org with a built-in answer by tor itself. - Documentation P - Make documentation realize that location of system configuration file will depend on location of system defaults, and isn't always /etc/torrc. - Small controller features - A status event for when tor decides to stop fetching directory info if the client hasn't clicked recently: then make the onion change too. - Add a status event when new consensus arrives - Windows build P - Figure out why dll's compiled in mingw don't work right in WinXP. P - create a "make win32-bundle" for vidalia-privoxy-tor-torbutton bundle - Refactor bad code: - Refactor the HTTP logic so the functions aren't so large. - Refactor buf_read and buf_write to have sensible ways to return error codes after partial writes - Router_choose_random_node() has a big pile of args. make it "flags". - Streamline how we pick entry nodes: Make choose_random_entry() have less magic and less control logic. - Don't call time(NULL) so much; instead have a static time_t field that gets updated only a handful of times per second. - Move all status info out of routerinfo into local_routerstatus. Make "who can change what" in local_routerstatus explicit. Make local_routerstatus (or equivalent) subsume all places to go for "what router is this?" - deprecate router_digest_is_trusted_dir() in favor of router_get_trusteddirserver_by_digest() - Make Tor able to chroot itself o allow it to load an entire config file from control interface - document LOADCONF - log rotation (and FD passing) via control interface - chroot yourself, including inhibit trying to read config file and reopen logs, unless they are under datadir. - Should be trivial: - Base relative control socket paths (and other stuff in torrc) on datadir. - Tor logs the libevent version on startup, for debugging purposes. This is great. But it does this before configuring the logs, so it only goes to stdout and is then lost. - Make TrackHostExits expire TrackHostExitsExpire seconds after their *last* use, not their *first* use. - enforce a lower limit on MaxCircuitDirtiness and CircuitBuildTimeout. - Make 'safelogging' extend to info-level logs too. - don't do dns hijacking tests if we're reject *:* exit policy? (deferred until 0.1.1.x is less common) - More consistent error checking in router_parse_entry_from_string(). I can say "banana" as my bandwidthcapacity, and it won't even squeak. - Interface for letting SOAT modify flags that authorities assign. (How to keep the authority from clobbering them afterwards? Later, unless people want to implement them now: - Actually use SSL_shutdown to close our TLS connections. - Include "v" line in networkstatus getinfo values. [Nick: bridge authorities output a networkstatus that is missing version numbers. This is inconvenient if we want to make sure bridgedb gives out bridges with certain characteristics. -RD] [Okay. Is this a separate item, or is it the same issue as the lack of a "v" line in response to the controller GETINFO command? -NM] - Let tor dir mirrors proxy connections to the tor download site, so if you know a bridge you can fetch the tor software. - when somebody uses the controlport as an http proxy, give them a "tor isn't an http proxy" error too like we do for the socks port. - MAYBE kill stalled circuits rather than stalled connections. This is possible thanks to cell queues, but we need to consider the anonymity implications. - Make resolves no longer use edge_connection_t unless they are actually _on_ a socks connection: have edge_connection_t and (say) dns_request_t both extend an edge_stream_t, and have p_streams and n_streams both be linked lists of edge_stream_t. - Generate torrc.{complete|sample}.in, tor.1.in, the HTML manual, and the online config documentation from a single source. - It would be potentially helpful to respond to https requests on the OR port by acting like an HTTPS server. - Make the timestamp granularity on logs configurable, with default of "1 second". This might make some kinds of after-the-fact attack harder. Can anybody remember why we wanted to do this and/or what it means? - config option __ControllerLimit that hangs up if there are a limit of controller connections already. [This was mwenge's idea. The idea is that a Tor controller can "fill" Tor's controller slot quota, so jerks can't do cross-protocol attacks like the http form attack. -RD] - Bridge issues . Ask all directory questions to bridge via BEGIN_DIR. - use the bridges for dir fetches even when our dirport is open. - drop 'authority' queries if they're to our own identity key; accept them otherwise. - give extend_info_t a router_purpose again If somebody wants to do this in some version, they should: - Create packages for Nokia 800, requested by Chris Soghoian - More work on AvoidDiskWrites - Make DNSPort support TCP DNS. * * * * Roger, please sort these: * * * * - bridge communities with local bridge authorities: - clients who have a password configured decide to ask their bridge authority for a networkstatus - be able to have bridges that aren't in your torrc. save them in state file, etc. - Consider if we can solve: the Tor client doesn't know what flags its bridge has (since it only gets the descriptor), so it can't make decisions based on Fast or Stable. - Some mechanism for specifying that we want to stop using a cached bridge. ======================================================================= Future versions: - Protocol - Our current approach to block attempts to use Tor as a single-hop proxy is pretty lame; we should get a better one. - Allow small cells and large cells on the same network? - Cell buffering and resending. This will allow us to handle broken circuits as long as the endpoints don't break, plus will allow connection (tls session key) rotation. - Implement Morphmix, so we can compare its behavior, complexity, etc. But see paper breaking morphmix. - Other transport. HTTP, udp, rdp, airhook, etc. May have to do our own link crypto, unless we can bully DTLS into it. - Need a relay teardown cell, separate from one-way ends. (Pending a user who needs this) - Handle half-open connections: right now we don't support all TCP streams, at least according to the protocol. But we handle all that we've seen in the wild. (Pending a user who needs this) - Directory system - BEGIN_DIR items - handle connect-dir streams that don't have a chosen_exit_name set. - Have a "Faster" status flag that means it. Fast2, Fast4, Fast8? - Add an option (related to AvoidDiskWrites) to disable directory caching. (Is this actually a good idea??) X Add d64 and fp64 along-side d and fp so people can paste status entries into a url. since + is a valid base64 char, only allow one at a time. Consider adding to controller as well. [abandoned for lack of demand] - Some back-out mechanism for auto-approval on authorities - a way of rolling back approvals to before a timestamp - Consider minion-like fingerprint file/log combination. X Have new people be in limbo and need to demonstrate usefulness before we approve them. - Hidden services: d Standby/hotswap/redundant hidden services: needs a proposal. - you can insert a hidserv descriptor via the controller. - auth mechanisms to let hidden service midpoint and responder filter connection requests: proposal 121. - Let each hidden service (or other thing) specify its own OutboundBindAddress? - Server operation - If the server is spewing complaints about raising your ulimit -n, we should add a note about this to the server descriptor so other people can notice too. - When we hit a funny error from a dir request (eg 403 forbidden), but tor is working and happy otherwise, and we haven't seen many such errors recently, then don't warn about it. - Controller - Implement missing status events and accompanying getinfos - DIR_REACHABLE - BAD_DIR_RESPONSE (Unexpected directory response; maybe we're behind a firewall.) - BAD_PROXY (Bad http or https proxy) - UNRECOGNIZED_ROUTER (a nickname we asked for is unavailable) - Status events related to hibernation - something about failing to parse our address? from resolve_my_address() in config.c - sketchy OS, sketchy threading - too many onions queued: threading problems or slow CPU? - Implement missing status event fields: - TIMEOUT on CHECKING_REACHABILITY - GETINFO status/client, status/server, status/general: There should be some way to learn which status events are currently "in effect." We should specify which these are, what format they appear in, and so on. - More information in events: - Include bandwidth breakdown by conn->type in BW events. - Change circuit status events to give more details, like purpose, whether they're internal, when they become dirty, when they become too dirty for further circuits, etc. - Change stream status events analogously. - Expose more information via getinfo: - import and export rendezvous descriptors - Review all static fields for additional candidates - Allow EXTENDCIRCUIT to unknown server. - We need some way to adjust server status, and to tell tor not to download directories/network-status, and a way to force a download. - Make everything work with hidden services - Performance/resources - per-conn write buckets - separate config options for read vs write limiting (It's hard to support read > write, since we need better congestion control to avoid overfull buffers there. So, defer the whole thing.) - Rate limit exit connections to a given destination -- this helps us play nice with websites when Tor users want to crawl them; it also introduces DoS opportunities. - Consider truncating rather than destroying failed circuits, in order to save the effort of restarting. There are security issues here that need thinking, though. - Handle full buffers without totally borking - Rate-limit OR and directory connections overall and per-IP and maybe per subnet. - Misc - Hold-open-until-flushed now works by accident; it should work by design. - Display the reasons in 'destroy' and 'truncated' cells under some circumstances? - Make router_is_general_exit() a bit smarter once we're sure what it's for. - Automatically determine what ports are reachable and start using those, if circuits aren't working and it's a pattern we recognize ("port 443 worked once and port 9001 keeps not working"). - Security - some better fix for bug #516? - Directory guards - Mini-SoaT: - Servers might check certs for known-good ssl websites, and if they come back self-signed, declare themselves to be non-exits. Similar to how we test for broken/evil dns now. - Authorities should try using exits for http to connect to some URLS (specified in a configuration file, so as not to make the List Of Things Not To Censor completely obvious) and ask them for results. Exits that don't give good answers should have the BadExit flag set. - Alternatively, authorities should be able to import opinions from Snakes on a Tor. - Bind to random port when making outgoing connections to Tor servers, to reduce remote sniping attacks. - Audit everything to make sure rend and intro points are just as likely to be us as not. - Do something to prevent spurious EXTEND cells from making middleman nodes connect all over. Rate-limit failed connections, perhaps? - DoS protection: TLS puzzles, public key ops, bandwidth exhaustion. - Needs thinking - Now that we're avoiding exits when picking non-exit positions, we need to consider how to pick nodes for internal circuits. If we avoid exits for all positions, we skew the load balancing. If we accept exits for all positions, we leak whether it's an internal circuit at every step. If we accept exits only at the last hop, we reintroduce Lasse's attacks from the Oakland paper. - Windows server usability - Solve the ENOBUFS problem. - make tor's use of openssl operate on buffers rather than sockets, so we can make use of libevent's buffer paradigm once it has one. - make tor's use of libevent tolerate either the socket or the buffer paradigm; includes unifying the functions in connect.c. - We need a getrlimit equivalent on Windows so we can reserve some file descriptors for saving files, etc. Otherwise we'll trigger asserts when we're out of file descriptors and crash. - Documentation - a way to generate the website diagrams from source, so we can translate them as utf-8 text rather than with gimp. (svg? or imagemagick?) . Flesh out options_description array in src/or/config.c . multiple sample torrc files - Refactor tor man page to divide generally useful options from less useful ones? - Add a doxygen style checker to make check-spaces so nick doesn't drift too far from arma's undocumented styleguide. Also, document that styleguide in HACKING. (See r9634 for example.) - exactly one space at beginning and at end of comments, except i guess when there's line-length pressure. - if we refer to a function name, put a () after it. - only write foo when foo is an argument to this function. - doxygen comments must always end in some form of punctuation. - capitalize the first sentence in the doxygen comment, except when you shouldn't. - avoid spelling errors and incorrect comments. ;) - Packaging - The Debian package now uses --verify-config when (re)starting, to distinguish configuration errors from other errors. Perhaps the RPM and other startup scripts should too? - add a "default.action" file to the tor/vidalia bundle so we can fix the https thing in the default configuration: http://wiki.noreply.org/noreply/TheOnionRouter/TorFAQ#PrivoxyWeirdSSLPort ======================================================================= Documentation, non-version-specific. - Specs - Mark up spec; note unclear points about servers NR - write a spec appendix for 'being nice with tor' - Specify the keys and key rotation schedules and stuff . Finish path-spec.txt - Mention controller libs someplace. - Remove need for HACKING file. - document http://wiki.noreply.org/noreply/TheOnionRouter/TransparentProxy on freebsd and osx P - figure out why x86_64 won't build rpms from tor.spec P - figure out rpm spec files for bundles of vidalia-tor-polipo P - figure out polipo install scripts for bundles of vidalia-tor-polipo on osx, win32 - figure out selinux policy for tor P - change packaging system to more automated and specific for each platform, suggested by Paul Wouter P - Setup repos for redhat and suse rpms & start signing the rpms the way package management apps prefer Website: J . tor-in-the-media page P - Figure out licenses for website material. (Phobos reccomends the Open Publication License with Option A at http://opencontent.org/openpub/) P - put the logo on the website, in source form, so people can put it on stickers directly, etc. P - put the source image for the stickers on the website, so people can print their own P - figure out a license for the logos and docs we publish (trademark figures into this) (Phobos reccomends the Open Publication License with Option A at http://opencontent.org/openpub/) P - ask Jan/Jens to be the translation coordinator? add to volunteer page. I - add a page for localizing all tor's components. - It would be neat if we had a single place that described _all_ the tor-related tools you can use, and what they give you, and how well they work. Right now, we don't give a lot of guidance wrt torbutton/foxproxy/privoxy/polipo in any consistent place. P - create a 'blog badge' for tor fans to link to and feature on their blogs. A sample is at http://interloper.org/tmp/tor/tor-button.png - More prominently, we should have a recommended apps list. - recommend pidgin (gaim is renamed) - unrecommend IE because of ftp:// bug. - Addenda to tor-design - we should add a preamble to tor-design saying it's out of date. - we should add an appendix or errata on what's changed. - Tor mirrors - make a mailing list with the mirror operators o make an automated tool to check /project/trace/ at mirrors to learn which ones are lagging behind. - auto (or manually) cull the mirrors that are broken; and contact their operator? - a set of instructions for mirror operators to make their apaches serve our charsets correctly, and bonus points for language negotiation. - figure out how to load-balance the downloads across mirrors? - ponder how to get users to learn that they should google for "tor mirrors" if the main site is blocked. - find a mirror volunteer to coordinate all of this