|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things|
|We maintain a number of public pages that contain graphs, rates, statistics, and other analyses of combined work being done by the collective cooperation of all participants. This section discusses some of the most common questions regarding this information.
Technical info about statsbox
Client speeds database pages.
Answers in this category:
A tutorial on statistics is also available to guide you through our stats system.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Teams|
Answers in this category:
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Teams : I can't figure out how to join a team!|
Before you can join a team, you must first have your password (see How come I haven't received my password?). Once you've gotten your password, and you've successfully edited your participant information, then you'll be in a position to join a team.
The simplest way is to simply go to your team's stats summary page and click the "I want to join this team" link. You'll be presented with a login dialog box where you should enter your email address and your password.
That's all there is to it!
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Teams : I created a team, but it never showed up like it was supposed to! Obviously, I can't join the team|
Perhaps there is a "team creation" faq out there that gives erroneous instructions, because this question is actually pretty common. When you created the team, it was explained pretty clearly so I'll simply repeat here in the FAQ what you read when the team was created:
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Teams : I administer a team, how do I let my team members see who is on my team? Can I keep them from seeing who else is on my team?|
|To set a password or to make your team lists public, you need to edit you team information. There you can edit the properties of the team member list. To restrict access, you would set it to password or completely disable your team list.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Teams : How do I un-affiliate myself from my former team?|
|You have to go into your personal preferences page via the "Edit your information" link and click on the "If you do not wish to be on a team, click here." link next to your current team affiliation.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Teams : I administer a team, and I would like to donate my team's blocks to another team. Can this be completed?|
|No. Once blocks are allocated to a team, they cannot be allocated to a different team. Blocks completed in the future can be set to a different team. This would be completed by re-setting the team affiliation of your team member's emails.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Teams : I am a member of a team, how do I check who else is on my team?|
|If you are currently on a team that allows public viewing of team members, you can click on the link below the team-specific status entitled "Click here to view this team's participant stats for yesterday or overall." If your team list is private, you will need a password for your team list. Contact your team administrator for the password needed to view your team member list. When entering your password needed to view the team lists, please note that it is not case-sensitive.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Teams : I have already checked and uploaded a couple hundred blocks. I'm not on a team. I'm wondering: will these will be credited to the team I sign up for, or have they become the domain of distributed.net?|
|All blocks completed previous to your joining a team will become permanent property of the team you join first. If you wish to change teams, you may, however, all blocks will be owned by the team you were working for while completing them.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Teams : You should add a link to everyone's summary page that shows what team they are on|
This is a commonly-requested feature, but it's our view that a person's team membership is private and to display that on stats would be a violation of participants' privacy.
Imagine a controversial team, like a pro-life team or a drug-legalization team. It's concievable that a person might wish to support that team by being a member yet wouldn't want to advertise their support or make it known that they are on that team.
Or, imagine being a participant in the top 10 who is not on any team at all. That person would (due to the suspicious lack of a team affiliation line on their summary) likely be deluged in requests to join people's teams. "dood! you rool! join team luser!" We'd prefer to allow people to participate individually in peace and privacy.
To offset this, please feel free to mention your team affiliation in your participant motto, which shows up on your summary page. The motto allows html, so you can even embed a link if you wish. We feel that this compromise is the best of both situations.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Teams : Can you tell me more about teams?|
To make things more interesting, some people have decided to work together in groups (rather than individually) and thus increase interest and participation. It is important to note that this is not about one team versus another, or about who actually cracks the key. This is an effort to demonstrate that a collection of amateur internet enthusiasts can break strong 64-bit encryption using only their existing computing resources.
Teams are optional! They are fun and the rivalries between them help to increase project participation but you are not required to join a team if you don't want to. Also there are no official teams for anything. Some people might claim that their team is the official team for an operating system or group, but you are not required to join it. You can simply run for yourself, or run for others if you wish.
For those of you who participated in the Bovine RC5-56 project you should be aware that teams are being handled differently this time around. Previously a team was identified by the email address of the team coordinator. To join you had to submit blocks using that address. This time around we have set up a true team structure. Individual contributors should report all checked blocks under their OWN email addresses. They can then choose to affiliate themselves with a particular team. If they do so their individual stats will continue to report their total effort but their contribution will also be counted in their team's total.
If you want to create a new team, simply visit http://stats.distributed.net/newteam1.php3 Be sure to carefully read the notice that appears. You only need to register if your team doesn't already exist. If you really mean to join a pre-existing team proceed as follows. First, get the ID number of the team you wish to join (this can be done using the Search for Team facility on the stats page). Then, from your individual stats summary page, you can affiliate yourself with your team by choosing the Edit Participant Information button. Note that this last step requires a password. If you don't already have one you can get one automatically mailed to you from the link at the bottom of your individual stats summary page.
One other note about joining teams. If you've never joined a team before
then ALL the blocks you have checked will go to bolster your team's stats
when you join. If you do already belong to a team and then switch to a
different one only new blocks that you check will be attributed to your
new team. Old blocks stay with your old team. In either case your individual
stats always show your total contribution since starting the effort regardless
of whether or not you have joined a team or switched teams.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Teams : How come a team just went up several places in the overall ranking, but did less work yesterday than other teams?|
|Whenever a participant joins a team, any work they have submitted that wasn't previously assigned to a team will be assigned to the team that they just joined. So if team X has done 10,000 work units and a participant joins them that has 50,000 work units that aren't already assigned to a team, as soon as that team join takes effect, team X will have 60,000 work units total.
The confusing part is that team X's summary page won't show 50,000 blocks being done yesterday. They will move up a large amount in the overall ranking, even though they did very few (or even 0) work units yesterday.
This is intentional. Those 50,000 work units might have been assigned to the team yesterday, but they weren't actually submitted yesterday... they were submitted over many days or even months.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Teams : I created a team a few days ago, but stats says it's 100 days old!|
The age of a team, as reported in stats, is not related in any way to the day the team was actually created. The "start date" for a team is instead the oldest work unit that's been credited to that team.
If a participant joins your team and brings with them blocks which they've done in the past but have never assigned to a team, then your team will get credit for all those blocks on the days that those blocks were actually completed.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Technical info about statsbox|
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Technical info about statsbox : What type of machine is the stats server running on?|
The current stats servers, standard and deviation, are a pair of machines with the second acting a warm standby failover. The second machine also serves as a staging/development server. Both machines have the same identical configuration:
And prior to that, statsbox-i:
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Technical info about statsbox : Can stats processing be distributed to other computers?|
|Not really. Most of the work is adding up all the rows in a table. If
the connections between all the machines were very fast maybe it could
be done. It would be cheaper to just buy a faster server though.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Technical info about statsbox : Neat stats. Do you, like, use Cold Fusion or something?|
We use the very powerful VIM development environment to design the stats html. The powerful PHP package is used as the scripting language for all of the dynamic pages. PostgreSQL is the database backend that is used to store and access of all of the statistical data.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Client speeds database pages.|
|distributed.net maintains a public database of the benchmark speeds of its clients on various CPU models and speeds at http://www.distributed.net/speed/
This database is useful primarily for comparing the relative performance between different processor models. It can also be very useful for determining if your system is running the client as fast as other users are, and can help identify the existence of potential hardware configuration problems.
Keep in mind that the Client Speeds Database pages are completely unrelated to the Participant and Teams Stats database that shows your cumulative amount of work completed. The Client Speeds Database is meant purely to show the representative speed that a single client on a single processor type/speed can expect for a single benchmarked block.
|General speed benchmarking:|
My computer's speed is significantly slower/faster than the other entries already reported for my system!
What about benchmarks of multiprocessor machines?
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Client speeds database pages. : My computer's speed is significantly slower/faster than the other entries already reported for my system!|
This can occur because of any of the following reasons (and possibly others):
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Client speeds database pages. : What about benchmarks of multiprocessor machines?|
|The speed pages also provide the ability to enter the benchmarks of multiprocessor machines. However, keep in mind that the keyrate of any individual processor within a multi-processor machine will generally be comparable to that of identical MHz single-processor machine. As such, the overall speed for a multiprocessor machine is generally merely the speed of one processor multiplied by the number of processors within that machine.
The primary purpose of the separate multi-processor speed listing is to provide a single page that summarizes the types and speeds of the available multi-processor machines on the market against each other. Also many casual users simply aren't familiar with the availability/limits of specific processor counts for different architectures, and these multi-processor speed pages gives those users a convenient place to find that information.
Please keep in mind that the benchmark speed reported by the client is always for one processor! When submitting the speed for a multi-processor machine, you will need to manually multiply the single-processor before entering the speed within the submission form.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Client speeds database pages. : I've found a speed in the database that is obviously incorrect.|
Sometimes people accidentally submit speeds into our database that is obviously incorrect, for reasons such as the following:
If you suspect that you've found such an entry, use the "submit a bug request" link that is on the front http://www.distributed.net/speed/ page and we will manually remove such entries.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Client speeds database pages. : What are the StdDev and StdErr% columns?|
|Keep in mind for the following discussion that the blue column that is labelled as "Speed" is actually the mean (average) of all reported speeds for that CPU MHz combination.
StdDev represents the "standard deviation" of all reported speeds for that CPU MHz. The standard deviation is a statistical figure that can be used to determine the typical range in which all of the reported speeds lie within. Typically when there is a high amount of correlation between all of the reported values, then a high majority of the values will lie within reported the mean spead +/- the standard deviation amount. Values that lie outside of that range have a greater possibility of being anomalous.
StdErr% is simply a different representation of the standard deviation. It is simply the result of the following (stddev / mean * 100). StdErr% is useful for comparing the relative amount of deviation between each of the values within the summary listing pages. Typically, if the StdErr% exceeds 15% or 20%, then there is a high amount of deviation within that group of values, and one or more of the values is anaomalous to a large degree.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Client speeds database pages. : What is the "Record id" column?|
|This column is simply a unique number assigned to each new entry that each person enters into the database. It has no statisical value and only serves as a convenient way for us to point out individual anamalous entries and delete or correct them.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Client speeds database pages. : What is the best way to determine benchmarked speeds for entry to the database?|
|The way that I recommend you generate stats is by running the client 3 times, and taking the best rate of the three. Also, when possible, it is best to run the clients in as similar an environment as possible. Not all UNIX or multi-user machines have that luxury, so there may be some fluctuation between measured and actual speed.
If your processor, operating system, or client version is not one of the available options, then go back to the main speed index page and use the bugzilla link at the bottom of http://www.distributed.net/speed/ to report the value that you need.
Please make an effort not to submit bogus or invalid values. If you know that your speed value is suspiciously higher or lower than any of the other values reported by others for similar hardware, then recheck your values. Similiarly, if you know your system is grinding to a halt and about to crash and the speed reported by the client is totally non-sense, then don't report that value! Additionally, don't report excessively high/low values for favorite/hated operating system. The purpose of this database is provide a useful comparison for other users, not to promote platform advocacy or advertise bizarre values. By not submitting these bad values, you will save us the effort of having to remove it later.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Client speeds database pages. : My CPU, Operating System, or client Version is not one of the available options.|
|Please use the "submit a bug request" link that is available at the bottom of http://www.distributed.net/speed/ to submit a request and we will add it!
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Client speeds database pages. : When I submit a new entry into the Speeds Database, what is the email used for?|
|The email address that you enter is not used for any publicly displayed purpose. It only serves as a way to allow us to contact you if your entry seems to have been entered incorrectly and we need to ask you to re-enter it again or re-verify it.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Where can I find information of distributed.net's progress?|
|distributed.net maintains publicly viewable statistics and rankings
for every project. The server that hosts these statistics is
that server's main web page, you will we directed to the specific
stats subsections pertaining to each of our current projects and to user
or team specific statistics.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Are there any charts or graphs?|
|A large collection of plots, charts, and graphs are available at http://www.distributed.net/statistics/.
Graphs for individual participants and teams can be found on the stats server pages at http://stats.distributed.net/
|The regression lines on the RC5 rate graphs show best fits for assumptions of linear (in Mkeys/s / day) and exponential (in days to double) growth. By showing both regressions it helps cut short theoretical arguments, generally quoting Moore's Law, as to how we should be growing.|
An expanded vertical scale of the RC5 rate graph is available at http://www.distributed.net/statistics/rc5-64/
|The 3D OGR graphs located at http://www.distributed.net/statistics/ogr/ represent data that has been processed by clients working on the OGR-24 project. The two horizontal axes reflect the respective locations of the first and second mark on the Golomb ruler. The vertical axis accounts for the number of nodes tested for the corresponding first and second mark locations.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Can you please explain the statistics?|
A comprehensive set of statistics for the distributed.net projects can be found on the stats server.
There you can find the overall team summary as well as stats on teams and individuals, for each supported projects.
The stats pages are updated once per day at 00:00 UTC. There are several bits of
information displayed on the project, team and individual stats summary pages.|
Current Ranking -- team and individual summaries
Total blocks to search -- all summaries
Total blocks checked -- all summaries
Keyspace Exhausted -- all summaries
Total keys checked -- all summaries
Time Working -- all summaries
Overall Rate -- all summaries
Keyblocks and keyrate for yesterday -- all summaries
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : How often do stats update? When does that happen?|
|Stats update once per day, assuming everything is healthy.
The update starts at 00:00 UTC and typically takes a few hours to complete.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : What is UTC? Is that like GMT or something?|
Well, yeah, sorta. Unless you're into astronomy or extremely anal, UTC and GMT can
be considered the same thing.
If you need help figuring out what time UTC is in relation to your local time zone, this page may help. The trick is to remember that UTC doesn't change for daylight savings time.
If you're really curious about the difference, feel free to bore yourself to tears with
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : How come stats don't update more often?|
|Doing more frequent stats updates isn't really all that useful, and encourages behavior that's damaging to the network.
It's really in everyone's best interest if your clients only flush blocks once a day or so. When stats updated more frequently than daily, some people changed their client configuration to run the smallest block sizes and to flush after every block. This creates needless network traffic and makes for larger log files on the keymaster.
Besides, in a project that's taken over a year, does it really matter how many
blocks you had done *at noon*? Daily updates are quite sufficient for tracking
the progress of the project.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : How come I haven't received my password?|
Your stats password will not be automatically emailed to you once you join one of our projects. You will need to request your password by clicking on the link at the bottom of your personal stats page. You must first access your personal stats page by entering your complete email address in the "Participant Search" form located at the top of most stats server pages.
Note that in order to be able to request your password from your participant stats page, you will need to have entered a valid email address in the client configuration menu, flushed some work units back to the proxies, and waited up to 24 hours for the stats server's daily update that will detect that you have joined distributed.net.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : I'm losing my old email! Will all my stats be lost?|
No, the stats server has a feature for this situation. What you can do is "retire" your old address into
your new address. This will completely and permanently remove your old email address from the stats server
and attribute all of its activity to your new email address. This way you can gracefully migrate from one
email address to another without losing all your blocks.
To do this, you need a couple things:
Once all those things are in place, you simply visit the configuration page for the old email, and click the retire link. The script will walk you through the procedure of selecting your new email address and performing the retire.
|There is now a limit of 8 retired email addresses. Please read this other answer for information about this: Why can't I retire more than 10 email addresses into one?
|In addition, any new work that is submitted to your old address after you retire it will be credited to your new address. No work will be "lost" by retiring your accounts.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Do random blocks show up in stats?|
|Yes, and no. Like all blocks, randoms are subject to de-duping at the keymaster. This means that only the first person to submit any one random block will be credited for that block in stats. If two clients pick the same random block, only the first client to submit that block will be credited on stats.
You might assume, though, that the odds of duplicate random blocks is quite slim but in reality that's not always the case. There are two additional factors involved. Random blocks are not chosen from the entire 2**64 keyspace, but rather we've pre-defined some slices of keyspace as random slices, and when your client picks a random block it will come out of one of these areas. Which slice to use is determined by the keymaster and communicated to your client when it connects to the network. What this means is, the ratio of tested / untested keys in a random prefix can vary tremendously. As more and more random blocks are turned in, the current random keyslice gets more and more full and the odds of your client picking a duplicate block increase.
While allowing the clients to pick randoms from the entire keyspace might seem an attractive concept, there are two reasons why this is not wise. First, it wouldn't make any sense if we were 90% complete, since 90% of all randoms would end up as duplicates. Secondly, we are unable to track the entire keyspace on the keymaster. We can only accept blocks from "open" keyslices that are being actively tracked. If a block arrives at the keymaster for a keyslice that isn't open, it is discarded and will not be reflected in stats. Since we are limited in the number of keyslices we can simultaneously track, it's necessary
that we be able to control which areas from the keyspace a client pulls random blocks from.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : I think it would be really neat if stats showed X. Why doesn't it?|
It would be great if we could show every statistic every participant could think of, but we (the distributed.net staff) run in to some limits which force us to pick and choose:
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : I lost my stats password and lost access to my old email account.|
|Send email to email@example.com and write a message that is convincing enough to make them believe that the email address in question actually belonged to you. If they believe you, they can provide you with the password.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Why can't I retire more than 10 email addresses into one?|
|It was necessary to impose a limit because some malicious users were using this feature to combine email addresses that belonged to other users (and had gained access to the victim's email address and password through their own means).
We recognize that some users may need to legitimately retire more than 10 email addresses, and we will personally address these issues on a case by case basis. If you fall in this category, please send email to our general help email address describing your problem.
Please note that we do *not* support retires as a way of creating 'sub-teams'. We know that a number of participants would like to see this feature implimented, and it is on the to-do list. Please be patient, as this type of change is a lot more complicated than it looks, and it will take some time for us to devise a system to handle it.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : I'm missing work units!|
There are many reasons why work units could be 'disappearing'. One reason is latency. Our network is designed for maximum uptime and redundancy. This also means that it does not operate real-time... it takes some time between when you submit a work unit to a fullproxy and when it actually gets to the master. The stats are based on when a block is received by the keymaster, not when it was completed by the client. So if you flush at 2300UTC, those blocks might not make it into that night's statsrun. This means that you can not look at any single day and determine that you are missing blocks... they might show up tommorow, or even later. In extreme cases, we've even seen work units get delayed by 2 weeks.
If you are submitting work to a personal proxy (any proxy not officially run by distributed.net) then we can not guarantee that that work will make it to our network. This is because we can not control personal proxies. The person running a personal proxy can easily delete any work that has been submitted to the personal proxy without actually submitting the work to our network. Or their computer could crash. Or their dog could eat the blocks. You get the picture. }:8)
Many people have been confused about OGR stats. OGR is an ongoing projects, but it is divided into many small projects. The results for these seperate projects are reported seperately by stats, so please make sure and check any available OGR stats before reporting missing work.
If you still feel the need to contact us about missing work units, please follow these guidlines:
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : I mistyped the email address in my client and it has shown up in the stats. Is there any way for me to get credit for those work units?|
|If you have mistyped your email address in your client, and the client has already returned work units to distributed.net (i.e. the incorrect email address has shown up in the stats) you need to email firstname.lastname@example.org and explain the situation.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Stats are messed up! Some guy just jumped up 1,000 places in the ranking but only did a few blocks yesterday!|
|What you are seeing is the results of a retired account. The participant has retired another email account into the account you see listed, combining the statistics of both email addresses. This obviously affects their overall ranking, as their current email address suddenly gains the benefit of the blocks checked under their old, retired email resulting in an impressive jump in the overall rankings.
However, since the blocks in question weren't done yesterday, but rather are the result of their previous work under another email address, they have no impact on "yesterday" stats or the reported block count from the previous day.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : It's been over 24-hours, where are my stats?|
|A common comment towards distributed.net is that a new user can not find his or her stats after running the client for 24-hours (or more).
The key to this "24-hour" value is that the client has actually returned completed work to the distributed.net servers. If this is done, prior to 0:00 UTC, then you will receive credit for your work and should see your stats the "next day."
To return completed work to the distributed.net use the "dnetc -update" command or right-click on the cow-icon and choose "update." This command will force the client to "flush" (or return) work to the distributed.net servers. Note that you must be connected to the internet in order to accomplish this task.
The client, by default, can be configured a variety of ways to return completed work as the user desires. This can be done with changing the "Buffer Threshold" and/or the "Client Connectivity". Refer to these other topics for additional help:
How do I make the client start/stop/fetch/flush/etc at specific times of the day/week?
I use a modem. How can I get the client to download enough packets?
How can I fetch enough work for when I am offline?
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Why do statistics not update some times?|
|Some times the log files are not correctly processed by the statsbox and our audits that occur at the end of our statistics runs catch these problems before the data is updated completely. When this occurs someone must manually fix the problem, and some times that takes a while as the person could be out of town, or busy with other things (remember, we are all volunteers here) :)
Also some times the log files are not correctly transfered between tally and the keymaster, which again requires manual intervention to get them running again.
Your best bet is to check the .plans which are located at http://n0cgi.distributed.net/cgi/dnet-finger.cgi for more information, or visit us in #distributed on irc.distributed.net (most of the time people there know what's going on!)
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : Why is arbitrary HTML no longer allowed in team or participant mottos?|
|In January 2002, we had to instate a new policy that prohibited the use of arbitrary HTML in the customized team and participant "motto" sections of our website. http://cgi.distributed.net/cgi/planarc.cgi?user=bovine&plan=2002-01-22.20:52
Attempting to filtering general HTML is what a number of web-mail providers (like Hotmail, YahooMail, and others) attempt to do today, but if you've kept up on monitoring security vulnerabilities mailing lists (such as BugTraq or Vuln-Dev), you're probably aware of incident after incident of new tags being discovered that needed to also be filtered. These new discoveries have continued to occur every few months for the last several years, ever since the popularity of web-based mail took off. E-Bay also attempts to allow auctioneers to utilize HTML for their product descriptions, but they also need to filter malicious techniques that can be used to automatically submit bids and such. There are many other instances of websites that need to employ similar tactics.
Two well-written security advisories on this type of exploit are available from CERT:
Within the last month alone, there have been numerous cross-site scripting issues exposed throughout the web on extremely popular websites. Here are just a few of the articles on BugTraq within the last month that succinctly summarize the variety of the problems:
In summary, it is an ongoing job to maintain filters that attempt to explicitly exclude "known unsafe" activities, and we do not have the resources or desire to maintain such filtering technologies. It's far easier to do the opposite and create a filter that only allows "known safe" activities. Although it would be possible to try to construct a filter that allows good-HTML to be entered (but be very aggressive and only allow "known safe" forms), it's a little easier to create your own variant that does not have any of the operational expectations that HTML has. Attempting to filter HTML also has the problem of inadvertantly breaking "safe" HTML that simply does not conform to the strict filtering rules that would need to be used (after all, there are innumerable ways of expressing the same effect).
By creating our own markup language that is extremely limited and extremely strict in its allowances, there is far less ambiguity over why something isn't working or what is allowed. In actuality, the markup language that was chosen to be implemented is intentionally similar to the markup language used by UBB/phpBB/vBulletin and a number of other bulletin-board packages available today, so as to reduce the "non-standardness" of our technique. The available tags are shown on the team and participant configuration pages.
|distributed.net Faq-O-Matic : Statistics, Graphs, and other Statsbox things : The size of text on my stats page is too big/small|
|This is caused because we are utilizing the generic Cascading Style Sheet (CSS) font sizes. This _will_ cause different browsers to render the text slightly differently.
However, it also ensures that users can increase/decrease the font sizes as they see fit(there should be keyboard shortcuts to do this in all modern browsers) and have the browsers automatically adjust the font sizes accordingly.
© Copyright distributed.net 1997-2013 - All rights reserved