Introducing Measurement Lab
Wednesday, January 28, 2009
(Cross-posted from the Official Google Blog)
When an Internet application doesn't work as expected or your connection seems flaky, how can you tell whether there is a problem caused by your broadband ISP, the application, your PC, or something else? It can be difficult for experts, let alone average Internet users, to address this sort of question today.Last year we asked a small group of academics about ways to advance network research and provide users with tools to test their broadband connections. Today Google, the New America Foundation's Open Technology Institute, the PlanetLab Consortium, and academic researchers are taking the wraps off of Measurement Lab (M-Lab), an open platform that researchers can use to deploy Internet measurement tools.
Researchers are already developing tools that allow users to, among other things, measure the speed of their connection, run diagnostics, and attempt to discern if their ISP is blocking or throttling particular applications. These tools generate and send some data back-and-forth between the user's computer and a server elsewhere on the Internet. Unfortunately, researchers lack widely-distributed servers with ample connectivity. This poses a barrier to the accuracy and scalability of these tools. Researchers also have trouble sharing data with one another.
M-Lab aims to address these problems. Over the course of early 2009, Google will provide researchers with 36 servers in 12 locations in the U.S. and Europe. All data collected via M-Lab will be made publicly available for other researchers to build on. M-Lab is intended to be a truly community-based effort, and we welcome the support of other companies, institutions, researchers, and users that want to provide servers, tools, or other resources that can help the platform flourish.
Today, M-Lab is at the beginning of its development. To start, three tools running on servers near Google's headquarters are available to help users attempt to diagnose common problems that might impair their broadband speed, as well as determine whether BitTorrent is being blocked or throttled by their ISPs. These tools were created by the individual researchers who helped found M-Lab. By running these tools, users will get information about their connection and provide researchers with valuable aggregate data. Like M-Lab itself these tools are still in development, and they will only support a limited number of simultaneous users at this initial stage.
At Google, we care deeply about sustaining the Internet as an open platform for consumer choice and innovation. No matter your views on net neutrality and ISP network management practices, everyone can agree that Internet users deserve to be well-informed about what they're getting when they sign up for broadband, and good data is the bedrock of sound policy. Transparency has always been crucial to the success of the Internet, and, by advancing network research in this area, M-Lab aims to help sustain a healthy, innovative Internet.
You can learn more at the M-Lab website. If you're a researcher who'd like to deploy a tool, or a company or institution that is interested in providing technical resources, we invite you to get involved.
UPDATE: Check out the video from the launch event at the New America Foundation:
I really like the sound of these tools, unfortunatley I foresee these tools being used in a malicious manner as well. Without knowing more about the tools, I would say they could also be used to by attackers to footprint their marks.
ReplyDeletePerhaps my ISP is blocking this?
ReplyDeletehttp://www.measurementlab.net/measurement-lab-tools says
"Not Found
The requested URL /measurement-lab-tools was not found on this server.
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
Apache/2.0.61 (Unix) mod_ssl/2.0.61 OpenSSL/0.9.8b FrontPage/5.0.2.2635 mod_auth_passthrough/2.1 mod_bwlimited/1.4 PHP/5.2.5 Server at www.measurementlab.net Port 80"
ED
my ISP blocks the site
ReplyDeleteevery time I go there I get "The connection to the server was reset while the page was loading"
It would be extremely useful to understand more about how these tools actually work.
ReplyDeleteOn the surface they seem like the ycould be useful but one obvious concern is how often they yield a false positive.
How do the tools determine exactly what is happening end to end between a browser and a server?
For that matter what problems can these tools find on a server hosting an application? Can content providers be called out too??
There is a home network (or business network), access network, backbone, perhaps several networks that are transited, a hosting facility, etc etc.
How do we know a particular measurement is not biased. Good researchers always try to minimize that effect but it is a natural human tendency to want to get the reulst you want to see. The best way to avoid this is rigorous peer review.
have service providers gotten the opportunity to vet these tools to verify there is no bias?
How do the tools determine that there is not a network/equipment problem versus a provider "blocking bit torrent".
There is a strong tendency these days to shoot first and ask questions later. Take a look at the comments to this blog posting - the tendency was to assume the provider was blcoking access to your site (how ridiculous) when it appears the site may have been overloaded,
If a provider is unfairly implicated for blocking and the firestorm starts the damage is done. Subsequent attempts to remedy the situation tend not to get the same publicity as the initial news.