Federal Communications Commission
Reboot.FCC.gov
Home » Blog

Author Archive

New Market Opportunities and FCC Building Blocks

Posted March 23rd, 2011 by Michael Byrne - Geographic Information Officer

The National Broadband Map was developed to embody the spirit of the Internet.

 

Let me explain what I mean. The Internet is a two way street. At its most basic, the National Broadband Map shows how quickly Americans can give and take information over our national networks.

 

The Internet is also highly dynamic; it is constantly changing. The Map, too, was built to not only support but encourage change.

 

Since the product launched, we've seen some cool developments that use the Map's building blocks to make other projects better and more powerful. We think these developments lead to new opportunities -- new markets, new jobs, and new ways of tackling tough issues.

 

Here’s the first big example: The team at Broadband.com has integrated two of the FCC's APIs -- one for Census Block Conversion, the other to access the crowdsourced data points of the FCC Consumer Broadband Speed Test -- into their own mapping tool. This helps show the speed at businesses' locations -- a hugely important data point to surface as the speed of broadband to business and industry only climbs in importance in our connected market place.

 

On the government side, the excellent data team at the U.S. Department of Education just released a map product that visualizes one of the most vital aspects of broadband deployment in America: broadband availability for U.S. schools. Moving forward, the National Broadband Map will be collecting more information about Community Anchor Institutions – the places like schools and libraries that often are the central locations for public broadband access – that will help grow the functionality in products like the Broadband Availabilty map.

 

These integrations are a great example of government data products helping businesses build new markets. It’s exactly the spirit of integration we had in mind, and we’re excited to see other products -- in the private and public sectors -- continue down this path.

 

The second example is even more exciting. Since launch, we have received over 32,000 user-suggested inputs which are location based.  These 32,000 data points come in two varieties: some support the data that the map shows, and others point out discrepancies where user data doesn’t match the data in the map.

 

These user inputs are being fed back to the states -- the collectors of the National Broadband Map’s data -- to continuously improve the quality of the product. And user data input -- like our speed test -- continue to show the value that hybridized data collection techniques can pass on to users and innovators alike.

Posted in Developer Maps
6 Comments

First 24 hours

Posted February 18th, 2011 by Michael Byrne - Geographic Information Officer

 The launch of the National Broadband Map marks the beginning of a promising new venture: empowering consumers, researchers, policy-makers, and developers to truly understand what broadband means in America.


This idea — a powerful way to navigate huge troves of data to increase transparency and understanding — drove the production of the map. In building the map, our team had a hunch that there would be a hunger for a tool that served up this level of detail and information. The talented designers, web architects, and geospatial pros kept that in mind throughout the entire building process.

When the map went live yesterday, the response was astounding, with the number of requests to the website averaging more than 1,000 per second! Below is just a short list of the metrics we observed on our first day;

  • Total hits yesterday: 158,123,884
  • Hits served by cache: 141,068,348 (89.21%)
  • Total Bytes Transferred: 863GB
  • Peak Requests per Second: 8,970
  • Average Requests per Second: 1,095
  • Visits in the first 10 hours: over 500,000

This phenomenal response shows that the investment of time, energy, and — not least of all — Congressional funds were well worth it. The National Broadband Map clearly has a market of interest, and we’re extremely proud to see that market being well served.

With this kind of traffic, we are tripling efforts to serve you better. The team has been working round the clock to make infrastructure enhancements to the site. These enhancements include horizontal scaling of servers, adding more memory and more caching to the maps, tuning the map server architecture with the software developers for the map, and working with outside partners to help with the application. We are also working to resolve known browser issues with the map. Most features of the website can be viewed in any browser, but the maps in the gallery are best viewed with Firefox and Chrome. You can help identify and solve these issues through feedback.

I can’t wait to keep making the National Broadband Map better, particularly because I know that feedback, new ideas, and innovation around the map will be driving that process.

[Cross-posted from the National Broadband Map Blog.]

Posted in Data Developer Api Maps
3 Comments

IssueMap Round-up

Posted February 16th, 2011 by Michael Byrne - Geographic Information Officer

We’re really proud and humbled by the splash that IssueMap made last week. Thanks to the team at FortiusOne for rolling out a high-quality product that obviously hit the mark.

It’s exciting to see some of the cool IssueMaps that are shared over social networks. You can follow @IssueMap on Twitter to catch the shared IssueMaps published there. We’ve also put up a new Reboot page that collects a few FCC data sets and maps them on IssueMap.

We continue to hold strong to the belief that -- done right -- mapping will significantly change the way we understand data, solve problems, and tell compelling stories.

Here are some of the different angles on IssueMap:


Keep posting your IssueMaps, and stay tuned for more mapping news very soon.

Posted in Reform - Redesign Open Government Reform - Data Data Developer Api Maps
No Comments

Announcing IssueMap: Copy, paste, map.

Posted February 7th, 2011 by Michael Byrne - Geographic Information Officer

Everyone has seen a good spreadsheet go bad. Students, lawyers, public servants, accountants -- if you’ve ever spent time with a complicated dataset, you’ve had a spreadsheet turn against you.

Maps are a data visualization tool that can fix a rotten spreadsheet by making the data real and rich with context. By showing how data -- and the decisions that produce data -- affect people where they live, a map can make the difference between a blank stare and a knowing nod. Maps are also a crucial part of a decision-maker’s toolkit, clearly plotting the relationship between policies and geographies in easy-to-understand ways.

I’m extremely proud today to announce the official launch of IssueMap, the result of a partnership between the FCC and FortiusOne. IssueMap is a long time coming. As a board member with the National States Geographic Information Council, some colleagues and I identified the need for a product that would produce maps from complicated data steps in just three steps: copy, paste, map. IssueMap is that product.

Along with FCC Deputy GIO Eric Spry, we shot a video to show this drop-dead simple tool in action. Check it out, then visit IssueMap.org and try it. You can use the social media functionality in IssueMap to share your map with your community, or even export in a KML file to mash up your map other online services. Leave links to your maps in the comments here, and let us know what you want to see from the next iteration of IssueMap.

 

Posted in Data Developer Maps
4 Comments

Video: Crowd-sourced mobile broadband data

Posted January 25th, 2011 by Michael Byrne - Geographic Information Officer

 We recently spoke at one of the largest federal mapping data events, the ESRI Federal Users Conference, where we presented a cool implementation of FCC APIs mashed up with other, powerful datasets.

Last Spring, the FCC launched a pioneering crowd-sourced data collection tool: the FCC Consumer Broadband Speed Test. Since then, the test has been run more than 1 million times, collecting results both from wired and wireless connections. This is real data, from real consumers, in real communities. To make the data more useful, we released an API to unlock those results and hand the keys to the developer community.

The presentation showed that crowd-sourcing data collections can yield great things – not just for agencies – but for developers in the private and public sectors that can take the data and build new products, services, and research.

By the numbers alone, we know the test has been popular. And for a crowd-sourced federal data container, we think it's a huge success.

The particularly exciting part of this presentation was the ability to display projected speeds at different geographies within standard error, all extrapolated out from the the speed test data points that were input by users. As we explain in the video, by using the 1 million+ records submitted by users, we were able to display a map that shows the probability of a certain level of mobile broadband speed at any given spot in the U.S.

These data sets are great tools at our disposal, especially in the run up to the release of the National Broadband Map. As we get closer to the product launch in February, watch this space for updates of interest to developers, geographers, and consumers.

We’re interested to know what you think about the results, and what other uses for these datasets and APIs you come up with. Watch the video below of the presentation, then leave your comments.

Posted in Data Developer
1 Comment

Big Data

Posted October 28th, 2010 by Michael Byrne - Geographic Information Officer

The emerging discussion around "Big Data" is capturing imaginations quickly, and with good reason. The arenas that compose Big Data -- transaction records (e.g. banks, telephone service providers), social networks, and geospatial data all fall into this field.

Thanks to a combination of the advancement of computer science, demand for data, and ability for businesses and large institutions to make use of large datasets, Big Data is here.

Last week, I was honored to be part of the Booz Allen Series for Fed News Radio "Expert Voices," speaking on the topic of Big Data. From a government perspective, we at the FCC are not dealing with big data (billions of records) like some partners are, but we oversee and work with many industries that do. But to do the FCC's work, we are constantly collecting and tracking records in in the tens and hundreds of millions of records to help make sense of the communications industry.

As you might be able to tell from our data innovation initative, we clearly are in the business of better understanding our data assets and knowing where and how to make our own data better. We feel strongly that one of the most important aspects to a more efficient and effective FCC is strong data assets -- and in particular, ensuring our data assets are fully geospatially-enabled. When we have geospatially enabled data assets our ability to analyze trends over space and time will provide benefit to the policy discussion and provide a more informed and empowered arena for consumers.

Listen to the whole big data discussion or watch my small trailer on big data.

We're curious: How are other government organizations and large industry partners looking out at the horizon of Big Data? Leave us thoughts in the comments below.

Posted in Data
No Comments

FCC /Developer update: Building off community input

Posted October 13th, 2010 by Michael Byrne - Geographic Information Officer

Based on your comments and suggestions, we've just released updates to our FCC.gov/developer API’s.  In particular we have made two enhancements: one to help  navigate the complexities of census geography, and one that's purely stylistic.

Census geography changes, while small or obscure, can be significant.  A tiny change in a census boundary can mean that a rate based calculation includes a completely different denominator for population or demographic value.  These changes, if not watched for carefully can be significant to the results of querying large federal databases. 

To assist the community of developers building off FCC tools, we'll try and point out these small but significant changes when we see them.

The biggest change between 2000 census boundaries and 2009 census boundaries was the addition of a sub identifier to the smallest unit of boundary, the block.  This addition allows for finer resolution to the map base.  However, because of other changes like population growth, demographic switches, and land use changes, the external boundaries of the block boundaries have also changed. 

In order to keep up with these issues, we are supplementing our FCC Census Block API with the ability to query for the current year as well as previous years.  From now on, the current (e.g. 2009) year search will be the normal REST query on the documentation page like this;

http://data.fcc.gov/api//block/find?latitude=37.21&longitude=-120.4356

To gain access to a previous year, all you need to do is insert the year in the url like this; 

http://data.fcc.gov/api//block/2000/find?latitude=37.21&longitude=-120.4356


The stylistic change we made removes the @ symbol from the return, moving to a natural JSON notation to allow for better integration with some client libraries.

These changes are live now. Here's a preview of /Developer updates coming down the pipeline:


See below for a more complete description of the most recent changes.  Happy Coding, and don’t forget to let us know what kind of applications you are building with FCC Developer tools.

-----------

1)Changed JSON output to follow the natural convention for Block and Speed Test APIs. This changes the output from:

{"@executionTime":"0.071","@status":"OK","Block":{"@FIPS":"060470009012065"},
"County":{"@name":"Merced","@FIPS":"06047"},
"State":{"@name":"California","@code":"CA","@FIPS":"06"}}

To

{"Block":{"FIPS":"060470009012065B"},
"County":{"FIPS":"06047","name":"Merced"},
"State":{"FIPS":"06","code":"CA","name":"California"},
"status":"OK","executionTime":"0.445"}

Please note that the @ symbol has been removed from the attributes within a JSON object. This should make it easier for these data structures to be consumed from some libraries such as jQuery.

2) Added version year for census block geospatial search

There is an additional option to request Census Block data for Census 2000. The original call will return the most recent Census Block information (2009)

Original query for Census information (current information)

http://data.fcc.gov/api//block/find?latitude=37.21&longitude=-120.4356

<Response executionTime="0.054" status="OK">
<Block FIPS="060470009012065B"/>
<County name="Merced" FIPS="06047"/>
<State name="California" code="CA" FIPS="06"/>
</Response>

Census 2000 block information

http://data.fcc.gov/api//block/2000/find?latitude=37.21&longitude=-120.4356

<Response executionTime="0.052" status="OK">
<Block FIPS="060470009012065"/>
<County name="Merced" FIPS="06047"/>
<State name="California" code="CA" FIPS="06"/>
</Response>

3) Fixed the following bug

When passing wrong coordinate information (i.e. outside of the U.S. and territories) the response returned a badly formed XML if no format for the response type was specified.
 

Posted in Data Developer Api
No Comments

Update for /Developer

Posted September 14th, 2010 by Michael Byrne - Geographic Information Officer

Last week we announced the release of four API’s and the site fcc.gov/developer at the Gov 2.0 conference.  We heard great feedback via twitter, direct email and blog comments.  We have taken some of these ideas and implemented the changes right away.  We want to make sure that these services are useful to the developer community and that you know we are listening to your concerns here.  The changes we have made are listed below, but please keep the comments coming.  Your help is required to make these services better.
Thanks.

1. Bug fixes

- We heard about a bug in the FRN API that would cause a timeout when querying certain FRNs.  Sorry about that, it should be fixed now.
- We head about a bug in the Speed Test API that would cause wrong Wireline Maximum Download and Maximum Upload values in some cases.  Again, sorry about that, it should be fixed now.

2. API changes (Block Search)

- You gave us a suggestion that would make the return more compact and usable as we grow the service, so we decided to change the xml and JSON returns.  Now the Block Search API returns data in the following structure to facilitate parsing and future expansion. This
will break client applications of this method call if you implemented calls already to this API. 

New Structure:
XML
<Response executionTime="0.047" status="OK">
<Block FIPS="560239782002133"/>
<County name="Lincoln" FIPS="56023"/>
<State name="Wyoming" code="WY" FIPS="56"/>
</Response>

JSON
{"@executionTime":"0.047","@status":"OK","Block":{"@FIPS":"560239782002133"},
"County":{"@name":"Lincoln","@FIPS":"56023"},
"State":{"@name":"Wyoming","@code":"WY","@FIPS":"56"}}

JSONP
jsonp({"@executionTime":"0.047","@status":"OK","Block":
{"@FIPS":"560239782002133"},"County":{"@name":"Lincoln","@FIPS":"56023"},
"State":{"@name":"Wyoming","@code":"WY","@FIPS":"56"}})

3. API Enhancements (Census and Speed Test)

We added the ability to select desired MIME return type from the URL using the parameter format, i.e. format=json. Possible values are xml, json and jsonp (in this last case, the parameter callback should also be used). If no format is specified XML is returned. This change doesn't break the API (old calls would still work, returning XML).  

Posted in Reform - Redesign Open Government Reform - Data Developer
9 Comments

FCC.gov/developer

Posted September 7th, 2010 by Michael Byrne - Geographic Information Officer

Today the Federal Communications Commission is releasing four Application Programming Interfaces (API) and our first developer pages at fcc.gov/developer.  These APIs are part of our Data Innovation Initiative and are foundation pieces in our own redesigning and improving FCC's web presence. We want the FCC's web presence to be larger than a single web site. We want the developer community to run with these APIs to make mash-ups and data calls connecting FCC data assets to other sources for creative and useful applications to the public.

When we publish data through open standards like these APIs and smart people make use of trusted government data in innovative ways, we realize the ideal of the Gov 2.0 movement of government and private sector innovating together to solve our great policy challenges.

Several aspects of this release deserve highlighting.  First, APIs are central to our efforts for data transparency and open government. Second, all of these API’s are RESTful in nature and return open structured data as a service.  RESTFul APIs are popular on the web because they involve less programming effort to incorporate dynamically in mash-ups. We want many developers to view and use our data assets. 

Finally, and perhaps most importantly, two of these API’s (Consumer Speed Test and Census Block search) are location-based services returning data based on a latitude and longitude.  We imagine use of these APIs to unlock a myriad of government data based any user location.  As many notice the sea change in the use of geography by the public, the government has to respond with strong leadership.  In our case, leadership means publishing via open standards which fosters consumer and citizen connecting, communicating and collaborating to solve policy issues. 

The release of these APIs marks an important day for us at the FCC. The FCC has long published many data sets. Now we are allowing developers direct access to our data via live queries.  Your feedback on these APIs—what you think, how you are using them, what needs to be improved—helps us continue in this direction.

Below is a quick digest of each service.

Consumer Broadband Speed Test API
This API returns the Consumer Broadband Test speed test statistics for a US County given the passed Latitude and Longitude. The statistics are grouped into wireline and wireless and are the number of tests, average download speed, average upload speed, maximum download speed and maximum upload speed. This data is calculated nightly and includes all tests to date performed through the consumer broadband test.

Census Block Search API
This API returns the US Census Bureau Census Block number (aka the 15 character FIPS Code) given a passed Latitude and Longitude. The API also returns the US State and County name associated with the Block. Just about every major dataset in the US Federal Government uses some form of Census geography. The lowest level of that geography is the Census Block. This geography is highly nested, such that removing coded values from the right hand site aggregates to the next highest geography (e.g. from block to tract, and from tract to County). Providing this search allows developers to build applications which foster connectivity from their individual location to the full array of federal databases based on census geography.

FCC Registration Number (FRN) Conversion
Filers with the FCC must get a registration number (aka FRN) whose company names, parent names and subsidiary names often change from state to state and region to region. We are providing this API to increase the value of the transparency of broadband providers providing service in each state. Providing a single place for understanding the complex naming of these providers will benefit consumers.  This API has 2 method calls: getList and getInfo. GetList call takes the parameters of a state's abbreviation or state FIPS code, and a yes-or-no for multiple-state indicator, and returns all the broadband providers in that state. GetInfo call takes FRN number of a broadband provider, and return information about the provider. 

FCC License View
The FCC issues licenses for use of the nation's airwaves and other purposes. License View API provides snapshots such as the number of licenses across different services, how many licenses different entities have, and how many licenses are up for renewal in the near future. The following APIs represent a step toward reform of our licensing systems and improvement in how the FCC makes licensing information available to the public.

Posted in Open Government Data Developer
7 Comments