Archive for the ‘Site News’ category.

Posts about new GovTrack features, media coverage, and other site developments.

July 19, 2012

Data license changes, take two

By Josh Tauberer. Categorized in Site News.

Last post I wrote that I’d be changing the data licensing terms on GovTrack. I sincerely asked for feedback, and I got it. Gunnar, for instance, rightly pointed out that this isn’t everyone’s fight. He also noted that my dry sense of humor wasn’t really working.

The change I planned would have created substantial burdens for re-users of GovTrack’s data and yet would have had little impact, except possibly to annoy end-users. So I’m going to make a different change. This is a lot simpler:

You may not disparage services for being nonpartisan.

This will go into the terms of service to access the regularly updated raw data and API starting tomorrow. (It does not affect regular users of GovTrack or of any of the users of the tools that use GovTrack data.)

I don’t know of any current licensees that were a part of the boycott that started all of this, but I don’t know everyone who uses GovTrack’s data. If your organization can’t handle the new term, then I’m not above saying you can get your data elsewhere.

That said, the source code for the scripts that gather the data remain open source under the GNU AGPL license (see this github project for v2). And I do create special license agreements with other organizations as necessary. So there are at least two routes around this.

All along I suspected I wasn’t going to go ahead with the original changes that I announced. Like I said originally, it was ridiculous. I was trying to make a point, and I did.


This all began because some random guys said some other company should be boycotted for being nonpartisan. It really had nothing to do with me. But as I explained to techPresident I was appalled that:

“This is the first time that someone’s called what our community does ‘evil,’” Tauberer said in an e-mail. “I don’t take that lightly. PCCC’s Rosenbaum had better stand behind that if he is going to be so brash. Is he going to take the links to GovTrack off of the PCCC web site? Because right now those links support the right’s ability to get the same information.”

I think the open gov community is used to me curmudgeonly complaining about various things. Sometimes I try to be polite. But, honestly, I’ve gotten tired of being mostly complacent. In June I called out Rep. Crenshaw for trying to slow down legislative transparency and got almost 1,500 letters sent to Congress about it. This month I’m calling out this ridiculous boycott. Yeah, this might be the start of a new pattern.

A colleague pointed out to me over the weekend that the open source movement has a long history of using licenses to promote ideological positions. The GNU GPL license — the ‘viral’ license that is part of the backbone of the open source world — says you can use my software if you believe in the same sort of openness that I do. The GPL also prevents licensees from exercising software patent rights — which is in many ways a political statement. One of the earliest leaders of the free software movement believes it is a moral imperative for computing technology to be free (free as in freedom). Over at Creative Commons, licenses make a distinction between commercial and non-commercial use, which is something that to a for-profit guy like me thinks is pretty arbitrary.

So I’m not the first to think that giving stuff away can come with substantive terms and still be open. Though in my case, the license is a terms of service, not a copyright license, so the comparison with open source only goes so far.

July 12, 2012

Data license terms to change in response to Netroots conference boycott of nonpartisan tool NationBuilder

By Josh Tauberer. Categorized in Site News.

I sent the following message out to the GovTrack data developers mail list. It affects anyone who uses GovTrack’s bulk raw data downloads and API.

Dear data re-users,

This is advanced notice that on July 20 I’ll be revising the terms of GovTrack’s generic license agreement by adding the following paragraph:

* During the time in which your organization is reusing GovTrack’s database, your website must block visitors referred by the websites of sponsors of the Netroots Nation conference. If you make the data available in bulk to others, your license agreement must carry over the same terms.

In techPresident today, Netroots Nation’s executive director Raven Brooks encouraged progressive advocacy groups to boycott NationBuilder, a non-partisan technology platform that helps campaigns build their websites, because NationBuilder sells services to right-leaning organizations. There’s a complicated history here that’s touched on in the techPresident article (, but Brooks’s point seems to boil down to a belief that there can be no nonpartisan political tools.

GovTrack, and most tools that reuse its database, is a nonpartisan tool that has played an important role in political activism over the last several years on both sides of the political aisle. It is astonishing to me that anyone would think that technology infrastructure should choose sides. Especially since it appears that the sponsors of past Netroots Nations conferences have been users of nonpartisan political technology platforms like GovTrack. If they are going to boycott tools like GovTrack, they certainly won’t notice the change to GovTrack’s license terms.

Your feedback on these changes is welcome, especially if the feedback is in the style of satire. But this isn’t a joke. Ridiculous boycotts of technology startups require ridiculous responses.

June 24, 2012

Site Updates: Spring 2012

By Josh Tauberer. Categorized in Site News.

A lot’s been happening here at GovTrack over the last few months. Here’s a quick run-down of the most recent changes:

  • If you use GovTrack to build lists of items to track, the Your Lists page has gotten a much needed visual refresh. It also now provides RSS feed links for each of your lists (which we had in the old GovTrack but hadn’t yet brought over). Finally, note that Your Lists has now been tucked inside a new Track menu at the top of the page.
  • We’ve put up a new page called Other Tools in the new Track menu featuring other ways you can track Congress besides using GovTrack.
  • Bill pages have been slightly reorganized. They now highlight the major votes on bills so you can find the votes you are looking for faster. We’re also now starting to include links to news articles for bills.
  • The Washington Post used our legislative data in an analysis of stocks being traded by Members of Congress published today, showing that conflicts of interest are pervasive.
  • A lot of bugs have been fixed, e.g. voting absentee rates were missing for Barack Obama‘s time as a senator.
  • The ordering of bills in the advanced search results has been improved with our “special sauce” that brings bills you’re probably looking for to the top.
  • We’ve had a few great press mentions lately: check out the full list.
  • For developers, we have a new experimental API to complement our bulk data downloads. And since launching the API we added to it information on roll call votes. The developer documentation has been improved.

We recently announced:

And coming up:

  • You may notice that bill text comparisons are coming back. Our automatic red-lining feature is better than ever. I’ll write separately on this in a few weeks, since right now there are still some bugs to take care of.
June 19, 2012

Welcoming the 50 States to GovTrack

By Josh Tauberer. Categorized in Site News.

If tracking one Congress was great, then tracking 51 of them must be even better! This week GovTrack added legislative tracking for the 50 states. You can now find:

The new state legislative information is available here through a partnership with LegiNation, Inc. and LegiScan, Inc. Some data is additionally from Open States. Check out LegiNation’s BillTrack50 website for professional (i.e. paid) legislative tracking tools if you’re ready for a step up from GovTrack and need a 50-state-plus-federal solution.

State tracking is a beta feature of GovTrack. We’ll be improving it over time in response to feedback from users like you.

The full announcement is below:

New  Open Data Partnership Improves Government Transparency at State and Federal Levels

GovTrack and LegiNation trade legislative data and help citizens engage with government

DENVER and WASHINGTON –June 18, 2012 – In a significant step toward greater government transparency, two organizations focused on publishing government data have joined forces to present  major new comprehensive, user friendly, publicly accessible databases of state and federal bills.  LegiNation, creator of the BillTrack50 solution to track state bills will exchange information with open government technology company Civic Impulse, developer of the federal legislative monitoring tool GovTrack. The aim of the data exchange is to provide individuals, advocacy groups, businesses and legislative professionals with access to information to better engage with government.

Joshua Tauberer is president of Civic Impulse and the author of Open Government Data: The Book which frames the open government data movement as the application of Big Data to civics. Mr. Tauberer said the partnership with LegiNation creates an important source of government data and presents citizens with the opportunity for increased participation in and influence of government.

“Citizens are far more savvy about their government and more interested in participating in it than most people realize. Apathy is a myth. But what they need are better tools to participate effectively and efficiently,” Mr. Tauberer contends.

Federal and state bills can be accessed on the and websites for free, and a professional version allowing for advanced tracking and sharing is available through BillTrack50. The sites present bill text, status, and summaries in a format easily located by Google-like word search or by using more exact information like legislator names or bill numbers. Tools to stay informed of legislative activity are also provided.

Karen Suhaka, president of LegiNationl said the combined data will present a more accurate picture of the political landscape across the country, making it is easier to identify state and federal policy trends.  The partners also hope to improve the ability to analyze how ideas spread in government.

“People who are frustrated with Congress must look at the possibility of successful engagement at the state level. Making this data available to the public and to cost-conscious legislative professionals encourages everyone, not just moneyed interests, to make their voice heard in government,” Ms. Suhaka said.  “Individuals really can engage at the state level and have an impact.”

About is a tool by Civic Impulse, LLC to help the public research and track the activities in the U.S. Congress and state legislatures, promoting and innovating government transparency and civic education through novel uses of technology. Launched in 2004 initially as a hobby, was the first website worldwide whose primary goal was to provide free and comprehensive legislative tracking for everyday citizens and to embrace Web 2.0 and open data. was a 2006 Webby Award nominee and has been covered in The New York Times and The Washington Post, and it is the data provider for many other websites displaying legislative information.

About LegiNation

LegiNation, Inc. was founded in 2011 with the goal of making state level legislation more readily available to the professionals who need it, and even more importantly to the public at large. LegiNation is building products and websites that will spark a renaissance in American politics, leveraging the Internet to create the dialog so desperately needed amongst our elected officials, legislative professionals, and everyday citizens.

June 7, 2012

Rep. Crenshaw backs down, loses control over bulk data issue

By Josh Tauberer. Categorized in Legislation In The News, Site News.

UPDATE: The Washington Post covered this story online on June 8, and on the front page in print on June 9.

The government data that makes GovTrack go has been the center of what looks like a failed political power play over the last week. Rep. Crenshaw, whose appropriations subcommittee issued a draft report last week that nearly halted access to “bulk data downloads,” now “agree[s] to free legislative information” according to a statement written jointly with House leaders yesterday.

Throughout the week, the Sunlight Foundation, GovTrack, and others had been working with legislative staff and raising awareness among the public. More than 1,400 letters were written by citizens to their representative about this issue. (In fact, you can read those letters at POPVOX.) The data in question contains the status of all bills currently being considered by Congress and having proper access to it would make applications like GovTrack (among many others) more timely and more accurate.

The committee’s report for the legislative branch appropriations bill H.R. 5882 confused a fear of technology with a reason to withhold public files from the public. It said that technology doesn’t exist when it does, it suggested Congress find funding to “confirm or invalidate third party analyses of legislative data,” and it established a task force to investigate the issue that had no deadline and no incentive to ever make the legislative data files available to the public. (More on that in my previous post and in a post by Sunlight’s Daniel Schuman.)

If Crenshaw was attempting to slow down and bring legislative transparency under the purview of his subcommittee — the House Appropriations Subcommittee on the Legislative Branch — he might have not realized he was crossing his party superiors, the House Republican Leadership, who have made significant advances in transparency over the last year and a half. Crenshaw has no history with any efforts related to government transparency, as far as I’m aware.

The task force originally proposed by the appropriations committee has now been preempted by a new task force called at the discretion of House Leadership to investigate how to make the data files available. Although the new task force still has no deadline and lacks public input (see Daniel’s post), we know that House Leaders — Speaker Boehner and Majority Leader Cantor — have made good on those sorts of promises recently.

It was a circuitous path to this point. After Sunlight raised concerns with the draft committee report published last Wednesday, the report was revised in its final version to not halt existing bulk data files. Unfortunately, the committee didn’t tell anyone about the change and so no one noticed the disappearance of 10 words until Tuesday morning.

Meanwhile on Tuesday Rep. Issa scrambled on proposing an amendment to the bill that would immediately direct the Library of Congress to make the files available — no task force required. (Recall that the original problem was in the committee report for the bill. The amendment, as a change to the bill itself, was a considerably stronger directive.) Issa has been a strong supporter of the use of data for government transparency. His DATA Act, if enacted, would enormously improve how federal dollars are spent in the executive branch by instituting government-wide data standards. Although Issa’s amendment to H.R. 5882 was ultimately mooted by the joint statement, which Issa was a part of, his initiative in fixing the problem that Crenshaw started should be remembered.

Rep. Honda has been supporting bulk data all along. A member of the appropriations subcommittee, he supported bulk data as early as 2009 and did so again earlier this year when the committee was first considering H.R. 5882. As a member of the minority party, Honda was out-gunned early on in this round and did as much as he could.

So where are we now? While we still don’t have the data we want, at least it is not being studied by a task force both unfriendly to the idea of legislative transparency and based on severe technological misunderstandings. Instead, we have a commitment from the House Republican leadership that they will look into it soon, and based on their past performance that should be taken seriously. The crisis has been averted. Congress won’t be taking a step backward. It remains to be seen if the result was a step forward.

Still, this is only one half of the picture. Even with the House on board, the Senate still remains completely uninterested in bulk data availability of legislative information. We won’t get those files until both the House and Senate concur that making these data files available is a good thing.

June 4, 2012

Rep. Crenshaw thinks American public can’t be trusted with overseeing Congress

By Josh Tauberer. Categorized in Legislation In The News, Site News, Uncategorized.

UPDATE: See how this issue resolved.

Rep. Ander Crenshaw (FL4) and the House subcommittee he chairs decided this week that the American public can’t be trusted with more thorough records about what Congress is doing.

/// Take Action: Write your rep to oppose Crenshaw’s report.
/// Start a Letter >

Read it all..

May 12, 2012

New: Legislation Coming Up

By Josh Tauberer. Categorized in Site News.

It used to be that us outsiders never exactly knew when a bill was going to come up for a vote. But thanks to recent developments in the House we can be more involved in the legislative process in the crucial days before a bill comes to a vote.

GovTrack’s new Docket page and Coming Up feed give you a heads-up about bills coming up in the House and Senate. We’re also tweeting the upcoming bills and posting them to our Facebook page as they are posted. For instance, we just tweeted that H.R. 4119: Border Tunnel Prevention Act of 2012 and a handful of other bills were added on Friday to the House’s schedule for next week.

The House schedule is based on a new House website called that came online this past January thanks to the work of House Majority Leader Eric Cantor, the Office of the Clerk, the Committee on House Administration, and the Speaker’s office. The Senate schedule is based on the Floor Schedule for the next day published on

The new website is a major milestone for legislative transparency. For the first time, House leadership is not only committing to a rough schedule a week in advance but is also publishing the schedule in a re-usable, machine-processable format that allows websites like GovTrack to include the information quickly and reliably. Which we’ve done.

We’ve been pushing Congress to share more information in technologically-enabled ways for the last five years. This new website came out of a pledge from Cantor’s office about a year ago to make more legislative data available. It was a response to the concerns we raised along with a handful of other government transparency organizations. While there’s much more legislative data we want, is itself a laudable milestone.

May 6, 2012

New: Faceted Bill Search

By Josh Tauberer. Categorized in Site News.

One of my own favorite new parts of GovTrack 2.0 is the advanced bill search, which you can find in Browse => Bills => Search & Track => Advanced Search.

The advanced search — or in technical jargon the “faceted” search — can be used to drill-down into all of the bills that are currently before the U.S. Congress or actually any bill introduced since 1973. Besides searching bill titles, you can narrow your search using a number of other filters. What’s really cool is that the filters change as you drill-down so that you can find the available choices quickly.

For instance, choose Rep. Eric Cantor as the sponsor and then the filter for bill current status will only show the statuses of Cantor’s 17 bills in the current Congress. Next to each status is the number of Cantor’s bills with that status: 3 signed by the president, 2 bills passed the House, 11 resolutions passed, 1 bill waiting on committee action. Cantor has gotten a lot of bills passed.

You can also use this to get a quick count of the number of bills enacted since the start of the current session of Congress (2011-2012). Start at the advanced search and then find the filter “Signed by the President.” Next to it, it currently reads 106 bills. That’s how many bills became law. The count is updated as fast as the other bill information on GovTrack — typically it’s one or two days behind. Congress’s parliamentary procedure is pretty complex, so if you hover your mouse over any of the status options you’ll get a little explanatory text for it.

The filters in advanced search are: when the bill was introduced, who sponsored it, its current status in the legislative process, its subject area, and the bill’s type (e.g. bill versus resolution). You can also sort the results by the bill’s most recent status change date, by its date of introduction, or by its relevancy to a title text search.

The same search interface appears in a few other places on GovTrack. Can you find them all?

April 8, 2012

Even Better Bill Prognosis: Now with Real Probabilities

By Josh Tauberer. Categorized in Site News, Uncategorized.

Bill prognosis has gotten an upgrade. A few weeks ago I wrote about a new addition to GovTrack, the bill prognosis, and because it proved to be useful I expanded on it to provide a more detailed and numerical assessment of the future of each bill. For an example, check out H.R. 4323, which we’re currently listing as having a 2% chance of being enacted.

If you missed the first post, here’s an overview: Only a small fraction of the bills and resolutions introduced will ever be voted on. How do you know which bills to pay attention to? We can’t predict the future, but we can highlight factors that favor a bill’s progress and use statistics from past years to assess the likelihood that a bill will be enacted.

In the first version of bill prognosis, we listed whether the bill’s sponsor is the chair of the committee considering the bill and whether any cosponsors of the bill are on the committee. Now, we use factors like these and more to compute an actual probability that the bill will be enacted (or for resolutions that they will be passed).

For H.R. 4323: Consumer Mortgage Choice Act, we are currently listing a 2% chance of the bill being enacted and we show that the factors used in the computation are:

  • 3-5 cosponsors serve on a committee to which the bill has been referred.
  • There is at least one cosponsor from the majority party and one cosponsor outside of the majority party.
  • The sponsor is a member of the majority party.

I unfortunately confused things a bit by running a separate experiment recently. We were listing a probability that each bill would be enacted and were asking whether you thought the number was too low or too high. That wasn’t based on the bill prognosis — I’ll write about where that number was coming from another time. But I saw that GovTrack users were finding it useful, and so I knew enhancing the bill prognosis with a rigorously computed probability would be a good idea.

For the data wonks out there, the new prognosis is based on a logistic regression model. The model predicts a bill’s success based on the following binary factors:

  • the title of the bill (such as if it is a bill to name a post office)
  • whether the sponsor is a member of the majority party (in the House or Senate as appropriate)
  • whether the sponsor is the chair, ranking member (most senior minority party member), or a majority-party member of a committee that the bill has been referred to
  • if any cosponsor is the chair or ranking member of a committee the bill has been referred to
  • if there are 3-5 cosponsors of the bill serving on a committee the bill has been referred to
  • if the bill has a cosponsor from both parties
  • if the bill’s sponsor is in the majority party and at least 1/3rd of the cosponsors are from the minority party
  • if the bill was a reintroduction of a bill from the previous Congress (same sponsor and title, ignoring any year found in the title) and, separately, if the previous bill had been reported by committee (suggested by Tom Lee and Daniel Schuman shortly after this post was first published)
  • if the bill’s sponsor or cosponsors have a high leadership score based on GovTrack’s analysis of bill sponsorship (based on a suggestion from Mackenzie Morgan shortly after this post was first published)
  • and if any of these factors are true of a companion (identical) bill introduced in the other house of Congress

Success is for bills if they are enacted and for resolutions if they successfully reach the end of their life cycle (simple resolutions passed, concurrent resolutions passed by both chambers, joint resolutions enacted). A separate model was constructed for each of the eight bill types (H.R., S., S.Res., etc.). Additional models were created for bills that were at least reported by committee, and the prognosis for such bills is based on those separate models. The models were trained on bills and resolutions from 2009-2010 (the previous session of Congress).

Here are some results of the model. Of Senate bills in 2009-2010, only 2.8% were enacted. The regression coefficients for the model for Senate bills are listed below, in order from most indicative of a successful bill to least indicative. Also listed with each is the percentage of bills with that attribute that were enacted, to compare against the 2.8%.

  • 2.7 — the bill’s title starts with “A bill to designate the” (usually naming a post office) (24% enacted)
  • 2.1 — a companion bill has a cosponsor that is the ranking member of a relevant committee (22% enacted)
  • 2.0 — a companion bill’s sponsor is the chair of a relevant committee (29% enacted)
  • 1.7 — the bill’s title starts with “A bill to authorize” (10% enacted)
  • 1.0 — a cosponsor is the ranking member of a relevant committee (13% enacted)
  • 0.94 — a cosponsor is the chair of a relevant committee (9.4% enacted)
  • 0.76 — there are cosponsors from both parties (5.2% enacted)
  • 0.63 — the sponsor is from the majority party and at least 1/3rd of the cosponsors are from the minority party (6.8% enacted)
  • 0.47 — the bill was a re-introduction of a bill that was reported by committee in the previous session of Congress (7.6% enacted)
  • 0.37 — the sponsor is the chair of a committee to which the bill was referred (7.7% enacted)
  • 0.27 — a companion bill has a cosponsor from the majority party and a cosponsor from the minority party (6.3% enacted)
  • 0.12 — a companion bill is a reintroduction of a bill that was reported by committee in a previous session of Congress (11% enacted)
  • 0.080 — a companion bill has 6 or more cosponsors on a relevant committee (4.9% enacted)
  • 0.013 — a cosponsor in the majority party has a high leadership score (7.4% enacted)
  • -0.032 — a companion bill’s sponsor is in the majority party (5.4% enacted)
  • -0.13 — a companion bill’s cosponsor is the chair of a relevant committee (8% enacted)
  • -0.14 — 3-5 cosponsors are members of a relevant committee (4.7% enacted)
  • -0.85 — the sponsor is in the minority party (1.2% enacted)
  • -1.5 — the bill was a re-introduction of a bill from the previous session of Congress that had no major action (<1% enacted)
  • -34 — the bill’s title starts with “A bill to extend the temporary suspension of duty” (0% enacted)

The regression coefficients are not easily interpretable on their own, except that higher numbers mean that they more importantly indicate they help a bill get enacted. The largest negative numbers indicate those bills pretty much never get enacted. Small negative numbers don’t necessarily mean that the factor hurts a bill’s chances

(Before using the regression model I tested that each factor taken independently was statistically significant, but in a way that in retrospect was not a particularly good way to do it. My intuition is that the regression model factors are probably all statistically significant anyway. Based on that initial test of significance I excluded some of these factors from some of the models. For instance, no factors were statistically significant for the model for Senate bills reported by committee, in part because the sample of 523 bills is so much smaller. No regression model is used in that case, and the overall probability of 21% for all such bills is used as the prognosis for all bills in this category. Note how much larger 21% is than the overall success rate for all Senate bills, 2.8%.)

UPDATES: After folks suggested new factors to consider, I re-created the models and the regression coefficients above were updated.

March 24, 2012

New on GovTrack: Bill Prognosis

By Josh Tauberer. Categorized in Site News.

Only a small fraction of the bills and resolutions introduced will ever be voted on. How do you know which bills to pay attention to? Now on GovTrack you can find a bill’s prognosis.

We can’t predict the future, but we can highlight factors that favor a bill’s progress. Check out H.Con.Res. 112 for example. Below the bill status chart in the middle of the page you’ll now find Prognosis. We’ll list there whether the bill’s sponsor is the chair of the committee considering the bill and whether any cosponsors of the bill are on the committee. Committees are where bills come for their real judgement. If a committee chair introduces a bill that gets assigned to his own committee, that bill is likely to be important.

Having support from other members of the committee is helpful as well. On the page for S. 2170 we list three relevant factors: the chair of the committee that the bill has been referred to is a cosponsor of the bill, the sponsor of the bill is on that committee, and one other cosponsor is also on that committee.

If no favorable factors are present, the prognosis section is omitted.