Farina, C.R., Epstein, E., Heidt, J., and Newhart, M.J. (2012). Knowledge in the People: Rethinking "Value" in Public Rulemaking Participation. Wake Forest Law Review, 47(5), 1185-1241.
A companion piece to Rulemaking vs. Democracy: Judging and Nudging Public Participation that Counts, this Essay continues to examine the nature and value of broader public participation in rulemaking. Here, we argue that rulemaking is a “community of practice,” with distinctive forms of argumentation and methods of reasoning that both reflect and embody craft knowledge.More
Rulemaking newcomers are outside this community of practice: Even when they are reasonably informed about the legal and policy aspects of the agency’s proposal, their participation differs in kind and form from that of sophisticated commenters. From observing the actual behavior of rulemaking newcomers in the Regulation Room project, we suggest that new public participation is often, if not predominantly, experiential in nature and narrative in form. We argue that it is unrealistic to expect that rulemaking newcomers can be significantly inculcated into the norms and methods of the existing rulemaking community of practice. Yet, the potential policymaking value of the on-the-ground, situated knowledge they can bring to the discussion justifies efforts to expand our understanding of the kinds of comments that should “count” in the process. We take some first steps in that direction in this Essay.Less
Solivan, J. and Farina, C.R. (2103). Regulation Room: How the Internet Improves Public Participation in Rulemaking. Proceedings of the Marine Safety & Security Council, 69(4) and 70(1), 58-82.
Cornell eRulemaking Initiative (CeRI) designed and operated Regulation Room, a pilot project that provides an online environment for people and groups to learn about, discuss, and react to selected proposed federal rules.More
The project is a unique collaboration between CeRI academic researchers and the government. The U.S. Department of Transportation (USDOT) was CeRI's first agency partner and chose Regulation Room as its first open government "flagship initiative." USDOT received a White House Open Government Leading Practices Award for its collaboration in the project. CeRI owns, designs, operates, and controls Regulation Room, but works closely with partner agencies to identify suitable "live" rulemakings for the site and to evaluate success after a rule closes.Less
Epstein, D., Vernon, R., & Newhart, M. (in press). Not by Technology Alone: The “Analog” Aspects of Online Public Engagement in Policymaking. Government Information Quarterly.
Farina, C.R., Epstein, D., Heidt, J.B. and Newhart, M.J. (2013). Regulation Room: Getting "More Better" Civic Participation in Complex Government Policymaking. Manuscript submitted for publication. (Paper available from authors on request.)
Rulemaking (the process agencies used to make new health, safety, social and economic regulations) is one of the U.S. government's most important policymaking methods and has long been a target for e-government efforts. Although broad transparency and participation rights are part of its legal structure, significant barriers prevent effective engagement by many citizens.More
Regulation Room, an experimental open-government partnership between academic researchers and government agencies, is a socio-technical online participation system that uses multiple methods to alert and effectively engage new voices in rulemaking. Initial results give cause for optimism. We caution, however, that successful use of new technologies to increase civic participation in complex government policymaking is more difficult and resource-intensive than many e-government proponents expect. On a more positive note, our experience has opened a new perspective on the value that broader citizen participation can add to public policymaking – i.e., information about impacts, enforceability, contributory causes, unintended consequences, etc. that is known by the commenter because of lived experience in the complex reality into which the proposed policy would be introduced. This “situated knowledge” is often conveyed through stories and accounts of the commenter’s experience. By recognizing the valuable information that can come in these non-conventional kinds of comments, participation designers can create an environment in which citizens are able to recognize the relevance of their life experiences and contributing knowledge the government agency does not itself possess.Less
Farina, C.R., Heidt, J., & Newhart, M.J. (2012). Designing an Online Civic Engagement Platform: Balancing "More" vs. "Better" Participation in Complex Public Policymaking. Manuscript submitted for publication. (Paper available from authors on request.)
A new form of online citizen participation in government decisionmaking has arisen in the United States under the Obama Administration. “Civic Participation 2.0” attempts to use Web 2.0 information and communication technologies to enable wider civic participation in government policymaking, based on three pillars of open government: transparency, participation, and collaboration.More
Thus far, the Administration has modeled Civic Participation 2.0 almost exclusively on a universalist/populist Web 2.0 philosophy of participation. In this model, content is created by users, who are enabled to shape the discussion and assess the value of contributions with little information or guidance from government decisionmakers. We suggest that this model often produces “participation” unsatisfactory to both government and citizens. We propose instead a model of Civic Participation 2.0 rooted in the theory and practice of democratic deliberation. In this model, the goal of civic participation is to reveal the conclusions people reach when they are informed about the issues and have the opportunity and motivation seriously to discuss them. Accordingly, the task of civic participation design is to provide the factual and policy information and the kinds of participation mechanisms that support and encourage this sort of participatory output. Based on our experience with Regulation Room, an experimental online platform for broadening effective civic participation in rulemaking (the process federal agencies use to make new regulations), we offer specific suggestions for how designers can strike the balance between ease of engagement and quality of engagement – and so bring new voices into public policymaking processes through participatory outputs that government decisionmakers will value.Less
Park, J., Cardie, C., Farina, C.R., Klingel, S., Newhart, M. & Vallbé, J.J. (2012). Facilitative Moderation for Online Participation in eRulemaking. Proceedings of the 13th Annual International Conference on Digital Government Research. College Park, MD: ACM.
This paper describes the use of facilitative moderation strategies in an online rulemaking public participation system. Rulemaking is one of the U.S. government’s most important policymaking methods.More
Although broad transparency and participation rights are part of its legal structure, significant barriers prevent effective engagement by many groups of interested citizens. Regulation Room, an experimental open-government partnership between academic researchers and government agencies, is a socio-technical participation system that uses multiple methods to lower potential barriers to broader participation. To encourage effective individual comments and productive group discussion in Regulation Room, we adapt strategies for facilitative human moderation originating from social science research in deliberative democracy and alternative dispute resolution [24, 1, 18, 14] for use in the demanding online participation setting of eRulemaking. We develop a moderation protocol, deploy it in “live” Department of Transportation (DOT) rulemakings, and provide an initial analysis of its use through a manual coding of all moderator interventions with respect to the protocol. We then investigate the feasibility of automating the moderation protocol: we employ annotated data from the coding project to train machine learning-based classifiers to identify places in the online discussion where human moderator intervention is required. Though the trained classifiers only marginally outperform the baseline, the improvement is statistically significant in spite of limited data and a very basic feature set, which is a promising result.Less
Farina, C.R., Heidt, J., Newhart, M.J., & Vallbé, J.J. (2012). RegulationRoom: Field-Testing An Online Public Participation Platform During USA Agency Rulemakings. In M. Gascoì (Ed.), Proceedings of 12th European Conference on eGovernment. Reading: Academic Publishing International.
Rulemaking is one of the U.S. government's most important policymaking methods. Although broad transparency and participation rights are part of its legal structure, significant barriers prevent effective engagement by many groups of interested citizens.More
RegulationRoom, an experimental open-government partnership between academic researchers and government agencies, is a socio-technical participation system that uses multiple methods to alert and effectively engage new voices in rulemaking. Initial results give cause for optimism but also caution that successful use of new technologies to increase participation in complex government policy decisions is more difficult and resource-intensive than many proponents expect.Less
Farina, C.R., Newhart, M. & Heidt, J. (2012). Rulemaking vs. Democracy: Judging and Nudging Public Participation that Counts. Michigan Journal of Environmental & Administrative Law, 2(1), 123-217.
Open government enthusiasts assume that more public participation will lead to better government policymaking: If we use technology to give people easier opportunities to participate, they will use these opportunities to participate effectively.More
However, experience with technology-enabled rulemaking (e-rulemaking) belies this assumption. Engagement of new participants most often takes the form of mass comment campaigns orchestrated by advocacy groups. Challenging the conventional highly negative response to mass commenting, Prof. Nina Mendelson has recently argued that, in a democratic government, agencies should give at least some weight to the value preferences expressed in such comments when rulemaking involves value judgments. Engaging this important argument, we propose a framework for assessing the value of technology-enabled rulemaking participation. Our position -- that the types of preferences expressed in mass comments may be good enough for electoral democracy but they are not good enough for even heavily value-laden rulemaking -- challenges both the Web 2.0 ethos and the common open-government belief that more public participation, of any kind, is a good thing. In rulemaking and similar complex policymaking processes, more public participation is good only if it is the kind of participation that has value in the process. We offer specific principles of participation-system design that are drawn from both normative conceptions of the responsibilities of a democratic government and from the design-based research being carried on by the CeRI (Cornell eRulemkaing Iniative) in the Regulation Room project. We argue that design of civic engagement systems must involve a purposeful and continuous effort to balance “more” and “better” participation, and stress that a democratic government should not actively facilitate public participation that it does not value.Less
Farina, C. R., Newhart, M. J., Cardie, C., & Cosley, D. (2011). Rulemaking 2.0. University of Miami Law Review, 65( 2), 395-448.
In response to President Obama's Memorandum on Transparency and Open Government, federal agencies are on the verge of a new generation in online rulemaking. However, unless we recognize the several barriers to making rulemaking a more broadly participatory process, and purposefully adapt Web 2.0 technologies and methods to lower those barriers, Rulemaking 2.0 is likely to disappoint agencies and open-government advocates alike.More
This article describes the design, operation, and initial results of Regulation Room, a pilot public rulemaking participation platform created by a cross-disciplinary group of Cornell researchers in collaboration with the Department of Transportation. Regulation Room uses selected live rulemakings to experiment with human and computer support for public comment. The ultimate project goal is to provide guidance on design, technological, and human intervention strategies, grounded in theory and tested in practice, for effective Rulemaking 2.0 systems.
Early results give some cause for optimism about the open-government potential of Web 2.0-supported rulemaking. But significant challenges remain. Broader, better public participation is hampered by 1) ignorance of the rulemaking process; 2) unawareness that rulemakings of interest are going on; and 3) information overload from the length and complexity of rulemaking materials. No existing, commonly used Web services or applications are good analogies for what a Rulemaking 2.0 system must do to lower these barriers. To be effective, the system must not only provide the right mix of technology, content, and human assistance to support users in the unfamiliar environment of complex government policymaking; it must also spur them to revise their expectations about how they engage information on the Web and also, perhaps, about what is required for civic participation.
Farina, C. R., Miller, P., Newhart, M. J., Cardie, C., Cosley, D., & Vernon, R. ( 2011). Rulemaking in 140 Characters or Less: Social Networking and Public Participation in Rulemaking. Pace Law Review, 31(1), 382-463.
Rulemaking – the process by which administrative agencies make new regulations -- has long been a target for e-government efforts. The process is now one of the most important ways the federal government makes public policy. Moreover, transparency and participation rights are already part of its legal structure. The first generation of federal e-rulemaking involved putting the conventional process online by creating an e-docket of rulemaking materials and allowing online submission of public comments. Now the Obama Administration is urging agencies to embark on the second generation of technology-assisted rulemaking, by bringing social media into the process.More
In this article we describe the initial results of a pilot Rulemaking 2.0 system, Regulation Room, with particular emphasis on its social networking and other Web 2.0 elements. (A companion article, Rulemaking 2.0, gives a more general overview of the project and is forthcoming in Miami Law Review). Web 2.0 technologies and methods seem well suited to overcoming one of the principal barriers to broader, better public participation in rulemaking: unawareness that a rulemaking of interest is going on. We talk here about the successes and obstacles to social-media based outreach in the first two rulemakings offered on Regulation Room. Our experience confirms the power of viral information spreading on the Web, but also warns that outcomes can be shaped by circumstances difficult, if not impossible, for the outreach effort to control.
There are two additional substantial barriers to broader, better public participation in rulemaking: ignorance of the rulemaking process, and the information overload of voluminous and complex rulemaking materials. Social media are less obviously suited to lowering these barriers. We describe here the design elements and human intervention strategies being used in Regulation Room, with some success, to overcome process ignorance and information overload. However, it is important to recognize that the paradigmatic Web 2.0 user experience involves behaviors fundamentally at odds with the goals of such strategies. One of these is the ubiquitousness of voting (through rating, ranking, and recommending) as “participation” online. Another is what Web guru Jacok Neilsen calls the ruthlessness of users in moving rapidly through web sites, skimming rather than carefully reading content and impatiently seeking something to do quickly before they move on. Neither of these behaviors well serves those who would participate effectively in rulemaking. For this reason, Rulemaking 2.0 systems must be consciously engaged in culture creation, a challenging undertaking that requires simultaneously using, and fighting, the methods and expectations of the Web.
Bruce, T.L., Cardie, C., Farina, C.R. & Purpura, S. (2008). Facilitating Issue Categorization & Analysis in Rulemaking. Proceedings of the 9th Annual International Conference on Digital Government Research. Montreal, Canada: Digital Government Society of North America.
One task common to all notice-and-comment rulemaking is identifying substantive claims and arguments made in the comments by stakeholders and other members of the public. Extracting and summarizing this material may be helpful to internal decisionmaking; to produce the legally required public explanation of the final rule, it is essential.More
When comments are lengthy or numerous, natural language processing and machine learning techniques can help the rulewriter work more quickly and comprehensively. Even when a smaller volume of comment material is received, the ability to annotate relevant portions and store information about them in a way that permits retrieval and generation of reports can be useful to the agency, especially over time. We describe a prototype application for these purposes. The Workspace for Issue Categorization and Analysis (WICA) allows the rulewriter to create a list of relevant substantive categories and assign them to marked portions of comment text. She can then retrieve all instances of a given issue within the comment pool. Preliminary results of experiments that apply text categorization and active learning methods to comment sets suggest that these techniques can facilitate the marking and category assignment process in lengthy or numerous comment sets. WICA will incorporate these techniques. Other possible applications of WICA within the rulemaking process are discussed.Less
Cardie, C. , Farina, C.R., Aijaz, A., Rawding, M. & Purpura, S. (2008). A Study in Rule-Specific Issue Categorization for e-Rulemaking. Proceedings of the 9th Annual International Conference on Digital Government Research. Montreal, Canada: Digital Government Society of North America.
We address the e-rulemaking problem of categorizing public comments according to the issues that they address. In contrast to previous text categorization research in e-rulemaking, and in an attempt to more closely duplicate the comment analysis process in federal agencies, we employ a set of rule-specific categories, each of which corresponds to a significant issue raised in the comments.More
We describe the creation of a corpus to support this text categorization task and report interannotator agreement results for a group of six annotators. We outline those features of the task and of the e-rulemaking context that engender both a non-traditional text categorization corpus and a correspondingly difficult machine learning problem. Finally, we investigate the application of standard and hierarchical text categorization techniques to the e-rulemaking data sets and find that automatic categorization methods show promise as a means of reducing the manual labor required to analyze large comment sets: the automatic annotation methods approach the performance of human annotators for both flat and hierarchical issue categorization.Less
Cardie, C., Farina, C.R., Rawding, M. & Aijaz, A. (2008). An eRulemaking Corpus: Identifying Substantive Issues in Public Comments. Proceedings of the 9th Annual International Conference on Digital Government Research. Montreal, Canada: Digital Government Society of North America.
We describe the creation of a corpus that supports a real-world hierarchical text categorization task in the domain of electronic rulemaking (eRulemaking). Features of the task and of the eRulemaking domain engender both a non-traditional text categorization corpus and a correspondingly difficult machine learning task.More
Interannotator agreement results are presented for a group of six annotators. We also briefly describe the results of experiments that apply standard and hierarchical text categorization techniques to the eRulemaking data sets. The corpus is the first in a series of related sentence-level text categorization corpora to be developed in the eRulemaking domain.Less
Purpura, S., Cardie, C & Simons, J. (2008). Active Learning for e-Rulemaking: Public Comment Categorization. Proceedings of the 9th Annual International Conference on Digital Government Research. Montreal, Canada: Digital Government Society of North America.
We address the e-rulemaking problem of reducing the manual labor required to analyze public comment sets. In current and previous work, for example, text categorization techniques have been used to speed up the comment analysis phase of e-rulemaking by classifying sentences automatically, according to the rule-specifc issues or general topics that they address.More
Manually annotated data, however, is still required to train the supervised inductive learning algorithms that perform the categorization. This paper, therefore, investigates the application of active learning methods for public comment categorization: we develop two new, general-purpose, active learning techniques to selectively sample from the available training data for human labeling when building the sentence-level classiers employed in public comment categorization. Using an e-rulemaking corpus developed for our purposes, we compare our methods to the well-known query by committee (QBC) active learning algorithm and to a baseline that randomly selects instances for labeling in each round of active learning. We show that our methods statistically signifcantly exceed the performance of the random selection active learner and the query by committee (QBC) variation, requiring many fewer training examples to reach the same levels of accuracy on a held-out test set. This provides promising evidence that automated text categorization methods might be used effectively to support public comment analysis.Less
Committee on the Status and Future of Federal e-Rulemaking (U.S.), & American Bar Association. (2008). Achieving the Potential: The Future of Federal E-Rulemaking. Chicago: Section of Administrative Law and Regulatory Practice, American Bar Association.
Cardie, C., Farina, C.R., Bruce, T., & Wagner, E. (2006). Using Natural Language Processing to Improve e-Rulemaking. Proceedings of the 7th Annual International Conference on Digital Government Research. San Diego, CA: Digital Government Research Center.
Cardie, C., Farina, C.R., Bruce, T., & Wagner (2006). Better Inputs for Better Outcomes: Using the Interface to Improve e-Rulemaking. Proceedings of the Workshop on eRulemaking at the Crossroads, 7th Annual International Conference on Digital Government Research. San Diego, CA: Digital Government Research Center.
We believe that e-rulemaking does indeed have potential to increase both the transparency of, and participation in, regulatory policymaking. We argue in this paper that this potential can be realized only if the public interface at www.regulations.gov is substantially redesigned.