I recently participated in a panel (PDF) at the Berkeley Center of Law and Technology, “Technology, Transforming the Regulatory Endeavor“.
Code can be viewed as an implicit form of regulation by embedding assumptions and decisions in the technology.
Interestingly enough, code often does not reflect regulator intent. Travis Breaux from CMU noted that engineers often miss important constraints in regulations and sometimes amplify the requirements when dealing with ambiguity in the regulations. System design can have an impact, for example direct screen writing in an OS might run counter of implicit accessibility assumptions in a regulation. Danielle Citron from the Maryland School of Law reinforced this theme, noting that programmers can distort policy through simplifications or make policy changes of their own. An example is that incorrect rules in a system design resulted in food stamps being denied to those with prior drug convictions – a decision against the law.
Automation is an issue since people tend to believe a computer result (“automation bias”), notice is lacking, and it is hard to have a hearing without the information implicit in the software.
Has policy been unintentionally been delegated to programmers?