"Reducing Software Failures:
Addressing the Ethical Risks of the Software Development Lifecycle"

Don Gotterbarn

Software engineering has been evolving and refining techniques to help them produce software products that meet the needs of their clients. These techniques emphasize a specific set of risks -missed schedule, over budget, and failing to meet the system's specified requirements. Nevertheless, software development has been characterized as a "software crisis". Some software is being delivered late, over budget, and not meeting all requirements.

In this paper I address a different sense of software failure. Software has is considered to have failed even though it was produced on schedule within budget and met the customer's specified software requirements. Software has been developed which, although meeting stated requirements, has significant negative social and ethical impacts. The Aegis radar system, for example, met all requirements that the developer and the customer had set for it. The system designer's did not take into account the users of the software nor the conditions in which it would be used. The system was a success in terms of budget, schedule, and requirements satisfaction, even so, the user interface to the system was a primary factor in the Vincennes shooting down an Iranian commercial airliner killing 263 innocent people.

There are two factors that contribute to these professional and ethical failures. There is significant evidence that many of these failures are caused by limiting the consideration of relevant system stakeholders to just the software developer and the customer. This limited scope of consideration leads to developing systems that have surprising negative affects because the needs of relevant system stakeholders were not considered. In the case of the Aegis radar system the messages were not clear to the users of the system operating in a hostile environment. These types of failures also arise from the developer limiting the scope of software risk analysis just to technical and cost issues. A complete software development process requires the identification of all relevant stakeholders and broadening the risk analysis to address social, political, and ethical issues. Software development lifecycle methods include a risk analysis process but with current methods limit the types of risks considered. The risk analysis is primarily instrumental-addressing corporate bottom lines. Software projects have ethical dimensions that need to be identified before and during the development process. I propose some modifications to the stand development models that will address these additional types of risk.

I briefly examine some techniques developed by software engineers to which attempt to include a broader consideration of stakeholders, such as viewpoint requirements definition. Some of these software development methods articulate a distinction between direct system stakeholders-- (those who)"receive services from the system and send control information to the system"-and indirect stakeholders-- those who "have an interest in some of the services that are delivered by the system but do not interact directly with it". These would include the passengers on the Iranian airline or the driver of an automobile whose breaks are controlled by a computer program. Unfortunately 1) these methods do not provide an ethical or philosophical foundation for this distinction to reach beyond identifying those who have a business relation to the customer. They would not have identified as indirect stakeholders the 47 people killed by falling debris from a patriot missile. These methods also fail to 2) provide a method of identifying the social and ethical impacts on the indirect stakeholders.

Barry Boehm's has developed a methodology which comes close to meeting 1) the stakeholder identification problem. His Win-Win spiral software development technique is used to elicit project requirements for all stakeholders. At each phase of a project's development the analyst identifies the stakeholders for that stage, determines the win conditions for each new stakeholder, and then negotiates to have these new win condition requirements fit into a set of Win-Win conditions that have already been established for all concerned. There is a set of win conditions for the Aegis radar customer. These conditions would be identified and a process developed to meet those conditions. Then new stakeholders would be identified, for example the sailor's using the system on the Vincennes, and their win conditions would be identified. They would consider it important to be able to clearly determine if an approaching aircraft were hostile. This win-condition would be incorporated, via negotiation, into the existing process plan. Although this approach is similar to Rawl's wide reflective equilibrium in deriving a coherent set of requirements through negotiation, the ethical element is missing from this method. There is no methodology to identify ethically relevant stakeholders nor is there an ethical foundation for the negotiation process.

The method is also limited in that it assumes all stakeholders are equal and that they will equally be aware of and able to describe their own win conditions. The negotiation amongst stakeholders will be unjust and will likely lead to a failed systems, unless, contrary to fact, each stakeholder has such an equal identification and descriptive skill of their own win conditions. There is also an implicit assumption that all requirements are negotiable. As the method is constructed, all requirements have equal status-non are rejected because they are morally impermissible or required because they are morally mandatory.

This model has two strengths. First it comes closer to properly expanding stakeholder identification than other software engineering methodologies. Unlike all of the other approaches that presume the impact analysis is done as a single process, the Win-Win model is iterative requiring a re-identification of system stakeholders at each stage of the development process. This iterative approach is consistent with the model of ethical reasoning argued for by van den Hoven in "Computer Ethics and Moral Methodology."

The major portion of the paper develops a methodology, which incorporates the strengths of Boehm's Win-Win model, to help software engineers address the ethical issues that lead to failed systems. The methodology contains a technique for stakeholder identification and an approach to ethical analysis in software development. The method is based in part on the work by Mitchell, Agle, and Wood, who provide techniques for identifying stakeholders in terms of their roles. I also draw on Pouloudi's work on stakeholders and software engineering. I incorporate normative principles into these techniques. This method avoids many difficulties with business ethics methods of stakeholder identification that fail to capture requirements that emerge from the relationship between stakeholders. The goal of the method is to help the software engineer identify all f the ethically relevant stakeholders and provide structure to the process through a series of ethics principles. This method is consistent with Rawl's philosophy. I have argued elsewhere that the obligations of professional software engineers can best be understood and justified using Rawl's reflective equilibrium. A software development methodology that has roots in that philosophy is consistent with the professional and moral obligations of software developers.

Software, which has been developed to test the feasibility of this method, will also be presented.

Back to Accepted Papers
Back to Top