The construction of a computer-based system (CBS) begins ideally with the elicitation of its requirements and the writing of a specification of its requirements, usually in the form of a natural-language (NL) requirements specifications (RS). This chapter begins by describing (1) three kinds of NL RSs: software requirements specification, a.k.a. SRSs, user manuals, and user stories, and (2) the qualities, such as correctness and lack of ambiguity, that they as NL RSs must have to serve their purpose of adequately specifying the requirements of a CBS. The chapter then defines the kinds of defects that an RS, being written in an NL, may suffer, with an emphasis on ambiguity as the kind of defect arising specifically from the use of NL to write the RS. The chapter describes how some defect-detection tools work. It observes that whether an NL RS has any of these kind of defects is fundamentally algorithmically undecidable. However, searching for instances of a finite number of indicators of some kinds of defects, such ambiguity or vagueness, is feasible with 100% recall. While an RS may not have such a defect without the presence of one its indicators, not every occurrence of an indicator of a defect is part of an instance of the defect. The chapter reviews efforts to empirically evaluate a defect-detection tool’s (1) effectiveness: how well a defect-detection tool detects the defects in its scope, often expressed in terms of recall and precision, and (2) usefulness: whether detecting these defects in an RS positively affects the quality of the CBS that is constructed from the RS. The chapter observes that the construction of many a defect-detection tool is accompanied by an empirical demonstration of its effectiveness, but not by any demonstration of its usefulness. Moreover, each of three attempts to empirically demonstrate usefulness of abstraction finding failed; neither did any of the found ambiguities cause serious problems nor were any of the serious problems caused by ambiguities. Instead, in each case, as a result of a robust requirements analysis process, all the stakeholders happened to agree on the meaning of every considered ambiguity. The chapter concludes by wondering whether any defect-detection tool is useful.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Detecting Defects in Natural Language Requirements Specifications

  • Daniel M. Berry,
  • Erik Kamsties,
  • Cristina Ribeiro,
  • Sri Fatimah Tjong

摘要

The construction of a computer-based system (CBS) begins ideally with the elicitation of its requirements and the writing of a specification of its requirements, usually in the form of a natural-language (NL) requirements specifications (RS). This chapter begins by describing (1) three kinds of NL RSs: software requirements specification, a.k.a. SRSs, user manuals, and user stories, and (2) the qualities, such as correctness and lack of ambiguity, that they as NL RSs must have to serve their purpose of adequately specifying the requirements of a CBS. The chapter then defines the kinds of defects that an RS, being written in an NL, may suffer, with an emphasis on ambiguity as the kind of defect arising specifically from the use of NL to write the RS. The chapter describes how some defect-detection tools work. It observes that whether an NL RS has any of these kind of defects is fundamentally algorithmically undecidable. However, searching for instances of a finite number of indicators of some kinds of defects, such ambiguity or vagueness, is feasible with 100% recall. While an RS may not have such a defect without the presence of one its indicators, not every occurrence of an indicator of a defect is part of an instance of the defect. The chapter reviews efforts to empirically evaluate a defect-detection tool’s (1) effectiveness: how well a defect-detection tool detects the defects in its scope, often expressed in terms of recall and precision, and (2) usefulness: whether detecting these defects in an RS positively affects the quality of the CBS that is constructed from the RS. The chapter observes that the construction of many a defect-detection tool is accompanied by an empirical demonstration of its effectiveness, but not by any demonstration of its usefulness. Moreover, each of three attempts to empirically demonstrate usefulness of abstraction finding failed; neither did any of the found ambiguities cause serious problems nor were any of the serious problems caused by ambiguities. Instead, in each case, as a result of a robust requirements analysis process, all the stakeholders happened to agree on the meaning of every considered ambiguity. The chapter concludes by wondering whether any defect-detection tool is useful.