Software quality 1301906 223312944 2008-07-03T14:50:05Z Mhaitham.shammaa 1608031 {{Cleanup|date=April 2008}} {{Software development process}} In the context of [[software engineering]], '''software quality''' measures how well [[software]] is designed (''quality of design''), and how well the software conforms to that design (''quality of conformance''),<ref name="Press746">Pressman, Roger S. ''Software Engineering: A Practitioner's Approach''. Sixth Edition, International, p 746. McGraw-Hill Education 2005.</ref> although there are several different definitions. For their certification in software quality engineering (CSQE), the [[American Society for Quality]] (ASQ) lists seven major topic areas in the '''2008 CSQE body of knowledge.''' *General [quality] knowledge *Software quality management *Systems and software engineering processes *Project management *Software metrics and analysis *Software verification and validation (V&V) *Software configuration management<br /><br /> Whereas ''quality of conformance'' is concerned with implementation (see [[Software Quality Assurance]]), ''quality of design'' measures how valid the design and [[requirement]]s are in creating a worthwhile [[Product (business)|product]].<ref>Pressman, Roger S. ''Software Engineering: A Practitioner's Approach''. Sixth Edition, International, p 388. McGraw-Hill Education 2005.</ref> {{portal|Software Testing}} ==Definition== One of the problems with Software Quality is that "everyone feels they understand it."<ref>Crosby, P., ''Quality is Free'', McGraw-Hill, 1979</ref> In addition to the definition above by Dr. Roger S. Pressman, other software engineering experts have given several definitions. A definition in Steve McConnell's ''Code Complete'' similarly divides software into two pieces: '''internal''' and '''external quality characteristics'''. External quality characteristics are those parts of a product that face its users, where internal quality characteristics are those that do not.<ref>McConnell, Steve. ''Code Complete'' First Ed, p. 558. Microsoft Press 1993</ref> Another definition by [[Tom DeMarco|Dr. Tom DeMarco]] says "a product's quality is a function of how much it changes the world for the better."<ref>DeMarco, T., "Management Can Make Quality (If)possible," Cutter IT Summit, Boston, April 1999</ref> This can be interpreted as meaning that user satisfaction is more important than anything in determining software quality.<ref name="Press746"/> Another definition, coined by [[Gerald Weinberg]] in ''Quality Software Management: Systems Thinking'', is "Quality is value to some person." This definition stresses that quality is inherently subjective - different people will experience the quality of the same software very differently. One strength of this definition is the questions it invites software teams to consider, such as "Who are the people we want to value our software?" and "What will be valuable to ''them''?". == History == ===Software product quality=== *[[Product (business)|Product]] quality **conformance to [[requirements analysis|requirements]] or [[program specification]]; related to [[Software reliability|Reliability]] *[[Scalability]] **[[Correctness]] *[[Completeness]] *absence of [[Computer bug|bugs]] *[[Fault-tolerance]] **[[Extensibility]] **[[Maintainability]] *[[Documentation]] ===Source code quality=== To a computer, there is no real concept of "well-written" source code. However, to a human, the way a program is written can have some important consequences for the human maintainers. Many source code [[programming style]] guides, which stress readability and some language-specific conventions are aimed at the maintenance of the software source code, which involves debugging and updating. Other issues also come into considering whether code is well written, such as the logical structuring of the code into more manageable sections. *[[Readability]] *Ease of [[Software maintenance|maintenance]], [[software testing|testing]], [[debugging]], fixing, modification and portability *Low [[complexity]] *Low resource consumption: [[computer storage|memory]], [[Central processing unit|CPU]] *Number of compilation or [[Lint programming tool|lint]] warnings Methods to improve the quality: [[refactoring]]. ==Software reliability== Software '''reliability''' is an important facet of software quality. It is defined as "the probability of failure-free operation of a computer program in a specified environment for a specified time".<ref>Musa, J.D, A. Iannino, and K. Okumoto, ''Engineering and Managing Software with Reliability Measures'', McGraw-Hill, 1987</ref> One of reliability's distinguishing characteristics is that it is objective, measurable, and can be estimated, whereas much of software quality consists of subjective criteria.<ref>Pressman, Roger S. ''Software Engineering: A Practitioner's Approach, Sixth Edition International,'' McGraw-Hill International, 2005, p 762.</ref> This distinction is especially important in the discipline of [[Software Quality Assurance]]. These measured criteria are typically called ''[[software metric]]s''. ===History=== With [[Embedded system|software embedded]] into many devices today, software failure has caused more than inconvenience. Software errors have even caused human fatalities. The causes have ranged from poorly designed user interfaces to direct [[programming error]]s. An example of a programming error that lead to multiple deaths is discussed in Dr. Leveson's paper [http://sunnyday.mit.edu/papers/therac.pdf] (PDF). This has resulted in requirements for development of some types software. In the United States, both the [[Food and Drug Administration|Food and Drug Administration (FDA)]] and [[Federal Aviation Administration|Federal Aviation Administration (FAA)]] have requirements for software development. === The goal of reliability === The need for a means to objectively determine software quality comes from the desire to apply the techniques of contemporary engineering fields to the development of software. That desire is a result of the common observation, by both lay-persons and specialists, that computer software does not work the way it ought to. In other words, software is seen to exhibit undesirable behavior, up to and including outright failure, with consequences for the data which is processed, the machinery on which the software runs, and by extension the people and materials which those machines might negatively affect. The more critical the application of the software to economic and production processes, or to [[life-sustaining system]]s, the more important is the need to assess the software's reliability. Regardless of the criticality of any single software application, it is also more and more frequently observed that software has penetrated deeply into most every aspect of modern life through the technology we use. It is only expected that this infiltration will continue, along with an accompanying dependency on the software by the systems which maintain our society. As software becomes more and more crucial to the operation of the systems on which we depend, the argument goes, it only follows that the software should offer a concomitant level of dependability. In other words, the software should behave in the way it is intended, or even better, in the way it should. === The challenge of reliability === The circular logic of the preceding sentence is not accidental — it is meant to illustrate a fundamental problem in the issue of measuring software reliability, which is the difficulty of determining, in advance, exactly how the software is intended to operate. The problem seems to stem from a common conceptual error in the consideration of software, which is that software in some sense takes on a role which would otherwise be filled by a human being. This is a problem on two levels. Firstly, most modern software performs work which a human could never perform, especially at the high level of reliability that is often expected from software in comparison to humans. Secondly, software is fundamentally incapable of most of the mental capabilities of humans which separate them from mere mechanisms: qualities such as adaptability, general-purpose knowledge, a sense of conceptual and functional context, and [[common sense]]. Nevertheless, most [[software]] [[programs]] could safely be considered to have a particular, even singular purpose. If the possibility can be allowed that said purpose can be well or even completely defined, it should present a means for at least considering objectively whether the [[software]] is, in fact, reliable, by comparing the expected outcome to the actual outcome of running the [[software]] in a given environment, with given data. Unfortunately, it is still not known whether it is possible to exhaustively determine either the expected outcome or the actual outcome of the entire set of possible environment and input data to a given [[program]], without which it is probably impossible to determine the [[program|program's]] reliability with any certainty. However, various attempts are in the works to attempt to rein in the vastness of the space of [[software]]'s environmental and input variables, both for actual [[programs]] and theoretical descriptions of [[programs]]. Such attempts to improve [[software]] reliability can be applied at different stages of a [[program's]] development, in the case of real [[software]]. These stages principally include: requirements, design, programming, testing, and run time evaluation. The study of theoretical [[software]] reliability is predominantly concerned with the concept of [[Program correctness|correctness]], a mathematical field of [[computer science]] which is an outgrowth of [[Formal language theory|language]] and [[automata theory]]. === Reliability in program development === ==== Requirements ==== A [[program]] cannot be expected to work as desired if the developers of the program do not, in fact, know the [[program]]'s desired behavior in advance, or if they cannot at least determine its desired behavior in parallel with development, in sufficient detail. What level of detail is considered sufficient is hotly debated. The idea of perfect detail is attractive, but may be impractical, if not actually impossible, in practice. This is because the desired behavior tends to change as the possible range of the behavior is determined through actual attempts, or more accurately, failed attempts, to achieve it. Whether a [[program]]'s desired behavior can be successfully specified in advance is a moot point if the behavior cannot be specified at all, and this is the focus of attempts to formalize the process of creating requirements for new [[software]] projects. In situ with the formalization effort is an attempt to help inform non-specialists, particularly non-programmers, who commission [[software]] projects without sufficient knowledge of what [[computer]] [[software]] is in fact capable. Communicating this knowledge is made more difficult by the fact that, as hinted above, even programmers cannot always know in advance what is actually possible for [[software]] in advance of trying. ==== Design ==== While requirements are meant to specify what a [[program]] should do, [[software design|design]] is meant, at least at a high level, to specify how the [[program]] should do it. The usefulness of design is also questioned by some, but those who look to formalize the process of ensuring reliability often offer good [[software]] design processes as the most significant means to accomplish it. [[Software]] design usually involves the use of more abstract and general means of specifying the parts of the [[software]] and what they do. As such, it can be seen as a way to break a large [[program]] down into many smaller [[programs]], such that those smaller pieces together do the work of the whole [[program]]. The purposes of high-level design are as follows. It separates what are considered to be problems of architecture, or overall [[program]] concept and structure, from problems of actual coding, which solve problems of actual [[data processing]]. It applies additional constraints to the development process by narrowing the scope of the smaller [[software component]]s, and thereby — it is hoped — removing variables which could increase the likelihood of programming errors. It provides a [[program]] template, including the specification of interfaces, which can be shared by different teams of developers working on disparate parts, such that they can know in advance how each of their contributions will interface with those of the other teams. Finally, and perhaps most controversially, it specifies the [[program]] independently of the implementation language or languages, thereby removing language-specific biases and limitations which would otherwise creep into the design, perhaps unwittingly on the part of programmer-designers. ==== Programming ==== The history of computer [[programming language]] development can often be best understood in the light of attempts to master the complexity of computer programs, which otherwise becomes more difficult to understand in proportion (perhaps exponentially) to the size of the programs. (Another way of looking at the evolution of programming languages is simply as a way of getting the computer to do more and more of the work, but this may be a different way of saying the same thing.) Lack of understanding of a program's overall structure and functionality is a sure way to fail to detect errors in the program, and thus the use of better languages should, conversely, reduce the number of errors by enabling a better understanding. Improvements in languages tend to provide incrementally what software design has attempted to do in one fell swoop: consider the software at ever greater levels of abstraction. Such inventions as statement, sub-routine, file, class, template, library, component and more have allowed the arrangement of a program's parts to be specified using abstractions such as layers, hierarchies and modules, which provide structure at different granularities, so that from any point of view the program's code can be imagined to be orderly and comprehensible. In addition, improvements in languages have enabled more exact control over the shape and use of data elements, culminating in the abstract data type. These data types can be specified to a very fine degree, including how and when they are accessed, and even the state of the data before and after it is accessed.. ==== Testing ==== {{Main|Software_testing|l1=Software Testing}} '''Software testing''', when done correctly, can increase overall software ''quality of conformance'' by testing that the product conforms to its [[requirement]]s. Testing includes, but is not limited to: # [[Unit Testing]] # [[Functional Testing]] # [[Performance Testing]] # [[Fail over Testing]] # [[Usability Testing]] A number of [[Agile software development|agile methodologies]] use testing early in the development cycle to ensure quality in their products. For example, the [[test-driven development]] practice, where tests are written before the [[Code (computer programming)|code]] they will test, is used in [[Extreme Programming]] to ensure quality. ==== Run time ==== [[Run time]] reliability determinations are similar to tests, but go beyond simple confirmation of behavior to the evaluation of qualities such as performance and interoperability with other code or particular hardware configurations. == Software Quality Factors == {{Expert|date=April 2008}} A software quality factor is a non-functional requirement for a software program which is not called up by the customer's contract, but is nevertheless desirable and enhances the quality of the software program. Some software quality factors are: ; Understandability: The purpose of the software product is clear. This goes further than just a statement of purpose - all of the design and user documentation must be clearly written so that it is easily understandable. Obviously, the user context must be taken into account, e.g. if the software product is to be used by software engineers it is not required to be understandable to lay users. ; Completeness: All parts of the software product are present and each of its parts are fully developed. For example, if the code calls a sub-routine from an external library, the software package must provide reference to that library and all required parameters must be passed. All required input data must be available. ; Conciseness: No excessive information is present. This is important where memory capacity is limited, and it is important to reduce lines of code to a minimum. It can be improved by replacing repeated functionality by one sub-routine or function which achieves that functionality. This quality factor also applies to documentation. ; Portability: The software product can be easily operated or made to operate on multiple computer configurations. This can be between multiple hardware configurations (such as server hardware and personal computers), multiple [[operating system]]s (e.g. [[Microsoft Windows]] and [[Linux]]-based operating systems), or both. ; Consistency: The software contains uniform notation, symbology and terminology within itself. ; Maintainability: The product facilitates updating to satisfy new requirements. The software product that is maintainable is simple, well-documented, and should have spare capacity for processor memory usage. ; Testability: The software product facilitates the establishment of acceptance criteria and supports evaluation of its performance. Such a characteristic must be built-in during the design phase if the product is to be easily testable, since a complex design leads to poor testability. ; Usability: The product is convenient and practicable to use. The component of the software which has most impact on this is the [[user interface]] (UI), which for best usability is usually [[graphical user interface|graphical]]. ; Reliability: The software can be expected to perform its intended functions satisfactorily over a period of time. Reliability also encompasses environmental considerations in that the product is required to perform correctly in whatever conditions it is operated in; this is sometimes termed robustness. ; Structure: The software possesses a definite pattern of organization in its constituent parts. ; Efficiency: The software product fulfills its purpose without wasting resources, e.g. memory or CPU cycles. ; Security: The product is able to protect data against unauthorized access and to withstand malicious interference with its operations. Besides the presence of appropriate security mechanisms such as authentication, access control and encryption, security also implies reliability in the face of malicious, intelligent and adaptive attackers. ===Measurement of software quality factors=== There are varied perspectives within the field on measurement. There are a great many measures that are valued by some professionals, or in some contexts, that are decried as harmful by others. Some believe that quantitative measures of software quality are essential. Others believe that contexts where quantitative measures are useful are quite rare, and so prefer qualitative measures. Several authorities in the field of [[software testing]] have written about these difficulties, including Dr. Cem Kaner [http://www.kaner.com/pdfs/metrics2004.pdf](PDF) and Douglas Hoffman [http://www.softwarequalitymethods.com/Papers/DarkMets%20Paper.pdf](PDF). One example of a popular metric is the number of [[computer bug|faults]] encountered in the software. Software that contains few faults is considered by some to have higher quality than software that contains many faults. Questions that can help determine the usefulness of this metric in a particular context include: # What constitutes 'many faults'? Does this differ depending on the purpose of the software (e.g. blogging software v. navigational software)? Does this take into account the size and complexity of the software? # Does this account for the importance of the bugs (and the importance to the stakeholders of the people those bugs bug)? Does one try to weight this measure by the severity of the fault, or the incidence of users it effects? If so, how? And if not, how does one know that 100 faults discovered is better than 1000? # If the count of faults being discovered is shrinking, how does one know what this means? For example, does it mean that the product is now of higher quality that it was before? Or that this is a smaller/less ambitious change than before? Or that less tester-hours have gone into the project than before? Or that this project was tested by less skilled testers than before? Or that the team has discovered that less faults reported is in their interest? This last question points to an especially difficult one to manage. All software quality metrics are in some sense measures of human behavior, since humans create software[http://www.kaner.com/pdfs/metrics2004.pdf](PDF). If a team discovers that they will benefit from a drop in the number of reported bugs, there is a strong tendency for the team to start reporting less defects. That may mean that email begins to circumvent the bug tracking system, or that four or five bugs get lumped into one bug report, or that testers learn not to report minor annoyances. The difficulty is measuring what is intended to be measured, without creating incentives for software programmers and testers to consciously or unconsciously "game" the measurements. Software Quality Factors cannot be measured because of their vague description. It is necessary to find measures, or metrics, which can be used to quantify them as non-functional requirements. For example, reliability is a software quality factor, but cannot be evaluated in its own right. However there are related attributes to reliability, which can indeed be measured. Such attributes are mean time to failure, rate of failure occurrence, availability of the system. Similarly, an attribute of portability is the number of target dependent statements in a program. A scheme which could be used for evaluating software quality factors is given below. For every characteristic, there are a set of questions which are relevant to that characteristic. Some type of scoring formula could be developed based on the answers to these questions, from which a measure of the characteristic may be obtained. ====Understandability==== Are variable names descriptive of the physical or functional property represented? Do uniquely recognizable functions contain adequate comments so that their purpose is clear? Are deviations from forward logical flow adequately commented? Are all elements of an array functionally related? ====Conciseness==== Is all code reachable? Is any code redundant? How many statements within loops could be placed outside the loop, thus reducing computation time? Are branch decisions too complex? ====Portability==== *Does the program depend upon system or library routines unique to a particular installation? Have machine-dependent statements been flagged and commented? Has dependency on internal bit representation of alphanumeric or special characters been avoided? *The effort required to transfer the program from one hardware/software system environment to another. ====Consistency==== Is one variable name used to represent different physical entities in the program? Does the program contain only one representation for physical or mathematical constants? Are functionally similar arithmetic expressions similarly constructed? Is a consistent scheme for indentation used? ====Maintainability==== Maintainability of a software is highly dependent on the process used to develop it. Assessing the quality of the software maintenance process is done using a software maintenance maturity model [[S3M]]. Assessing the maintainability of the software product is done by assessing four different perspectives:its ANALYZABILITY, its CHANGEABILITY, its STABILITY afer a change, and its TESTABILITY. ====Testability==== Are complex structures employed in the code? Does the detailed design contain clear pseudo-code? Is the pseudo-code at a higher level of abstraction than the code? If tasking is used in concurrent designs, are schemes available for providing adequate test cases? ====Usability==== Is a GUI used? Is there adequate on-line help? Is a user manual provided? Are meaningful error messages provided? Effort required to learn, operate, prepare input, and interpret output of a program. ====Reliability==== *Are loop indexes range tested? Is input data checked for range errors? Is divide-by-zero avoided? Is exception handling provided? *The extent to which a program can be expected to perform its intended function with required precision. ====Structures==== Is a block-structured programming language used? Are modules limited in size? Have the rules for transfer of control between modules been established and followed? ====Efficiency==== *Have functions been optimized for speed? Have repeatedly used blocks of code been formed into sub-routines? Checked for any memory leak, overflow? *The amount of computing resources and code required by a program to perform its function. ==== Security ==== Does the software protect itself and its data against unauthorized access and use? Does it allow its operator to enforce security policies? Are appropriate security mechanisms in place? Are those security mechanisms implemented correctly? Can the software withstand attacks that must be expected in its intended environment? Is the software free of errors that would make it possible to circumvent its security mechanisms? Does the architecture limit the impact of yet unknown errors? security testing is any develop system is about finding loops and weaknesses of the system. ==User's perspective== In addition to the technical qualities of software, the end user's experience also determines the quality of software. This aspect of software quality is called [[usability]]. It is hard to quantify the usability of a given software product. Some important questions to be asked are: * Is the user interface intuitive? * Is it easy to perform easy operations? * Is it feasible to perform difficult operations? * Does the software give sensible [[error message]]s? * Do [[Widget (computing)|widgets]] behave as expected? * Is the software well [[software documentation|documented]]? * Is the user interface [[self-explanatory]]/ [[self-documenting]]? * Is the user interface responsive or too slow? Also, the availability of (free or paid) support may determine the usability of the software. ==See also== *[[ISO 9126]] *[[ISO 15504|Software Process Improvement and Capability Determination - ISO 15504]] *[[Software testing]] *[[Software testing outsourcing]] *[[Quality]]: [[Quality control]], [[Total Quality Management]] *[[Capability Maturity Model]] *[[Software Engineering]] *[[Software Maintenance]] *[[Performance Engineering]] *[[Assertion (computing)]] *[[Splint (programming tool)]] **[[Optimization (computer science)]] **[[Algorithmic efficiency]] **[[Performance analysis]] *[[Order (information processing)|Order]] **[[Programming paradigm]] **[[Programming style]] **[[Software architecture]] *[[Software metric]]s **[[Cyclomatic complexity]] **[[Cohesion (computer science)|Cohesion]] and [[Coupling (computer science)|Coupling]] *[[Standards (software)]] *[[Software reusability]] *[[List of System Quality Attributes]] **[[Accessibility]] **[[Availability]] **[[Dependability]] *[[Security]] *[[Security engineering]] *[[Computer bug|bugs]] *[[anomaly in software]] *[[Software Quality Model]] *[[Software Quality Assurance]] *[[SQO-OSS|Software Quality Observatory for Open Source Software]] ==Bibliography== * International Organization for Standardization. ''Software Engineering — Product Quality — Part 1: Quality Model''. ISO, Geneva, Switzerland, 2001. ISO/IEC 9126-1:2001(E). * Diomidis Spinellis. ''[http://www.spinellis.gr/codequality Code Quality: The Open Source Perspective]''. Addison Wesley, Boston, MA, 2006. * Ho-Won Jung, Seung-Gweon Kim, and Chang-Sin Chung. [http://doi.ieeecomputersociety.org/10.1109/MS.2004.1331309 Measuring software product quality: A survey of ISO/IEC 9126]. ''IEEE Software'', 21(5):10–13, September/October 2004. * Stephen H. Kan. ''Metrics and Models in Software Quality Engineering''. Addison-Wesley, Boston, MA, second edition, 2002. * Robert L. Glass. ''Building Quality Software''. Prentice Hall, Upper Saddle River, NJ, 1992. ==References== <references/> [[Category:Software quality|*]] [[Category:Anticipatory thinking]] [[ar:جودة البرمجيات]] [[de:Softwarequalität]] [[it:Qualità del software]] [[ja:ソフトウェア品質]] [[ru:Качество программного обеспечения]] [[zh:软件质量]]