IT Risk Assessment Report
Topic: Risk Assessment in Software Testing
Risk assessment is a significant IT; no matter what the system we are using, there is a need to do a risk assessment to make the network secure. In this report, we are going to discuss the risk assessment in the software testing phase. Although the field is too broad that we can’t discuss it thoroughly, so we have decided to discuss a particular approach (Risk-Driven testing) of the risk assessment as well as in which specific area we are going to apply it; for this purpose, we have chosen the “Risk in Mobile application development” different scenarios will be discussing for this purpose in the report.
Table of Contents
Testing for the risk assessment is a well-known & reasonable technique to overcome the problem that holds the software's vulnerabilities & threats under test. It's based on focusing the test activities on such scenarios with critical situations for a software system. The testing for the risk assessment of a software application optimizes resource allocation like budget, time etc. that leads in the risks mitigations also use to help identify the critical areas & provides the support to fix that. The approach risk-based testing is also part of the risk assessment (Bubevski, 2010). The significant functionalities hold the identification of the risk, analyze the risk & the evaluation of the danger that overall combines & referred to as the assessment of risk & the risk usage for guiding the test process.
It's because identifying the risk, analysis of the risk & the evaluation of the threat used to determine the risk values significance that is assigned for test & the overall test process quality, these are the core activities of the risk assessment of software testing (Risk-based approach). Whereas several RBT methods are available as well & the International standard of testing technique (ISO/IEC 29119) like process, documentation also considers the risks as a vital part of the planning of the test process (Bubevski, 2010). Besides, it has been found that there is a framework that is proposed by authors for integrating a risk assessment (Felderer, Breu and Haisjackl, 2014).
This report represents the risk assessment that is responsible for surveying the risk out of the software application. This report presents the full overview of risk management & review in the testing period of a software application; here, we have considered the improvement of mobile application and the risks found in it though a methodology of risk driven testing is likewise talked about the threat & vulnerabilities close by the impact and the resultant risk assessment will be utilized for choosing risk mitigation plan with models of the industry.
2. Risk Management Overview
Risk management is the way of investigating exposure to risk and deciding how to deal with such disclosure. Georgetown's risk management process embraces a prescribed procedure approach and spotlights understanding the key risks and overseeing them inside satisfactory levels. It is a shared procedure where plans of risk responses are created by working together with the partners who comprehend the risks and are ready to oversee them (Risk Management, 2017)
The accompanying advances plot the best way to deal with risk management:
Identify the most significant risks emerging from activities on an on-going premise.
Prioritize risks in light of the probability of an event and potential effect.
Implement techniques to alleviate risks
Monitor adequacy of risk management endeavours.
Figure 1: The complete process of Risk Management
When reacting to risks, one can utilize various procedures for dealing with the threat, including:
Avoidance – Avoiding the risk by ceasing the action that produces it.
Acceptance – Retaining the risk (self-protection).
Mitigation – Reducing the probability and effect of a risk (loss control programs)
Transfer – Shifting responsibility to an outsider (protection)
2.1 Extensive Risk Assessment Approaches:
Identify the scope of threats & hazards:
Threat & hazards identifications that impact or may impact your association.
Threat & hazards identifications that impact or may impact your system.
Threat & hazards identifications that impact or may impact the surrounding area.
Determine the potential effect of each threat & hazards by:
Estimating the relative seriousness of each threat & hazards.
Estimating the relative recurrence of each threat & hazards.
Estimating the weakness of each threat & hazards.
Develop procedures to (Prevent, Mitigate, prepare for, respond to, recover from) threats & hazards that affect or may affect your association, furthermore, its people, tasks, property, & environment.
3. Overview of Risk in Software Testing:
Risk-based testing (RBT) is a pragmatic & known approach to address the issue of ever restricted testing resources that have increased much consideration. It depends on the automatic plan to center test activities around those situations that trigger the most basic circumstances for a software system. Its appropriate application may then have a few advantages. RBT advances the designation of resources (spending plan, time, people), means of the risk mitigation, helps identify critical areas as early as possible and supports a decision to the management (Sauve, 2008). Risk-based testing includes the analysis, identification, and assessment of the product under test risks, which are alluded to as a risk appraisal, and the utilization of risks to control the test procedure.
Since risk identification, investigation, and assessment decide the essentialness of the risk value allocated to tests and, along these lines, the test process, they are core activities in each risk-based test process. Even though a few RBT approaches are accessible, and the up and coming universal standard ISO/IEC 29119 on testing methods, procedures, and documentation indeed even requires the thought of risks as an indispensable piece of the test planning process, a structure on the most proficient way to coordinate risk appraisal in a test procedure has not been proposed. However, such a system gives guidelines and support tests, which process managers to build up a risk-based test process concerning the premise of an existing test process (Puri, 2015).
3.1 Techniques Used
3.1.1 Risk Assessment Questionnaire
Self-assessment of testing the software takes place in a questionnaire to identify & overcome the risk during each phase.
3.1.2 Threat and Vulnerability Sources
Assessment of several vulnerability and threat sources identification & modelling was done by following the model given below:
Figure 2: Model of Threat and Vulnerability. (H. Alhazmi and K. Malaiya, 2005)
3.1.3 Review of Documentation
The complete thorough review of existing operational manual, system documentation (Software requirement specification) as well as testing documentation were reviewed under assessment
4. Basic Concepts of Test and Risk Management Processes
The test process contains the core test activities like the planning of test, test configuration of a trial, test usage, the test execution, and test assessment. Test planning is the phase of building or updating a test plan. A test plan is a record portraying the extension, approach, assets, & schedule of a test's regular activities. Amid the test configuration stage, the trial's general testing goals are changed into specific test conditions test cases. Tests are implemented, which contains remaining work like planning test tackles and data of test or making automated scripts for testing, which are essential to empower the execution of the test cases on the implementation level (Schneidewind and Hinchey, 2008). The tests' performance is then placed, and every vital detail of the account is recorded in a test log. Amid the test assessment phase and reporting, whereas the existing criteria are assessed, and the logged test results are condensed in a test report. The development projects commonly contain a few test cycles, and like this, all or a few phases of the test process are performed iteratively.
A process of risk management contains the core activities of risk (identification, analysis, assessment, treatment, and monitoring). In the phase of risk identification, the risk things are recognized. In the degree of risk analysis, the likelihood and effect of risky things and their risk exposure value are assessed. In the risk assessment stage, the centrality of risk is surveyed based on the assessed value of risk exposure. As a result, risky things might be doled out to risk levels characterizing a risk prioritization & classification. In the phase of risk, treatment activities for getting a palatable circumstance are resolved & implemented. If there should be an occurrence of risk-based testing, testing is connected as a measure to treat risks. In the phase of risk monitoring, threats are followed after some time, and their status is accounted for.
Furthermore, the impact of the executed activities is decided. The actions of risk (identification, analysis, assessment, treatment, and monitoring) are regularly, on the whole, alluded to as risk appraisal. In contrast, activities like risk treatment & monitoring are implied as risk control. As in the RBT context, testing is per definition applied for risk control, just risk assessment, i.e. risk (identification, analysis, assessment) must be incorporated into the test process as a separate (Noor and Hemmati, 2015).
4.1 The Concept of Risk:
A risk is the chance of harm, injury, misfortune or a loss and ordinarily dictated by the likelihood of its event and its effect. As it is the shot of a few things happening that will affect the objective, the standard risk formalization depends on the two variables probability (P), deciding the likelihood that a failure appointed to a risk happens, & the impact (I), deciding the expense or seriousness of a loss in the off probability that this will occur in the activity. Scientifically, the exposure of risk R of a personal asset a, i.e., something to which a party relegates value, is resolved given the probability P and the impact I (Felderer, Breu and Haisjackl, 2014).
5. Mobile Application Testing
With the development of millions of mobile applications over the world. It has been noticed that the research is done on the engineering process or, especially, the risk assessment process is minimal, which is necessary during the development of the application (Jing, G, Zhao, and Hu, 2015).
During the development of a mobile application, the dependencies that are encountered are thoroughly discussed below.
5.1 Dependency of Resources:
It has been noticed that any mobile applications, mostly depending on some of the shared resources like servers, memory, etc. The availability of each of these resources results in the functioning of an application properly.
5.2 Dependency of Platform:
At the point of development of an application, it relies on some of the platform's features & must be designed not to break the forum's security. For instance, some parts were upheld in one platform while not on another (like, files of music can't have shared in iOS while in different stages this is possible). To stay away from this situation, we should focus on developing the applications in which the GUI is separate from the core logic with the goal that we require the change just in the GUI part for various OS and the logical parts continue as before (Jing, G, Zhao, and Hu, 2015).
5.3 Investment of the Stakeholder:
Mobile application's development & launch, to a great extent, rely on the support from the stakeholder. Stakeholders assume a pivotal job in funding, planning & advancement of the application. The amount contributed, and the strategies utilized for marketing of the mobile application are of prime significance.
5.4 Technical Risks:
A correct design of an application is a large portion of the work done. In any case, a mobile application configuration devours a considerable measure of time and cash, Whereas frequently turns out to be hazardous over the long haul. Application configuration incorporates the accompanying:
5.4.1 Algorithm: The algorithm utilized for the application ought to have cost ideal and optimal space. The application ought not to possess a considerable measure of memory in the system.
5.4.2 Platform: The Platform on which the application is going to run should be practical and generally steady and secure. The most well-known mobile application platforms are Android, iOS, and Windows.
5.4.3 Graphical User Interface: The application GUI turns out to be a central point in pulling in more crowd ought to be dealt with top generally need. Unique platforms, for example, Android, iOS, RIM, and so on, have their particular manners of keeping an eye on the software methodologies & vulnerability to manage external threats. Improved security check algorithms must be utilized inside the software application to watch out for the security assaults. Legitimate approval and verification of the clients' accreditations alongside sound encryption-decryption algorithms can be demonstrated to anchor a system.
5.4.4 Security Measures: There are a few requirements that rely upon the platform and improvement process, and the client does not legitimately express that. They are exceptionally powerless and, if not mulled over, would lead to some severe flaws when the final product is released.
Moreover, Diminishing the high frequency of mobile application security risks must prioritize those managing the application advancement groups. Dividing the code into small modules, more sensible units can help diminish the number of blunders engineers present when working under the short development cycle stress (Kakkar and Kakkar, 2013). Most essential, however, is giving engineers the training to code safely to actualize security controls, for example, encryption, accurately as well as it will be beneficial for risk assessment purposes.
From the research, it is noticed that for testing a mobile application, the risk-driven approach will be helpful in the next section of the report for risk assessment of any software application, including mobile application, so it must have followed.
6. Risk Driven Testing Approach
Requirements must be appropriately analyzed.
Documents like (SRS, FRS, use cases) are looked on. This action is done to discover & overcome mistakes & ambiguities.
Signing off the requirement is one of the risk assessment strategies applied for keeping away from the project's new changes request. Any progressions to existing conditions after baselining a document include change control & ensuing endorsements.
Risk can be assessed by probability calculation & impact the documented requirement taking characterized criteria resembles cost, plan, assets, scope, reliability & complexity, & so on into consideration.
The probability of the failure is identified & areas of risks that are high. It's possible by utilizing a risk assessment matrix.
Usage of a register of risk to list recognized hazards set. At the same time, updating & monitoring are placed to track the risks occasionally at consistent interims.
At this phase, risk profiling should be done to comprehend the limit of risk & risk resistance levels.
We are prioritizing requirements in order of rating.
The process of risk-driven test is characterized.
Risks that are highly critical & medium must be considered immediately for risk mitigation planning & execution. At the same time, some of the low risks will be mitigated accordingly.
Assessment of risk is done for analysis of data quality.
Plan & characterize test as per the rating.
A proper testing approach is applied and a testing technique for planning the test so that most noteworthy risk items will be tested on priority. Risk items with high risks were tested with the help of complete domain knowledge.
Various techniques of testing were utilized, for example. Using the decision table method on items for the test with high risk & using equivalence partitioning only for general test items with low risk.
Most of the test cases are likewise intended to fulfill various functionalities & complete scenarios of business.
Preparation of test data & conditions.
A complete review of software testing and Test Strategy, Test cases, & Test reports by the testing team are done.
For the detection of a defect & risk mitigation, a peer survey is a vital step.
The need does the execution of the test cases for risk items.
The exit criteria evaluation. All areas of high-risk are tested thoroughly, with just little leftover risk remain as it is.
Risk-driven test results in the analysis of reporting as well as the metrics.
Existing events & new events of risk reassessment given Key Risk Indicators.
A complete backup or contingency plan is created.
Analysis & prevention of defects action is taken to wipe out the flaws.
Retesting & Regression testing for the defect validation given pre-ascertained risk evaluation & areas of high-risk ought to be most seriously secured.
Risk-based automation testing id performed
Calculation of Residual Risk
Controlling & monitoring the risk
For various risk levels, both exit or consummation criteria can be utilized. Every single risk has tended to proper activities or contingency plans of action. Whereas risk exposure is beneath level consented to the worth of task.
Reassessment of risk profiling is done along with feedback of the user of the system.
6.1 Risk Driven Testing Approach to the System Test:
6.1.1 (Technical) System Test– This alluded to an integration & environment test; the environment test incorporates testing in the production of development & the environment.
6.1.2 (Functional) System Test– By testing each functionality, modules, features etc. The main reason for such a test is to check whether the system can meet its predefined requirements.
6.1.3 (Non-Functional) System Test– By testing the non-useful requirements performance, configuration tests, tests for Security, reinforcement & recuperation techniques & documentation (system, installation & operational documentation).
The below figure demonstrates a review of the processes discussed above:
Figure 3: Example of system testing for risk driven approach
Software application Testing incorporates both requirements: Functional & Non-Functional & their tests. Functional testing guarantees that the application meets client & business necessities. Then again, testing non-functional requirements is done to confirm that the application faces a client's desires regarding quality, usability, security & so on (Schneidewind and Hinchey, 2008).
7. Software Testing Process (Risk driven)
It includes the following process:
The Identification of risk
The analysis of risk
Response to risk
The scoping of the test
The test process definition
Figure 4: Risk drove the testing process
7.1 Risk Driven Software Test Process Design Procedure:
A risk register must be prepared, which records the risk from the generic risk list, different checklist etc.
The risk which is associated with the software must include the software application Functional & non-functional requirements.
A unique identifier is allocated to each of the risks.
Probability of risk
A Likelihood of software application that is prone to this mode of failure
The mode of failure impacts the software under test.
Exposure of risk
It's a combination or a product of the probability & consequences.
Effectiveness of test
Is it confirm that the risk can be mitigated with the test?
Priority number of test
It's a combination or a product of the probability, consequences & test effectiveness.
Objective(s) of test
What kind of test objective will be helpful to mitigate the test
What kind of technique or method is to be used
How much time will be taken for the testing process?
Test stage A, B, C-Unit Test, Integration Test, System Test
The persons of a group name that is performing a particular activity.
Table 1: Risk Driven Software test procedure heading with description
For each risk, the consequences & probability are assessed. (1 Low -5 High).
Figure 4: Probability chart
Figure 5: Severity Chart
7.2 Risk Assessment & Prioritization Matrix:
The risk assessment matrix is the impact matrix probability. The team of a project comes to know about the perspective of risks & their priority to tend to each of these risks.
Rating of the risk = Severity x Probability
Probability is the proportion of the possibility of any unexpected event that will happen. Exposure regarding time, vicinity & redundancy. It is expressed regarding the percent-age (Schneidewind and Hinchey, 2008).
This can be named as:
7.2.1 Frequent-This has relied upon to happen a few times by & massive (91 - 100%)
7.2.3 Probable: This has likely to happen a few times much of circumstances (61 - 90%)
7.2.4 Occasional: This might happen at some point (41 - 60%)
7.2.5 Remote – This has unlikely to happen/could happen at some point (11 - 40%)
7.2.6 Improbable-This has happened in uncommon & extraordinary conditions (0 - 10%)
7.2.7 Eliminate- It’s impossible to happen (0%)
Severity is the level of impact of damage & loss caused due to uncertain events. 1 to 4 score & can be named 1) Catastrophic 2) Critical 3) Marginal 4) Negligible
Catastrophic – This is a challenging situation that results in an inefficient project & could even close down the task. During risk management, this should be a top priority.
Critical - The severe outcome that prompts a lot of loss. So, the project is hugely debilitated.
Marginal – This is short damage that can be fixed with rebuilding activities.
Negligible- Less or little loss or damage. Such failure will be observed & overseen by a standard methodology.
So, this is characterized by the four classifications mapped against the severity & probability of risk, as shown in the figure below.
Critical, High, Medium, Low
Figure 4: Matrix of Risk Assessment & Prioritization
Critical: Such risks that are part of this classification during software risk-driven testing are set apart in Amber. The activity must be ceased, & immediate action is taken seclude risk. At the same time, viable controls should be recognized & implemented. Moreover, the story shouldn't continue unless the risk is decreased to a low or medium level.
High-risk class risks during software risk-driven testing are set apart in colour Red ate activity or risk management methodologies. A quick move must be made to seclude, kill, substitute the risk & to execute adequate risk controls. If these issues can't be settled instantly, rigorous courses of events should have characterized for recognizing these issues.
Medium: The kind of risks that are part of this class during software risk-driven testing is set apart in colour Yellow. Practical & reasonable measures with immediate effect taken to overcome the threat.
Low: The low risks during software risk driven testing are set apart in colour green it can be taken lightly, as they don't represent any significant issue.
7.3 Results of Risk Driven Testing Reporting & Metrics
7.3.1 Preparation of Test Report for Software Applications:
The test reporting of a software application under test is about successfully conveying the results of tests to a software application stakeholder. Furthermore, to give a reasonable comprehension & to demonstrate, we consider examining test results with the trial's objectives following things in risk assessment (Noor and Hemmati, 2015).
Total Executed test cases out of planned ones
Total Passing & failed criteria of test cases
The identified defects counting as well as their status & severity
Fundamental defects counting that are still open.
Condition downtimes – assuming any
Test Summary & coverage report
7.3.2 Preparation of the Matrix:
Combining some measures (two or more) that are used to compare the processes, projects & products of the software under testing for the risk assessment to prepare a risk matrix.
Variation of schedule & efforts
Productivity in the preparation of test cases
Design coverage of a test
The productivity of the execution of test cases
Percentage Efficiency of the Risk Identification
Percentage Efficiency of the Risk Mitigation
Percentage Effectiveness of test
Execution coverage of the test
Execution productivity of test
Percentage of defect leakage
The efficiency of defect detection
Stability index of requirements
The risk analysis in the NFR (Nonfunctional requirements) of a software application that are usually performance, usability, security & reliability do base on the status of the defects & the status of test pass & fail tests as well as their correspondence towards the risk
The risk analysis in the functional categories metrics of testing, a group of defects & status of test pass or fail concerning their relationship to the dangers (Sauve, 2008).
Create the early warning indicators by identifying key lead & lag indicators and continuous monitoring & reporting of these key risk indicators by the data patterns & interdependencies analysis.
7.4 Risk Assessment (Inherent Risk Vs Residual Risk)
The process if risk management also include risk (Inherent, Residual, Secondary & recurrent)
7.4.1 Inherent Risk: Before the controls & responses were identified, such risk existed in the system.
7.4.2 Residual Risk: After implementing the controls & responses, here lies the excess risk, which is also known as net risks.
7.4.3 Secondary Risks: By the implementation of the risk resource plan, the new risk caused.
7.4.4 Recurrent Risk: the initial risk occurrence probability.
The test result is based on the risk used to help the team know about the residual level of quality risk during the execution of a test & to make some decisions based on the smart release.
7.4.5 Risk Profiling: Risk profiling must be done for finding the client's optimal level of investment the risk by considering the risk (required, capacity & tolerance). Here, the trouble required is a risk level that the client needs to take to obtain a satisfactory return. The Risk capacity is the financial risk level that is afforded by a client to take. At last, the risk tolerance is the risk level, which the client prefers to take (Sauve, 2008).
No one can test each & everything of a software application; moreover, testing until it is good enough is also no suitable sufficient. There's always a risk of vulnerabilities & threats. So understanding the business needs of software application, a testing approach must be defined & followed that ensures the best testing criteria; it is simple to know that we can't control everything, so by following a simple approach, we can control the SDLC process to do testing properly by minimizing the risk. This report focuses on the risk assessment of a mobile application by using a risk-driven approach; the approach is thoroughly discussed. Overall, it is concluded that it is best to follow the risk-caused method for risk assessment, no matter what software application is under test.
Bubevski, V. (2010). An Application of Six Sigma and Simulation in Software Testing Risk Assessment. 2010 Third International Conference on Software Testing, Verification and Validation. doi:10.1109/icst.2010.23
Felderer, M., Breu, R. and Haisjackl, C. (2014). A Risk Assessment Framework for Software Testing. [online] Available at: https://www.researchgate.net/publication/282182753 [Accessed 28 Sep. 2018].
Puri, S. (2015). A Risk Assessment Framework to Reduce Risk Level and Optimize Software Quality. SAMRIDDHI: A Journal of Physical Sciences, Engineering and Technology, 1(1).
H. Alhazmi, O. and K. Malaiya, Y. (2005). Quantitative Vulnerability Assessment of Systems Software. RAMS 2005. (H. Alhazmi and K. Malaiya, 2005)
Jing, Y., Ahn, G., Zhao, Z., & Hu, H. (2015). Towards Automated Risk Assessment and Mitigation of Mobile Applications. IEEE Transactions on Dependable and Secure Computing,12(5), 571-584. doi:10.1109/tdsc.2014.2366457
Kakkar, M. and Kakkar, K. (2013). Risk Analysis in Mobile Application Development. [online] Available at: https://www.researchgate.net/publication/271460333 [Accessed 28 Sep. 2018].
Noor, T., & Hemmati, H. (2015). Test case analytics: Mining test case traces to improve risk-driven testing. 2015 IEEE 1st International Workshop on Software Analytics (SWAN). doi:10.1109/swan.2015.7070482
Sauve, J. (2008). Risk-based service testing. 2008 3rd IEEE/IFIP International Workshop on Business-driven IT Management. doi:10.1109/bdim.2008.4540081
Schneidewind, N., & Hinchey, M. (2008). Risk-Driven Software Reliability and Testing. 2008 Second International Conference on Secure System Integration and Reliability Improvement. doi:10.1109/ssiri.2008.34
Risk Management Guide for Information Technology Systems. (2017.). Retrieved from https://www.hhs.gov/sites/default/files/ocr/privacy/hipaa/administrative/securityrule/nist800-30.pdf