Software assurance is the process of making sure that software:
- Performs all of the intended and required functions
- Does not perform unintended/unauthorized functions
- Does not contain vulnerabilities
- Is free from errors
One of the big issues with software assurance is that many analysts consider security requirements to be “non-functional.”
Functional requirements are the things the system needs to do. For example, a system might need to handle a customer purchase correctly. The software must be written to accomplish these functions.
Nonfunctional requirements refer to the characteristics of the overall form, or attributes of the system’s behavior. i.e. the attributes that result from the way in which it is designed and built. If requirements are considered nonfunctional, no specific code is written, as there’s no function to carry out. (note: some security features do require specific code).
Software assurance provides confidence that the software can meet stakeholders’ needs and essentially becomes a warranty of fitness. It indicates how well software meets the stated functional requirements. Typically it is done through organized workflows, and can be accomplished in various different ways.
One way to provide software quality assurance is through conducting software security risk assessments. There are four categories mentioned in the CBK:
1. Certification and accreditation (C&A)
This method looks at requirements and design documentation to see if security functionality is done correctly and validated against baselines. During certification, analysts look at the system owner’s response about how a control is implemented and determine whether the list of controls are met, not met, or partially met. Accreditation is the process of obtaining authority to operate the system, typically accompanied by the certification analysis. Acceptance testing leverages the C&A as a standard to develop use cases. This method is falling out of favor because it is costly and not as effective as other methods.
2. Risk management frameworks
Using frameworks such as NIST, ISO, or other risk management frameworks to evaluate the system can provide a bit more assurance than the C&A process.
3. Software process capabilities maturity models (or CMMs)
These models can be used to look at all business processes end-to-end in terms of their reliability, repeatability, and achievement of outcomes that meet/exceed specified quality metrics. One specific method mentioned in the CBK is the OWASP Software Assurance Maturity Model (SAMM) that focuses on the use of CMM processes and thinking to achieve greater quality, stability, and security for software systems. It’s an open framework approach for measuring and improving security in software development.
The OWASP SAMM Model is guided by the business functions:
Governance
Design
Implementation
Verification
Operations.
It can be tailored to the specific needs of each organization.
OWASP also has a new DevSecOps Maturity Model (DSOMM) that brings CMM thinking to DevSecOps.
4. Other software assurance methods
This last category is big and encompasses a lot of methods. Similar to C&A, these methods establish how well a software system meets its requirements while not exposing the organization and its users to undue risks.
a) Testing, Verification, and Security Assessments are processes of using controlled, scripted set of conditions, and using inputs to those conditions in order to produce required results, or results to be compared with desired results. Testing can be done at many different steps during the development process. Certain things like repairs, maintenance, and updates to any system component can require the system to undergo further testing. Ongoing security assessments can help determine if the system is still secure. Ethical penetration testing evaluates the present state of the system, and can be used throughout the operational life span of the system.
b) Acceptance Testing is a formal testing process conducted by the end-users to determine whether the system satisfies its acceptance criteria and to allow the data owner or customer to accept or reject the system. This was originally called “functional testing” because each acceptance test examines the functionality of the application to ensure it does what it is supposed to. A “case” is used to represent a specific operating condition of the environment of feature, and typically results in a pass or fail outcome.
c) Unit tests are written and carried out by the developers themselves.
d) Regression testing looks at applications to make sure that a change hasn’t affected existing functionality in a negative way. Its purpose is to catch code or bugs that may have been accidentally introduced into a new build of software (release candidate) and to prevent bug recurrence. Security-specific regression testing can also be used to in the same fashion, by re-running previously written test scenarios to verify that any modifications have not re-exposed the system to any known vulnerabilities from the past.
e) Known-good data should be used to ensure the system is acting like it should, with data that is expected. It should also contain a wide range of data inputs, including known-bad data, or unexpected data, also called “fuzzy” data.
Note: never use live production data to conduct testing unless the data owner has formally approved it, and provided the conditions of its use, and what might constitute misuse. Production data can be sanitized to mitigate risk of exposure. Also the production environment should be segregated from the testing environment. Data validation should be conducted after each test to ensure that the data hasn’t changed after each test. Bounds checking can be used to verify that a variable is within the proper bounds for its field or data type (e.g. “age” field must be 18 or higher for voter registration)
f) A Software Assurance Policy can also be used. It should be well-written and easily accessible. It should also be easy to understand.
g) Software Assurance During Acquisition can be done using five basic phases when obtaining software from third-party systems providers, integrators, or consultants. Security professionals can advise at each of these phases and activities.
Planning Phase
Letters are highlighted to indicate the accompanying mnemonic device. Planning contains several activities, which include:
Needs assessment – obviously this includes identifying the need for the project, its purpose, etc.
Magnitude assessment (identifying the scope of work)
Risk assessments are completed, to identify the risks of not doing, or doing the project.
Identifying Alternative implementations – what other options are there?
A return on investment (ROI) analysis is done, which looks at a break-even or hurdle rate (projects that do not return more than the hurdle rate must be overridden to be approved).
R – Developing software Requirements to be included in the statement(s) of work (SOWs) of the contract(s).
D – Identifying the total list of Deliverable products, including data, documentation, test data, software assurance records, configuration management and control data.
AS – Creating an Acquisition Strategy, to include screening bidders.
BEC – Developing Bid Evaluation Criteria with an evaluation plan.
Contracting Phase
RFP – request for proposal, or solicitation of contractor’s business.
Evaluate Proposals – looking at the RFPs to see who might be a good fit.
Finalizing – awarding the contract and negotiating changes to the terms. Software risks should be addressed and mitigated through terms and conditions, and mitigation requirements in the statement of work (SOW).
Monitoring, Acceptance and Deployment Phase
S – work Schedule; this involves establishing and agreeing to the contracted work schedule
CC – Change and Configuration control procedures must be agreed on
AD – Accepting Deliverables includes reviewing them for accuracy and ensuring that they meet requirements.
Ongoing Use and Support Phase
Maintaining the software, and sometimes called the sustainment phase.
Sustainment – managing risk, assurance case management, and change management.
Disposal/Decommissioning – self explanatory.
Follow-on Phase
MR – Managing the Risk
AC – analysis of the Assurance Case and adjusting to mitigate any changing risks.
Here is the video mnemonic, and the table/chart to help you memorize. We’ve also included this in our memorization sheet.
For third-party software development, security practitioners need to ensure that software assurance has been factored into the developer’s capabilities at a level sufficient to protect their organization.
For open-source code, assurance is a bit trickier. Be aware of Linus’s Law, which says that with enough eyeballs looking at the code, all bugs will be revealed. The flaw in Linus’s Law is that it provides a false sense of security and cannot possibly identify all potential security flaws.
Orphaned system refers to systems or components that rely on unsupported elements. An example would be an antiquated but functional medical system such as a heart monitor that is no longer made, and is dependent on a Windows XP operating system. The difference between orphaned and legacy systems is that legacy systems are supported in-house, or a fix can be found for any issues that pop up. Some level of assurance can still be achieved on orphaned systems by conducting ongoing security assessments.