- Design Tools – QA and testing are design tools, and the purpose of software testing is to design systems that are deterministic
- Efficiency-to-Thoroughness-Trade-Offs – (ETTO) We do not always pick the best option, we pick the one that best meets the immediate needs
- Ironies of automation – Automation makes things more complex and, while tools can make the process safer or faster, they cannot make things simpler
- Hawthorne Effect - Productivity (temporarily) goes up when you get a new process or tool
- Goodhart’s Law – Simplified for the tutorial, the law states that people will game the system. Period.
- Diseconomies of scale – The opposite of economies of scale, producing services at an increased unit cost
- Conway’s Law – Simplified for the tutorial, the law states that software looks like your organization
- Bikeshedding – It’s hard to build a complex, multipart system, but building a bike shed is easy, so organizations tend to spend too much time on trivial items
Automated MonitoringIn 2007 it was proposed by an engineer at Google that sufficiently advanced monitoring is indistinguishable from testing. This statement highlights the relationship that exists between monitoring and testing, and we can certainly use advanced monitoring to help us in our testing efforts. For example, we can use statsd as a means of instrumenting production code to gather high-volume data with minimal or no performance impact. The statement also highlights the issue of monitoring vs. testing. Noah provided a list of four things we should be doing as part of our monitoring efforts:
- We should monitor all things
- We should build real-time dashboards
- We should deploy continuously
- We should fix production bugs on the fly
We should perform these four things, keeping in mind that while monitoring does provide visibility into implementation, it has nothing to do with design, and so does not replace QA and testing because they are design tools. Thus, while monitoring and testing are both necessary, it is only when practiced jointly that they are sufficient.