CTO: Hey guys. I just got back from our monthly meeting with Dave the CEO. Boy is he furious. I actually thought at one point his moustache would catch on fire.
Lead Developer: Wow. What happened? Did you guys lose the backups again?
CTO: No, worse! During a recent demo our new UI broke again. Dave was irate because we, and by that of course I mean you, assured him that the software was ready for production. Now we might lose a major new client. And of course he is blaming me. I thought we had fixed all those bugs on staging? I am sick and tired of every single release we have being a huge bag of bugs. I can't even sleep at night.
Lead Developer: Wow that’s terrible. I'm sorry. You know, I think we should reconsider that decision we took at the kickoff not to hire a QA expert. And likewise, by we of course, I mean you and the CEO.
CTO: What? Again with this QA expert? Hire more people? We already have like 5 developers on staff. Why do we need to hire someone else? Why can't you guys just start writing more unit tests like they said at that developer conference last May? These problems are killing the company morale and worrying the stakeholders. Everyone hates us right now. They think we are incompetent morons with no attention to detail.
Lead Developer: Well, our recent move towards TDD and full test coverage has really helped our backend code, but it’s not always feasible on the front end for UI testing. To help with that we started doing automated testing with Selenium. We thought that would be enough. But I guess we still need more test cases to reach 100% coverage. But even then it still feels like we are missing something. We don't do any manual testing or even have an overall system testing strategy in place. I think we may need to bring in extra competence for that.
CTO: Really? Okay. I’ll go talk to the boss. You guys are doing a good job on most stuff, but these bugs have just got to be stopped. Now please get working on a new build without those nasty UI bugs! Maybe we can still salvage this client if we get a patch out soon enough.
Lead Developer: Okay, I hear you, and rest assured that we're on it. I won't go home until these new bugs are fixed. But please get us a QA person. Even a part-time consultant will do for now.
1 month later there is a new meeting with an external QA expert consultant
QA Expert: Hi! My name is Mary Jenkins and I am a QA specialist from Bugfree Enterprises. I heard you guys have had some problems with your Quality Assurance and testing strategy? Is that true?
CTO: Yes, well we don’t even really have an overall QA strategy in place. But we are interested in hearing about best practices and possibly implementing any reasonable proposals you can give us.
QA Consultant: Okay I see. That's not so rare. Most managers view QA as simply a “final additional task” to give the developers and not as a holistic process from start to finish.
CTO: Yes exactly. Holistic. We need some more holistic thinking!
QA Expert: No, that won't be necessary. We don’t need candles and incense to be holistic in this case. We just need to change our preconceptions a little and implement a few basic new processes.
CTO: Okay. Will this be expensive?
QA Expert: No. Quite the contrary. QA should actually save everyone time and money in the medium to long term. And QA gives everyone more peace of mind and pride in their work. You won’t necessarily even have to hire any additional staff members.
CTO: Thank God! The CEO will love that. Can you give us a quick overview of what we need to do? We can discuss the details later.
QA Expert: Sure. Basically, I like to divide my overall testing strategy into two categories: white box testing and black box testing. The developers will be in charge of white box testing, which includes unit testing, integration testing, regression testing and production error monitoring.
Lead Developer: Great, we are already doing most of that.
QA Expert: Cool. Now you need to do the rest. I can help educate your team on what's missing.
Lead Developer: Thank you. I knew this was good idea to bring in an expert.
CTO: Okay. So what about the black box testing? Can the developers do that part too?
QA Expert: Uh, no sorry. The black box testing should be done by non-developers. Mostly they will focus on smoke testing and functional testing. And then select customers with early access would be ideal to perform the acceptance testing. That's the basics right there. Just implementing those steps should really improve things.
CTO: Wait… you want non-developers and even the customers to test the software? Is that what you mean by a holistic approach?
QA Expert: Yes, quality is everyone’s responsibility. It’s easy to blame developers for all the problems, but software is so complex and important nowadays that it’s impossible for one division to be solely responsible for building and testing it.
Lead Developer: Wow, I get it now. Holistic. Even the customers get involved. It's brilliant! This is really eye-opening. I feel like this is all starting to make sense now.
CTO: Yeah I must admit it's a bit of a revelation to me as well. I can't wait to tell the CEO. This might be just what we need. Get the customers involved, wow!
Lead Developer: Okay great, but wait! I think we should honor this zen moment by splurging on some incense and candles for the meeting room. What harm could that do?
CTO: None at all I suppose! Go for it.
Lead Developer: Great. I'll be back in 15 minutes, I will get some donuts as well while I am out. They are my holistic pastry of choice.
-- EVERYONE LAUGHS --
- What formal testing processes did you learn about at work or school? Did you learn about any of the techniques discussed here? Which ones were missed in the reading?
- Quality Assurance involves many process-oriented activities. Can you name some of them? Which ones would you be good or bad at? Why?