Anyone who works in software has GOT to listen to this podcast. It does an excellent job of expressing so many of the quality and security frustrations that I've felt for so many years.
http://itradio.com.au/?p=189
http://itradio.com.au/?p=189
20080918
20080812
Top 10 Lame Developer Excuses
It's amazing the logical fallacies found in the excuses that are sometimes concocted by otherwise good and logical developers. The following are some of popular excuses I've been given for not fixing defects along with my response to the excuse.
10- It works on my machine. ->Are you going to ship your machine to the customer? The product needs to be installable and usable on other machines. This statement implies that there's something wrong with the tester or their configuration. Developers should not feel that this statement absolves them of any responsibility in helping to investigate and resolve the matter. Whatever the root cause of the problem is, it must be dealt with in some way (i.e. fix the bug, document the misunderstanding, make the install robust enough to recognize when prerequisites are not present, etc.). The fact that a tester encountered the problem means that customers WILL also encounter the problem. Customers can't write a bug or contact the developer when they encounter this problem, so the issue needs to be resolved so that either the user doesn't encounter the problem, or else the workaround is readily apparent and satisfactory to the customer.
9- Working as designed. ?You designed it to lose data and be unusable? A professional Quality Assurance person does not write frivolously bugs. Please try to trust us on this and we'll try our best to not ever give you any reasons not to trust us. If we didn't think that it was a violation of customer expectation for the product to behave in the way that we describe, we wouldn't have wasted our time writing up the defect report. Testers have a unique perspective into the product and the products use that Developers just can't see no matter how hard they try. It's just a result of the creation process, that you can't see it when your baby is ugly. When you have invested so much into making a product WORK, it's difficult to see all the ways that it might FAIL. Please trust us as your objective eyes on these things and don't give us a hard time about fixing them when we have spent the time and effort to identify them.
8- Have you tried it with *? Where * is some other product or upgrade with which there is no reasonable cause to believe that it would effect the bad behavior that is being witnessed. ->This is just a stall to assign the bug to someone else because the developer doesn't feel that they have time to investigate the issue.
QA professionals are not developer helpers. Trust us that we have investigated the problem before writing the defect report, far enough to determine that the problem lies in your component.
7- There's nothing I can do to fix the problem, the problem is in our depended library X or product Y. ->By virtue of using someone elses libraries, we take on certain responsibilities for working gracefully around their warts. If you aren't willing to make some compromises, and go to certain lengths to minimize problems in a library or other software that you take advantage of in your development (which there will ALWAYS be in any software product), then you shouldn't be using it and you should write your own version. You can't just throw your hands up in dispare and blame the problem on someone elses software when issues are encountered. End users are not going to care that it's the fault of xyz library, all they care is that they got software from your company and it had this problem.
6- Is a user REALLY going to do ___? ->If I didn't think a user might do ___, I wouldn't have tried it and I wouldn't have written the bug. If that isn't enough to convince you, then listen to this, if there's one thing that is sure, it is that you CANNOT predict WHAT a user will do. In a universe of infinite possibility given enough time, users will eventually try everything that is possible. My experience has shown that they often try those far out actions that break the software, far more often than we expect.
5- (Security Issues) Why should we close the back door when the front door is wide open. I've actually heard the argument phrased with just that analogy. I think that the analogy holds true, because if you were attempting to secure your house with that attitude, you would never lock either door, because the other one would still be open. Regardless of what doors are open, we need to deal with each of them, as we encounter them. The fact that you can identify two problems should be more of a motivator to spend more effort on security, rather than an excuse to do nothing.
I do not expect to see bulletproof-secure software anytime within my lifetime. The problem is just too difficult with any non-trivial software products. The best game plan we have now and for the forseeable future is to perform due dillignece and make our software difficult enough to hack that the bad guys will take some other avenue (someone elses weak software, social engineering). In order to do this, you need to fix every security bug, to the best of your ability, whenever one is identified.
4- It's too late in the game to consider fixing that bug / Are you going to hold shipping for that. First of all, in most organizations it's not for a mere developer or tester to decide when the actual ship date will be. In a well managed software development process, there are many factors that go into determining the "readyness" of the software and many peoples input that goes into when it will ship. The findings and opinions of QA are one of the major determiners. Next, the issue needs to be dealt with in some way, even if it's merely to add it to the documentation or the list of known issues in the readme. Sometimes I've found that when forced to write the documentation about a bug that developers were "too busy to fix", they somehow find the time to fix it afterall, when they see how embarrassing it is to admit that the problem was allowed to exist in the product and they realize how long it takes to describe to a user the steps required to workaround that problem.
3- Can you identify any customers that are asking for this feature? ->In our company we do not make our living from writing contracted custom software. We write "Commercial Off the Shelf" software and as such, we try to anticipate customers needs and expectations and craft software that is most usable and most satisfactory to the most people. As soon as a customer has to come and ask us for a change in our software just so that they can use it, we have failed. It is NOT a normal flow of typical off-the-shelf software design.
If you still can't see the feature as anything but fluff, take it to the PM or to Support, someone who is even more in tune with the customers than QA and let them judiciate it. We should not have to wait for a customer to tell us what changes they need before we take actions to improve our software.
2- That's not a bug report, it's a feature request. ->Well, sometimes that can be debated, and sometimes you're right; occasionally we may slip up and try to sneak in new features under the guise that it's a bug. However, there are a couple of criteria (maxims) that we use when defining a bug that developers should be aware of before accusing testers of committing feature-creep-through-the-bug-list.
(1) A product must meet reasonable user expectation. If it doesn't do something that a reasonable user would naturally expect it to do, then, the fact that the feature is missing IS A BUG in the product. Even though the requested feature is something that you did not plan to include, it becomes a bug when customers assume that the product must be broken because this expected feature does not work (when every other software product does this). e.g. A text field that doesn't support copy/cut/paste. Auto-formatting SSN and phone number data. Sort by on primary data columns.
(2)
1- In all my __ years as a developer, I've never done it that way; I'll be damned if I'm going to start now. ->You WILL literally be damned if you refuse to adapt and improve, despite good sense, when the opportunity presents itself. Just because something is new to you does not mean that it is wrong. The software industry is still too new and things are still developing so rapidly, to take that kind of attitude before hearing something out is nothing short of presumptuous and pompous.
10- It works on my machine. ->Are you going to ship your machine to the customer? The product needs to be installable and usable on other machines. This statement implies that there's something wrong with the tester or their configuration. Developers should not feel that this statement absolves them of any responsibility in helping to investigate and resolve the matter. Whatever the root cause of the problem is, it must be dealt with in some way (i.e. fix the bug, document the misunderstanding, make the install robust enough to recognize when prerequisites are not present, etc.). The fact that a tester encountered the problem means that customers WILL also encounter the problem. Customers can't write a bug or contact the developer when they encounter this problem, so the issue needs to be resolved so that either the user doesn't encounter the problem, or else the workaround is readily apparent and satisfactory to the customer.
9- Working as designed. ?You designed it to lose data and be unusable? A professional Quality Assurance person does not write frivolously bugs. Please try to trust us on this and we'll try our best to not ever give you any reasons not to trust us. If we didn't think that it was a violation of customer expectation for the product to behave in the way that we describe, we wouldn't have wasted our time writing up the defect report. Testers have a unique perspective into the product and the products use that Developers just can't see no matter how hard they try. It's just a result of the creation process, that you can't see it when your baby is ugly. When you have invested so much into making a product WORK, it's difficult to see all the ways that it might FAIL. Please trust us as your objective eyes on these things and don't give us a hard time about fixing them when we have spent the time and effort to identify them.
8- Have you tried it with *
QA professionals are not developer helpers. Trust us that we have investigated the problem before writing the defect report, far enough to determine that the problem lies in your component.
7- There's nothing I can do to fix the problem, the problem is in our depended library X or product Y. ->By virtue of using someone elses libraries, we take on certain responsibilities for working gracefully around their warts. If you aren't willing to make some compromises, and go to certain lengths to minimize problems in a library or other software that you take advantage of in your development (which there will ALWAYS be in any software product), then you shouldn't be using it and you should write your own version. You can't just throw your hands up in dispare and blame the problem on someone elses software when issues are encountered. End users are not going to care that it's the fault of xyz library, all they care is that they got software from your company and it had this problem.
6- Is a user REALLY going to do ___? ->If I didn't think a user might do ___, I wouldn't have tried it and I wouldn't have written the bug. If that isn't enough to convince you, then listen to this, if there's one thing that is sure, it is that you CANNOT predict WHAT a user will do. In a universe of infinite possibility given enough time, users will eventually try everything that is possible. My experience has shown that they often try those far out actions that break the software, far more often than we expect.
5- (Security Issues) Why should we close the back door when the front door is wide open. I've actually heard the argument phrased with just that analogy. I think that the analogy holds true, because if you were attempting to secure your house with that attitude, you would never lock either door, because the other one would still be open. Regardless of what doors are open, we need to deal with each of them, as we encounter them. The fact that you can identify two problems should be more of a motivator to spend more effort on security, rather than an excuse to do nothing.
I do not expect to see bulletproof-secure software anytime within my lifetime. The problem is just too difficult with any non-trivial software products. The best game plan we have now and for the forseeable future is to perform due dillignece and make our software difficult enough to hack that the bad guys will take some other avenue (someone elses weak software, social engineering). In order to do this, you need to fix every security bug, to the best of your ability, whenever one is identified.
4- It's too late in the game to consider fixing that bug / Are you going to hold shipping for that. First of all, in most organizations it's not for a mere developer or tester to decide when the actual ship date will be. In a well managed software development process, there are many factors that go into determining the "readyness" of the software and many peoples input that goes into when it will ship. The findings and opinions of QA are one of the major determiners. Next, the issue needs to be dealt with in some way, even if it's merely to add it to the documentation or the list of known issues in the readme. Sometimes I've found that when forced to write the documentation about a bug that developers were "too busy to fix", they somehow find the time to fix it afterall, when they see how embarrassing it is to admit that the problem was allowed to exist in the product and they realize how long it takes to describe to a user the steps required to workaround that problem.
3- Can you identify any customers that are asking for this feature? ->In our company we do not make our living from writing contracted custom software. We write "Commercial Off the Shelf" software and as such, we try to anticipate customers needs and expectations and craft software that is most usable and most satisfactory to the most people. As soon as a customer has to come and ask us for a change in our software just so that they can use it, we have failed. It is NOT a normal flow of typical off-the-shelf software design.
If you still can't see the feature as anything but fluff, take it to the PM or to Support, someone who is even more in tune with the customers than QA and let them judiciate it. We should not have to wait for a customer to tell us what changes they need before we take actions to improve our software.
2- That's not a bug report, it's a feature request. ->Well, sometimes that can be debated, and sometimes you're right; occasionally we may slip up and try to sneak in new features under the guise that it's a bug. However, there are a couple of criteria (maxims) that we use when defining a bug that developers should be aware of before accusing testers of committing feature-creep-through-the-bug-list.
(1) A product must meet reasonable user expectation. If it doesn't do something that a reasonable user would naturally expect it to do, then, the fact that the feature is missing IS A BUG in the product. Even though the requested feature is something that you did not plan to include, it becomes a bug when customers assume that the product must be broken because this expected feature does not work (when every other software product does this). e.g. A text field that doesn't support copy/cut/paste. Auto-formatting SSN and phone number data. Sort by on primary data columns.
(2)
1- In all my __ years as a developer, I've never done it that way; I'll be damned if I'm going to start now. ->You WILL literally be damned if you refuse to adapt and improve, despite good sense, when the opportunity presents itself. Just because something is new to you does not mean that it is wrong. The software industry is still too new and things are still developing so rapidly, to take that kind of attitude before hearing something out is nothing short of presumptuous and pompous.
20080130
Corporate Requirements
When creating new software or evaluating the quality of software, there are certain requirements of the company that the software needs to adhered to. I've started to put together a list of these "Corporate Requirements". I hope for this to be a living document; I encourage readers to add other items that they see that belong on this list.
- Copyright date in all License Agreement documents (EULAs) reflect the year of the last "significant" work on either the software or the document itself. Mere bug fixes are not considered "significant" and do not require updating the copyright year.
- Any non-compiled code or documentation must be copyrighted. This includes anything that is semi-human-readable and part of the distribution (scripts, XML files, examples, etc.).
- Section 508 compliance (accessability testing). For any software that might be sold to any government agency.
- I18N testing. Even for software that is not going to be localized, it will be run on non-English Operating Systems and so it still must be able to handle double-byte characters and different code pages.
- Export compliance. For any software which will be available in any way to users outside of the country, the software needs to undergo applicable Export Compliance process for that country. At a minimum this will require a statement explaining what kind and strength of cryptography is required by or used inside of a product.
- Branding. Correct corporate icons, look-and-feel and other eye-candy. Company or product icons need to appear in installers and in the Add/Remove programs window... Company name (with Trademark symbol) needs to be on everything related to the product.
- Security testing and vulnerability assessments. The value of a software company's reputation is often greater than the sum of it's physical assets. Protecting goodwill and reducing possibilities of lawsuits make security testing essential in this age.
- Copy protection and licensing testing. The company software needs to have a reasonable amount of protection from piracy.
- Uninstallability. Anything that can be installed must be able to be completely uninstalled. See the reasons for goodwill and legal protection in the "Security testing" item, above.
- Usability. See the "Bill of User Rights" blog posting for details and see above for reasons.
- Standard versioning conventions. The way that products are versioned should be consistant throughout the company and should roughly follow the standards of the industry.
- Standard naming conventions. For example, on Windows, There must be an entry in Add/Remove Programs, the company name should appear with the product name.
- Software should adhere to good software industry standards. For example: on Windows, logging for services should be done through the Windows Event Log.
- Delivery. Standard corporate packaging and delivery should be adhered to. If other products for Linux are compressed using .tgz format, do not arbitrarily use a tar.bz2 format.
20071218
End-User Bill of Rights
When considering product usability issues, testers should take the stance that these issues are not merely "Requests For Enhancement", but are violations of the end-users inalienable rights to workable and usable software. With that in mind, I've started a list of the "Bill of Rights" for End-Users, which are those things that I see, which testers should have in the forefront of their minds when performing usability testing.
These rights are not features, they are not version 2.0 enhancements, they are the inalienable and intrinsic to every user of our software.
These rights are not features, they are not version 2.0 enhancements, they are the inalienable and intrinsic to every user of our software.
- The end user has a right to change their mind.
* Anything that can be installed must be able to be easily and completely (cleanly) uninstalled restoring the system to it's previous state without damage to other dependent software.
* Users have the right to use the installer program to not only install and uninstall the program, but also to reinstall and repair.
* Anything that can be configured can be unconfigured...
* Anything added can be removed... - Users have a right to simplicity. They should not have to know about inter-library dependencies and the like. The install should take care of all of these things auto-magically behind the scenes for the user, or when that is not possible it should explain clearly to the user in simple terms, what is needed and how the user can go about getting it.
- Users have a right to quiet control. Do not pop up windows in front of a user and take the focus. Users have other things in their world besides your crappy application; they don't want to be interrupted every time your application wants attention. Do not ever capture and mess with control keys
.
- Users have a right to non-interference. Do not change data that is under the users control. For example, spelling and grammar corrections should be available to a user but should not be the default (e.g. iPhone keyboard autocorrection). As another example, never, ever, mess with the value in the clipboard (e.g. Firfox Address Bar). The clipboard is in the users space and belongs to the user, even though you have the means, you do not have the right to copy anything into the clipboard unless explicitly instructed by the user.
- Users have a right to privacy and peace of mind. Never scan the file system, never place anything in the filesystem or registry that a user may not be comfortable being accessed by others. Do not send ANY user data out to the network without notifying and gaining approval from the user.
- Users have the right to customize their view and environment. To change display settings, language preferences, ... Users have the right to run in 600x800 resolution and still navigate the page.
- Users have the right to use computers even if the users are differently-abled (various degrees of sight-impairment and colorblindness, hearing, one-handedness) (Section 508).
- Non-native English speaking users have the right to still use our software (I18N).
- Users have a right to the most simple and intuitive interface possible. Everyone should have the ability to be a power user in the application. Shortcut keys should be available and intuitive for common tasks. A user should be able to get to an area of the program in the least number of clicks and keystrokes possible; remove unnecessary transitional navigation pages.
- Users have a right to know what version they're running. Product name and exact version number need to be displayed in file names and in the install, before the installation process begins. It is unacceptable to display the version sometime after the install process has begun, or even worse, only after it has completed.
- Users have a right to graceful failures. Crashing the application or losing user data is unacceptable at any time under any condition short of OS intervention or powerfailure. Displaying ugly, unduly alarming, or unintuitive error messages are also unacceptable. The error messages need to avoid being unduly complicated, but they also must be honest about the cause of the problem. Sugar-coating a serious problem or treating an error condition as a mere warning does not help anyone.
- Users have the right to notification of service unavailability. An install should never restart any service without first asking the user.
- Intuitive interface. The onscreen text should explain exactly what is expected from the user. Users should never have to guess or go to some other documentation in order to configure or use. If additional information is needed, it should be redily available (links on the UI or in an obvious Help file).
- Identity Modifications. Users have the right to change names and passwords and everything should still work. Users have the right to make changes to other personal information (address, weight, spouse, etc.) and still maintain their identity and everything works.
- Instructions and Product Use Information. Documentation must be easy to find (all modern software products have a readme file at the root which contains vital information and links to all other needed info). It must contain instructions on the installation, configuration, administration, and use of the product. All documentation must be accurate and complete.
20071206
Software Engineering Manager Questionnaire
A few questions that I would want to ask before hiring or working for a Software Engineering Manager.
* What kinds of things do you do each week to improve the software development process of the projects under your control. What things do you do to improve your understanding of the process.
* How do you motivate people who are under you? Explain both the positive and negative motivators that you consciously use. Is motivating Engineers different from motivating other employees? If so, how?
* How often do you talk one on one with your engineers? How well do you understand what activities they are working on from day to day?
* What things do you do to encourage open, honest, productive communication with your Engineers? Is communicating with Engineers different from communicating with other employees? If so, how?
* What things do you do to empower your people to be productive? For example: How are meetings planned, how do you ensure that your engineers have adequate equipment to perform their jobs, how do you set expectations, how do you encourage people to set goals, how do you measure individual and group progress?
* Are you aware of the different talents and abilities of your engineers? Talk a little bit about how you manage your engineers in order to make the best use of their strengths. Do you have a well rounded team?
* What things do you do to ensure that you're evening out the workload among your people. No one is constantly expected to carry the whole load. How are people who choose to do more compensated/rewarded in your group.
* Do you lead by example and inspire confidence in the people that you lead? Do you backbite others in front of your engineers? Do you keep professional hours? When your engineers are in a crunch and must stay late to complete an assignment, are you willing to stay with them?
* Are you sensitive to burnout and dissatisfaction among the people under you? What do you do to avoid and to handle these situations?
* Describe the project life cycle of the typical project in your group. Include details about each of the project phases; what artifacts are produced and what metrics are used to gauge if you are on track for the cycle? What works well? What would you change to make things work better?
* Can you identify the top producers on your team? Do you recognize and compensate this talent? How much of the rewards you assign to your reports is based on their individual contribution and how much is based on seniority, likability, or other factors?
*Besides financial compensations (like money, gift certificates, and stock options), what are the other types of compensations you use to reward good employees?
* How do you encourage team members sharing information with each other and the team? Teaching, training and mentoring within the team?
* Software development is a constantly evolving field. The technologies are changing fast. What kinds of things do you do to ensure that your Engineers are up-to-date on (or at least familiar with) the latest technologies and patterns/methodologies of enterprise application architecture. How do you encourage Engineers to self-educate and stay abreast of the constant flood of changes.
* What are the most successful products that you've been involved with. Pick one project and describe who was involved in making it a success and specifically what each individual did to contribute to it's success.
* What kinds of things do you do each week to improve the software development process of the projects under your control. What things do you do to improve your understanding of the process.
* How do you motivate people who are under you? Explain both the positive and negative motivators that you consciously use. Is motivating Engineers different from motivating other employees? If so, how?
* How often do you talk one on one with your engineers? How well do you understand what activities they are working on from day to day?
* What things do you do to encourage open, honest, productive communication with your Engineers? Is communicating with Engineers different from communicating with other employees? If so, how?
* What things do you do to empower your people to be productive? For example: How are meetings planned, how do you ensure that your engineers have adequate equipment to perform their jobs, how do you set expectations, how do you encourage people to set goals, how do you measure individual and group progress?
* Are you aware of the different talents and abilities of your engineers? Talk a little bit about how you manage your engineers in order to make the best use of their strengths. Do you have a well rounded team?
* What things do you do to ensure that you're evening out the workload among your people. No one is constantly expected to carry the whole load. How are people who choose to do more compensated/rewarded in your group.
* Do you lead by example and inspire confidence in the people that you lead? Do you backbite others in front of your engineers? Do you keep professional hours? When your engineers are in a crunch and must stay late to complete an assignment, are you willing to stay with them?
* Are you sensitive to burnout and dissatisfaction among the people under you? What do you do to avoid and to handle these situations?
* Describe the project life cycle of the typical project in your group. Include details about each of the project phases; what artifacts are produced and what metrics are used to gauge if you are on track for the cycle? What works well? What would you change to make things work better?
* Can you identify the top producers on your team? Do you recognize and compensate this talent? How much of the rewards you assign to your reports is based on their individual contribution and how much is based on seniority, likability, or other factors?
*Besides financial compensations (like money, gift certificates, and stock options), what are the other types of compensations you use to reward good employees?
* How do you encourage team members sharing information with each other and the team? Teaching, training and mentoring within the team?
* Software development is a constantly evolving field. The technologies are changing fast. What kinds of things do you do to ensure that your Engineers are up-to-date on (or at least familiar with) the latest technologies and patterns/methodologies of enterprise application architecture. How do you encourage Engineers to self-educate and stay abreast of the constant flood of changes.
* What are the most successful products that you've been involved with. Pick one project and describe who was involved in making it a success and specifically what each individual did to contribute to it's success.
20071114
Testers Creedo
* Expose all issues. As a tester, you must report all software problems that you encounter. The ones in your own product take first priority, but defects found in other products along the way should not go unreported. A Tester has the moral obligation to make all problems visible to the appropriate people. It doesn't matter if the bug is in someone else's product. The user will see the issue in your product and they will blame you if they don't know what to do. At the very least, you should document issues like this in the Readme or Troubleshooting docs and provide warnings or workarounds.
* Every minute wasted is a bug lost. Good Testers are constantly active constantly looking at the product and constantly looking for ways to save time. There is never enough time to test the product completely and so every minute wasted is another bug that you're letting slip by. Cannot afford to take the stance of being "Perfectly Content to do absolutely nothing until someone tells me twice." Never stop searching.
* Every minute of someone else's time that is wasted is 2 lost bugs. The bug that you could have been finding and the bug that the other person could have been finding. DO NOT WASTE OTHER PEOPLES TIME. If you have a question, Google it, read the doc, stop and think about your question. There are not very many questions in this Universe whose answers exist only in the mind of one person. Dropping in on a developer or another tester who is busy working should be your last resort.
* Regressing defects. Testing that a defect is fixed, that the issue no longer happens, is only the beginning of regressing a defect. The most valuable part is also the hardest part, it comes in testing around the fixed area to verify that nothing else was broken. Knowing what is "around the fixed area", guessing what could have been affected by a fix (or not affected when it should have been), attaining a knowledge of the architecture of the product.
* Defects must be fixable. There is no point in entering a report for a defect that cannot be fixed. An issue cannot be fixed if it cannot be duplicated and there are not sufficient trace information to determine the cause. There cannot be a fix if the subject of a defect is not an identifiable, quantifiable issue and if a case cannot be made that the issue will cause damage or inconvenience to a user. Writing observations without qualifying them as Defects is an unacceptable waste of other peoples time. Part of writing fixable defects is to investigate the issue so that you can provide enough information to determine (1) Where the problem is happening (at a minimum, who's component it is happening in, i.e. who should be responsible for fixing) and (2) How to reproduce the problem consistently. When the resulting damage or inconvenience of a defect is not obvious, you may need to make a case for why it really is a defect and why it must be fixed.
* Developer helpers. Our job has nothing to do with helping the developers to do things that they are able to do but would rather not do. We are not machine jockeys. We are not customer support. Our #1 priority is the quality of the product that we have been assigned. Anything else is a distraction and should be avoided. Our job is to examine all aspects of a product to evaluate the products quality and it's fitness to meet the expectations of end users. Playing the part of developer-helper, or unit-test-automater should be reserved for Developer Interns. It wastes of the time of a QA Engineer and ultimately stunts the quality of the product. Occasionally we will help other groups when they're under the gun, because we're geared for it. However, we can not afford to let those groups believe that helping them is our only calling in life. The most useless waste of a Quality Assurance persons time is engaging in a task that a developer could do themselves. Sometimes for the overall good we must allow ourselves to be distracted from testing, but don't let distractions run your day. We are not Developer Helpers or lackys.
* Study voraciously. Google is your best friend. Always search for an answer to a question before asking it. Always research the technology of the product that you will be testing. In many ways the tester has a responsibility to know, even more intimately than the developer, the architecture of the product, it's environment, and the technologies that it is built upon and that it interacts with.
* Functional testing is only the first step. In order to be truly useful in this job, a tester must get beyond functionality testing. Functionality testing is preliminary to any other testing. If the product doesn't do what it's supposed to do, then any other issues are not terribly relevant. Developers could and should be doing functionality testing themselves in the form of Unit tests and other pre-check-in checks. If this is not being done, then that issue in the software development process of the team must be tackled before testing can archive full productivity. Where a tester adds value to the process is when he is able to go beyond those things that should be readily apparent to developers and casual users.
* Credibility is everything. There is no room for lies in this profession. Conjecture about something you don't really understand is the same as lying. Agreeing with something that someone else says that you don't know to be true, is the same as lying. Since our primary objective is to show the true state of the product, if what we say is not trusted by others then, even when we have done the work and the information is right, we have failed the primary objective. "Honra y dinero se ganan despacio y se pierden ligero" ~ "Reputation and money are earned slowly and lost quickly".
* Be a Devil for Details. Writing a bug report that just says "IT DON'T WORK", is NOT good enough. You must investigate the problem enough to provide the "Why it doesn't work", or the "When it doesn't work".
* If you're not part of the solution... Good testers don't just point out problems, they also suggest solutions. Examining the problem to the level that you can suggest solutions helps to verify that you understand the problem.
* Stand for quality. The #1 job of QA is to hold software development to a high standard of quality. This entails both the product and the process, in order to enforce both, you need to strive to have a maven understanding of both.
* Every minute wasted is a bug lost. Good Testers are constantly active constantly looking at the product and constantly looking for ways to save time. There is never enough time to test the product completely and so every minute wasted is another bug that you're letting slip by. Cannot afford to take the stance of being "Perfectly Content to do absolutely nothing until someone tells me twice." Never stop searching.
* Every minute of someone else's time that is wasted is 2 lost bugs. The bug that you could have been finding and the bug that the other person could have been finding. DO NOT WASTE OTHER PEOPLES TIME. If you have a question, Google it, read the doc, stop and think about your question. There are not very many questions in this Universe whose answers exist only in the mind of one person. Dropping in on a developer or another tester who is busy working should be your last resort.
* Regressing defects. Testing that a defect is fixed, that the issue no longer happens, is only the beginning of regressing a defect. The most valuable part is also the hardest part, it comes in testing around the fixed area to verify that nothing else was broken. Knowing what is "around the fixed area", guessing what could have been affected by a fix (or not affected when it should have been), attaining a knowledge of the architecture of the product.
* Defects must be fixable. There is no point in entering a report for a defect that cannot be fixed. An issue cannot be fixed if it cannot be duplicated and there are not sufficient trace information to determine the cause. There cannot be a fix if the subject of a defect is not an identifiable, quantifiable issue and if a case cannot be made that the issue will cause damage or inconvenience to a user. Writing observations without qualifying them as Defects is an unacceptable waste of other peoples time. Part of writing fixable defects is to investigate the issue so that you can provide enough information to determine (1) Where the problem is happening (at a minimum, who's component it is happening in, i.e. who should be responsible for fixing) and (2) How to reproduce the problem consistently. When the resulting damage or inconvenience of a defect is not obvious, you may need to make a case for why it really is a defect and why it must be fixed.
* Developer helpers. Our job has nothing to do with helping the developers to do things that they are able to do but would rather not do. We are not machine jockeys. We are not customer support. Our #1 priority is the quality of the product that we have been assigned. Anything else is a distraction and should be avoided. Our job is to examine all aspects of a product to evaluate the products quality and it's fitness to meet the expectations of end users. Playing the part of developer-helper, or unit-test-automater should be reserved for Developer Interns. It wastes of the time of a QA Engineer and ultimately stunts the quality of the product. Occasionally we will help other groups when they're under the gun, because we're geared for it. However, we can not afford to let those groups believe that helping them is our only calling in life. The most useless waste of a Quality Assurance persons time is engaging in a task that a developer could do themselves. Sometimes for the overall good we must allow ourselves to be distracted from testing, but don't let distractions run your day. We are not Developer Helpers or lackys.
* Study voraciously. Google is your best friend. Always search for an answer to a question before asking it. Always research the technology of the product that you will be testing. In many ways the tester has a responsibility to know, even more intimately than the developer, the architecture of the product, it's environment, and the technologies that it is built upon and that it interacts with.
* Functional testing is only the first step. In order to be truly useful in this job, a tester must get beyond functionality testing. Functionality testing is preliminary to any other testing. If the product doesn't do what it's supposed to do, then any other issues are not terribly relevant. Developers could and should be doing functionality testing themselves in the form of Unit tests and other pre-check-in checks. If this is not being done, then that issue in the software development process of the team must be tackled before testing can archive full productivity. Where a tester adds value to the process is when he is able to go beyond those things that should be readily apparent to developers and casual users.
* Credibility is everything. There is no room for lies in this profession. Conjecture about something you don't really understand is the same as lying. Agreeing with something that someone else says that you don't know to be true, is the same as lying. Since our primary objective is to show the true state of the product, if what we say is not trusted by others then, even when we have done the work and the information is right, we have failed the primary objective. "Honra y dinero se ganan despacio y se pierden ligero" ~ "Reputation and money are earned slowly and lost quickly".
* Be a Devil for Details. Writing a bug report that just says "IT DON'T WORK", is NOT good enough. You must investigate the problem enough to provide the "Why it doesn't work", or the "When it doesn't work".
* If you're not part of the solution... Good testers don't just point out problems, they also suggest solutions. Examining the problem to the level that you can suggest solutions helps to verify that you understand the problem.
* Stand for quality. The #1 job of QA is to hold software development to a high standard of quality. This entails both the product and the process, in order to enforce both, you need to strive to have a maven understanding of both.
20070829
Opening Comments
This blog spot is intended to be a repository of my rants and insights on the software testing process.
Subscribe to:
Posts (Atom)
