Software Development Assessments for the 21st Century

By Bruce J. Hayes

During the last 35 years, software development and technology processes in general have evolved at a rapid, even chaotic, rate. These processes range from small, Lean, agile (sometimes labeled iterative) development pockets to large, bureaucracy-laced legacy projects of tremendous scope (sometimes labeled waterfall development) and everything in between.

Many organizations have undertaken attempts to classify, characterize and assess these processes for purposes of driving significant improvement. What has resulted is a number of standards and associated assessments, all professing to be a magic bullet. Most of these standards share philosophies and some are derivatives of others. Although the majority of these efforts yield some improvement, it is often not the enterprise-wide, step-function improvement hoped for.

Of course there are best practice stories from each assessment, usually where management has taken the time to really understand their culture, needs and requirements, and committed to and aligned for a change in culture. But in today’s environment, the pressure is great to deliver results now. Quick fixes that drive never-ending reactive behavior and associated resource shuffling are common. The bottom line is that many organizations, unless mandated to do so, simply do not take an objective look at themselves, and thus become stuck in their ways. Or, they embrace ad-hoc improvement, trying many different things and hoping for the best.

So how does an organization start the process of driving enterprise-wide change, without investing significant time in resource-intensive assessments to establish a quantitative baseline and measure subsequent performance progress? The answer, as with many things today, may lie in web technology. But to understand this technology, it is important to first understand the characteristics and reasons assessments are done in the first place.

Why Do an Assessment

Process or organizational assessments, if done right, produce good information and data relative to how an organization performs its work. Assessments help provide objective viewing of processes, tools, behaviors and the consistency of application use across an organization. From that baseline, companies can identify process and organizational strengths as well as weaknesses, where opportunities for improvement exist. In some cases organizations also may receive a numeric score, ranking or dashboard color, relating its performance to best practices. The results also may be displayed demographically by location, division, product line, project team and so on, which is useful for planning and implementing an improvement strategy.

Normally, and especially for lower maturity processes and organizations, self-assessment is not an option. Objectivity is difficult to achieve. Plus, most companies do not have individuals with the skills to perform the assessment and compare their organization to best practices; if they do employ such an individual, that person is too busy with daily issues to respond to organization-wide needs.

Hence, a team of objective experts usually is brought in to perform the assessment. These on-site, interview-based assessments are high overhead events, especially for medium- to large-size organizations. They require lots of planning, and can be time consuming and disruptive. Further, the sample sizes end up being small relative to the total population within the subject organization, creating a risk in the accuracy of results. Further risk is created by interviewee manipulation, preprogramming, and bias of individuals and the methodology.

An Alternate Method: Web Technology

There is a strong correlation between organizational sample size and result accuracy. But trying to improve a score by increasing the sample has the potential to drive up the overhead of the assessment considerably. Consider that for a 2,000 person organization, a 5 percent sample size would require 100 interviews, or as much as 200 person-hours of effort. This would also mean 200 hours of lost labor. Add in planning, work disruption, travel, compilation of results and follow up – this is a significant resource drain. Even at a 5 percent sample size, the accuracy of results and conclusions may be at significant risk.

One alternative, web technology, makes it possible to scan large swaths of an organization, at a fraction of the cost, with far less work disruption. The key to the technology is capturing and documenting the best practices, developing a question set that can be answered by all layers of the organization, and processing the resultant information in a statistically valid and graphically friendly manner.

Web-enabled assessment processes are becoming more advanced and alleviate much of the downside of on-site, interview-based assessments. Because the question set is standard and fixed, the sample size can be up to 100 percent of the organization, the questions can be answered in a confidential way and at the interviewee’s leisure, and much of the bias can be eliminated or normalized. Plus, the cost per online interview is reduced by an order of magnitude over on-site assessments, so many more interviews can be conducted, improving accuracy and reducing risk. This results in a highly accurate assessment of the as-is behaviors and practices for the subject organization. It also provides great insight into differences in operational behaviors between business units, product lines, project teams or virtually any defined demographic.

Benefits of Technology

Technology allows for the acquisition of much more detailed responses, and results that may be organized demographically. In addition, an array of analytical techniques may be applied in an efficient manner. Through the use of fixed questions with Likert-type scaled responses, multiple choice questions and open-ended questions requiring a typed response, a complete and objective characterization is acquired.

The technology also brings significant efficiency to the process through the use of preloaded demographics, alignment of pertinent questions to respondent types and the ability for the respondent to answer the questions in multiple sittings, thus reducing the overhead required to sit with an interviewer for one or two hours.

As this technology continues to be adopted, the day may come where organizations will share data industry-wide, helping to propagate more meaningful standards databases.

About the Author: Bruce J. Hayes is a co-founder and managing partner at Six Sigma Advantage, Inc., where he focuses on business development, executive coaching and training, and curriculum development for integrated approaches to Lean Six Sigma. He is an experienced operations professional and consultant with more than 27 years of business and engineering experience. Hayes was formerly a senior executive at Motorola, where he was one of the key contributors to developing and “operationalizing” Six Sigma. He has served as an executive coach and mentor for many Fortune 500 companies that have successfully implemented Six Sigma. He can be reached at bhayes@sixsigma-advantage.com.

10 things I hate about websites

Ten Things I Hate About Web Sites
Date: Apr 28, 2006 By Daniel P. Dern.
Daniel Dern relies on the Web enough to get pretty irked about sites that make it unnecessarily difficult to find information, because of bad design, missing content, or bad browsers. Rant along with Daniel through 10 behaviors that make you want to put some web designers - and the people who approved their efforts - on the Mikado's "little list."
Web Sites Are Supposed to Make Things Easier, Aren’t They?
Like most daily Internet users, I go to the Web both professionally and personally. As a full-time freelance technology writer, I visit a lot of vendor web sites in search of such basic information as name and contact information for an appropriate PR person, the company’s full name, what the heck they do, specific product names and information, news, reviews, opinions, etc. I go to news sites for research. I may have to use a web interface to my email when at an Internet kiosk or other strange machine.
Like most consumers, I also use the Internet for a variety of more personal needs: ordering books and CDs on Amazon.com; shopping on eBay; making travel reservations; reserving and renewing library books; reading news and daily comic strips; looking up phone numbers, word spellings, definitions, and answers to medical and other sundry questions; and a myriad of other tasks.
I’m not disputing that the Web is a great, useful tool (as well as the ultimate timesink for info-philes). But a depressing number of professionally done, business-oriented web sites have serious problems, as far as I’m concerned—problems that keep me from getting my work- or consumer-related tasks done. And many or, more likely, most (possibly all) could have been avoided quite easily.
To paraphrase e. e. cummings, "S/he who thinks about your web site instead of its content will go to your competitor quickly."
Standardization, Please!
Within the U.S., telephones are pretty much standardized, generating the touch-tone sounds needed to navigate or to enter information. Would that we could say the same for web sites; I simply can’t count on all sites to work well—or at all—with my browser. I currently prefer Opera as a rule, turn to Netscape or Firefox where Opera won’t handle a site, and resort to Internet Explorer when all else fails. But even web sites that respond to one or more of these browsers may frustrate or annoy me because the information I want isn’t there, or is hidden in an unlisted menu option. Clearly, far too many of these sites are designed for people who already know where to find what they’re looking for—and were designed by people who were too close to the problem.
Ten particular aggravating behaviors make it unnecessarily hard for me, as a customer or as a press person, to get information from—and/or buy from—web sites. When will vendors learn that, on the Web, all their competitors are just a click away?
Requiring Unnecessary Features or Plug-ins for Navigation
Not everybody has a browser that supports ActiveX, JavaScript, Java cookies, or Flash. We may have browsers that can’t handle those features, or browser versions that don’t include them. (For example, Opera offers both Java and non-Java versions.) Many of us have deliberately chosen to turn off some or all of these features, or our firewalls and other security tools may block them.
Viewed from a non-JavaScript browser, a surprising number of top-level web pages show nothing but an empty white window, seem to have no navigation bars, or contain large blank areas where I suspect there should be content. The page may have things that look like they should be links, but aren’t, or have links that can’t be clicked.
Equally irritating are unnecessary security obstacles, such as calling for SSL, certificates, or an account to get to information that should be freely and easily available. The key word here is unnecessary. Obviously, you need to restrict access to customer accounts and private information. But if you want any visitor to be able to find and get information, you shouldn’t make it unnecessarily difficult.
Also, don’t design for specific screen resolutions or window sizes. If you must specify sizes, don’t make windows so wide that we have to side-scroll just to read the right-hand inch or two of text.
’Silent Failures’ for Unavailable Browser Features
Even more annoying than web sites that call unnecessarily for ActiveX, JavaScript, cookies, or Flash are parts of sites that do need these features—but don’t indicate that they do. Dell’s site is a "good" example of this bad behavior (at least as of February 21, 2006).
"Silent failures" make a site difficult or even impossible to navigate. They often result in not finding information—usually because I don’t even know it’s there—or in a conversation (phone or email) with a PR person like this:
PR: It’s right under the Resources link.
Me: What Resources link?
PR: The one at the right side of the navigation bar.
Me: What navigation bar?
I’ve seen plenty of sites that are smart enough to detect that my browser is a shy of a feature or two, and good web design dictates that such sites display a large-font advisory like "This site requires JavaScript. Some features may not work properly."
The web site for Newton, MA (where I currently reside) uses ActiveX and JavaScript, as best I can tell, both of which I leave out or turn off. If I attempt to access the site with Opera, I wouldn’t know that ActiveX is required, or that there might be information or links I’m not seeing; at least cranking up Internet Explorer gets me an advisory message. This results in some information being invisible. For example, try to find the hours of operation for the town’s recycling area—in a no-script browser. The town library’s site, by comparison, has been designed with enough savvy and courtesy to include the advisory note "Search history function requires JavaScript."
Failing Completely for Unavailable Features or Unsupported Browsers/Versions
Almost as frustrating—but equally annoying—is a web site that needs a feature, version, or specific browser to get to even the main page. There may be compelling reasons why visitors need IE version 7, or you won’t support Opera, or whatever. But that’s inappropriate on your main page; let the world at least see your company name and key contact information.
And don’t "blink and fail," either. Your home page shouldn’t start showing content and then switch over to a "You Must Have..." error page, the way that Microsoft’s Contact Us page does.
Browsers Not Offering ’Per Window’ or Easily Reversible Setting Changes
Every so often, I encounter a web site that requires me to make some browser setting changes. But I shouldn’t have to make changes that will then be applied to all my other browsing—or that I have to undo, one by one. There should be some way to tweak custom settings and then "untweak" them all with a single click—is that asking too much? (I know, "Use VMware and run disposable browser sessions.")
Companies Registering But Not <http://www.sitename>
If your company name is Foobar, my browser should be able to find you as foobar.com, not just http://www.foobar.com. How hard is it to register both versions of the URL, hmm?
Making Information Hard to Find
Basic information about your company—name, contact information, what you do/sell—should be on the home page or at most one click away, and easy to find from well-labeled main-page links. I’m talking about your company’s full name. Most customers don’t care about this, but if I’m writing about your company, you should care that I get the full name correct. Make it easy for me to determine which is actually your company’s name:
Blivits
Blivits Corporation
Blivits Inc.
Blivits, Inc.
Blivits Ltd.
Ideally, this information should be in the title bar and/or at the bottom of every web page. Failing that, the About Us page should start with the full name of the company.
Similarly, tell me what your company does/makes—and do it quickly, clearly, and tersely. One sentence. Two at most. For example, "Blivits Ltd. makes Linux-based jaunting appliances for small, remote, and branch offices." If I want a four-paragraph essay on the history and philosophy of your company, I’ll look for a link like History or Our Philosophy. The About Us page should display only a succinct statement of what your company is or does.
NOTE
You needn’t bother calling your company the "world leader" or other adverbial puffery, either. Reporters won’t bother copying those statements, so you’re just adding more work for us. If you really are the market leader, saying "number 1" or "the leading U.S. vendor of..." is sufficient.
Making It Hard to Find or Select Among Products
Don’t assume that site visitors know the names, models, or whatever of your product lines—especially if you keep changing them.
Don’t assume that we know whether a given product is for small office/home office (SOHO), medium-sized businesses, or enterprises.
In my experience, the biggest offenders in this area have been companies selling notebook computers: Dell, HP, IBM, and so on. Want an example? Consider Dell’s notebook categories as of February 20, 2006:
Dell Inspiron Notebooks
Home & Home Office
Small Business
Medium & Large Business
State & Local Government
What makes a notebook more appropriate for a large business than a home office or local government use?
Surge protectors and UPSes are another example of problematic "How can we tell what’s what?" categories, with a dozen products that have identical descriptions.
Being Impossible to Contact
If you actually want the press—or customers—to contact you with requests for information (or to buy something) when we can’t find what we need directly on your site, then provide phone numbers and email addresses on a Contact Us page. For the press, other good places for this information are pages named Press Contacts, Media Center, etc. Forcing visitors to root through your site’s brochures, press releases, and so forth doesn’t win you any favors.
Please note that if we care enough to try contacting your company—rather than simply proceeding to a competitor’s site—we’ll call the first phone number we can find on your site: Sales, PR, Tech Support, anybody. And whoever answers the phone will expend time helping find us the right contact information.
On the other hand, don’t provide contact info if you’re not serious about being contacted. If your PR, Sales, Tech Support, or other departments don’t actually check and respond to email/phone messages, don’t provide that information on your site. (Although it would be helpful to say, "Don’t bother contacting us, we won’t reply," so we don’t waste our time.)
Another deadly annoyance is companies that don’t use their own web sites. Let’s assume that I can’t find a specific PR contact name, email address, or phone number, because the company’s Press Contacts page uses JavaScript and I don’t have access to a JavaScript-capable web browser at the moment. Example: "Contact Intel PR." Here’s what happens when I call the main phone number:
Me: I’m from the press, please connect me with somebody from PR.
Operator: I need a specific name.
Me: It’s probably on the web site, but I can’t get to that page.
Operator: I’m sorry, but I don’t have access to that information from here.
While we’re on the subject, if your site links to any companies that are your resellers or indirect partners (as opposed to just general links), you should make sure that their sites are both usable and useful.
Putting Information Only in PDF or Document Format, Not in HTML or at Least Text Files
Putting information such as press releases, product specifications, or contact lists only in Acrobat files or spreadsheets adds several layers of aggravation:
It adds one to three steps to the process of getting documents.
It can make copying and saving information complex or hopeless.
Depending on the computer I’m using (I may be at a borrowed web kiosk), the information may be completely impossible to get.
Make it easy for visitors to get the information you want us to have!
Having No-Value-Added ’Click Here to Enter’ Splash Pages
Other than your web designers or somebody in your marketing department, who thinks anybody outside your company cares about the nifty photograph or cute graphic that, along with a "click here to enter" note, is your site’s top page? Don’t you understand that wasting my time and making me work harder to get at your site (not to mention wasting bandwidth) just inclines me to go somewhere else?
If you absolutely must waste everyone’s time with an "Enter here" splash page, please at least follow these rules:
Don’t make it an impossible-to-skip Flash page.
Don’t put the "Skip Intro" link only at the bottom, in tiny type.
Don’t make the "Skip Intro" link require JavaScript or ActiveX.
Check Your Site for Stupidity
How can you avoid creating web sites that annoy, waste the time of, or chase away people that you want staying on your site? It’s really pretty simple:
Try being a user and see how much of a pain it is to use your site.
Have several other people test your site.
Test via the Internet, not from inside your company’s network.
Test with Internet Explorer, Firefox, Netscape, and Opera.
Start your tests with ActiveX, JavaScript and cookies turned off.
Test from Windows, Mac, Linux, and two handhelds—Microsoft Pocket PC or Windows Mobile, and Palm OS.
If you avoid the ten annoying behaviors I’ve discussed here, and you test comprehensively—ideally, including having an outside party do some testing as well—you should end up with a web site that helps users to think about your business, not your site.
Daniel P. Dern is a freelance technology writer. His web site is http://www.dern.com. Over the years, he has chronicled some of his previous Internet gripes through his Internet Song Parodies, including "When You’re Still On The Net."
© 2005 Pearson Education, Inc. InformIT. All rights reserved.
800 East 96th Street Indianapolis, Indiana 46240