Overview

Application Development is easy. Coding is simple. Debugging is... well, debugging. But the process of application development, that's hard. Discussion of the steps to be taken to speed application development, including innovations in lowering the overhead of converting requirements to prototypes to working applications.

The Process

There are probably as many ways to build an application, as there are application developers. That is not to say that there are that many good ways to build an application. Application development, despite the fact that it is taught as many colleges and universities as "Software Engineering" has not really reached that level of sophistication, precision or predictability that engineering usually claims. We should not give up hope, however, for our business is still in its infancy. We should not feel that after 40 years of application development that we should have all of the answers. We should, however, be getting to the point of having all the questions.

Disclaimer

The tools and processes described here are not the solution to all problems. Small one-person firms probably would be unable to get any coding done under the burden of attempting to maintain all of these tools, while large firms with huge projects would find these tools underpowered. In addition, any project which starts out with the purchase of a dozen new tools and add-ons is almost certainly bound for failure. A gradual introduction of tools with the goal of simplifying and smoothing the development process should be the goal. The tools and experiences described in this document are those lessons I have learned over the past 10 years working on a series of development projects involving one to six developers and ranging in duration from 40 hours to two years. I hope you will gain some benefit from my observations. I do not expect that you will be able to walk away with a "cookbook" to successful development, but rather that you will be more cognizant of the issues involved and more alert to the opportunities they present.

Those who fail to heed the lessons of the past
are doomed to be annoyed by those of us who have.

- Ted Roche

The CMM model

No discussion of development issues would be complete without some discussion of the (Capability Maturity Model (CMM) of software development and the Software Engineering Institute. CMM describes a set of levels of achievement and predictability of future achievement to which a software development enterprise can hope for. Several good lessons can be derived from the information provided: Tools are not the solution to the problem. Tools are a means to solve problems. Maturity in software development, similarly to life, can only be achieved as a gradual set of steps.

The CMM model presents a set of "levels of maturity," and, like their equivalence in real life - infancy, childhood, puberty, adolescence, adulthood, middle age and seniority - these are likely to be stages to grow through. The levels are described briefly in the table attached, but anything short of a book does not do the CMM justice. Check out writings by Watts Humphrey and the SEI (some of which are in the bibliography) for much more information on this important model.

Level 1 a.k.a. Level 0	Chaos	The success of any one project is no indicator of future successes.
Level 2	Repeatable Process	The beginning stages of repeatable success: project plans, estimates, concurrence from all parties, coordination and responsibility for fulfillment of estimates.
Level 3	Defined Process	Oversight and review of procedures with an aim at creating a defined process which encompasses all tasks, development of project risk plans, SCM controls, a standard process framework.
Level 4	Managed Process	Process management and analyses provide constant feedback and refinement of all phases of the development process, yielding corresponding product quality improvements.
Level 5	Optimizing Process	Productivity improvements are aimed primarily at the process itself.

Unless you are a member of a rare organization, I would like to welcome you to level one. But like many 12-step programs, the first crucial step is the recognition that there is a problem. In this case, organizations that are aware of the SEI and CMM are almost always better than those who haven't, even if the progress at the former companies is slow in coming.

If you're not part of the solution, you're part of the problem.

- Malcolm X

The Process

The exact methodology we follow will depend in part on the needs of the client. If they require a drawn-out and formal analysis and design period, we will accommodate that. Our preference is a rapid application development style where we refine the design of the system by quickly developing a set of (initially non-data-bound) prototypes which more and more closely model the actual workings of the system.

The "Waterfall methodology" of developing software has proven to be too slow and cumbersome in many development situations. With "waterfall," the concept was that the business problem was completely described before the next stage was undertaken. Analysis and design would be completed before the selection of software tools and coding would begin. The problem with this model of handing off responsibility and work from group to group is that the system was designed to only work in one direction. If business needs changed as software development was underway, and entirely new waterfall needed to be constructed to handle the new wrinkle.

With the "spiral design" concept, first introduced by Barry Boehm in IEEE Computer in 1988, the idea is that small phases of development take place, followed by a step to plan the next phase, a re-examination of objectives, alternatives and constraints, and finally a step of risk analysis and alternative evaluation, then back into development. This is similar to the idea of "proof of concept" applications, which first show that the language or widget under consideration might be up to the task, followed by successive refinements of the application into a working model.

We form almost a "miniature spiral" in the rapid prototyping of the first few weeks of an assignment, using a prototyped form to change an ERD object to, in turn, modify the object design and the intial form. This cycle may last a few hours on a tiny system or small change, to several weeks for a large scale system.

Project Management

If you don't know where you're going, how will you know when you get there?

Project management takes many forms, from sophisticated Gantt and Pert charts to the "To Do" list scribbled on the whiteboard. It doesn't matter what tool you use, as long as it will tell you where you are, where you've been, and when you going to get where you are going. Depending on the scale of the project, different tools are appropriate. Simple checklists are sufficient for a 40-hour contract; sophisticated project management tools more appropriate for multi-developer, multi-month project planning and coordination.

What we have here is a failure to communicate.

- Cool Hand Luke

A second form of project management, certainly no less important than the first, is the control of the client's expectations. It is painfully easy to have a small misunderstanding on the part of either the client or the consultant snowball through the cycle of a project. Careful, regular checks on the client's opinion of the current project's status, reviews of the current prototype, simulations of real life situations, are key to ensuring that we're all "playing from the same score."

Anomaly Tracking

What percent of the bugs you fixed last year were "off by one" errors in loops? How many of your CASE statements should have had an OTHERWISE but didn't? (Hint: all of them). What is the most frequent problem that causes QA to reject the new forms from the programmer down the hall? What is the overall cost of the maintenance release you just shipped?

Answers to these questions are vital to understanding your business and identifying areas which need improvement. Gathering metrics at each stage of the development process can help you to identify weak areas, resolve problems in your process, and improve the bottom line.

Writing Specifications

Specifications serve a variety of needs and should be tailored for their particular application. Specifications which serve as the contract between client and consultant must state this explicitly, often, clearly. Specifications which are used as "living design documents" throughout the development cycle are very different animals and should be structured to reflect that. Ensure that you and your client agree on the roles those documents should play.

Estimating

There are a variety of techniques, typically depending on the scale of your application. Function Point Analysis is a powerful and sophisticated, but expensive and time-consuming, method of estimating projects. "Action Point Analysis," as suggested by Whil Hentzen in his Developer's Guide (see bibliography) is a smaller-scoped, less expensive technique. Finally, SWAG (Sophisticated Wild Aardvark Guessing) is more typical of many developers, who should consider upgrading this technique

Weekly builds

In coordination with our efforts at Rapid Application Development, delivering a build each week to the client demonstrated progress, instilled confidence, served as a springboard for further design refinement Weekly builds could culminate in an advanced and detailed functional specification, project plans and detailed project cost estimates, if that was the goal. The developers would have a very good chance of an accurate estimation with time spent developing working prototypes.

Selecting Frameworks

Pick one. Work with it. Check out reviews. Buying all of the frameworks and testing them yourself is really not that high an investment when you consider the billable hours you intend to get as a return on them.

Third-party Tools

I am hard-pressed to think of a third-party tool I could have written for the cost of licensing the tool. My company's typical application includes from a half-dozen to two dozen third-party FLLs, DLLs, routines, utilities, OCXs and stuff. Building tools from scratch does not make sense.

CASE tools

While large scale "Upper Case" full-cycle software lifecycle management tools are not yet within the reach of most developers, "lower case" tools for analysis and design of components of the system, such as object-oriented analysis and design (OOA&D) tools or entity-relationship diagram (ERD) tools are well within the budget of all developers. Like third-party tools, these provide tremendous return on a small investment.

Programming standards

Standards should come in as many forms as required by your organization. We have typically found that a general, global standard is good for most of the basics, but that the global standard should not specify every little detail. The choice of different frameworks, or different destination platforms makes a lot of the specific choices for you.

Code reviews, Inspections, walkthroughs

One of the most marked improvements occurred on one project when we added the simple requirement that a second person review the code. The number of comments increased markedly, algorithms were simplified, and code was completed before being added to the system. A code review is typically an informal process where developers get together to review code, learn new techniques, and share insights. A code inspection is more typically a process of checked a "completed" piece of code for compliance to standards and for completeness and robustness. Formal code inspection procedures often specify a series of inspectors, a coordinator or manager of the process, a moderator and a scribe. Often, this process has required some scaling-back for the smaller projects I am more familiar with, but retaining the spirit of the process is essential.

Training

You can't get too much. Whether it is formal class work, industrial training, attending conferences, or reading on your own, the industry we are involved in is changing too fast to count on getting by with what you know until retirement.

Data
Entities

Interface
Design

Objects

In Rapid Application Development, use VFP tools to rapidly prototype forms and reports to draw the details of the data entities' properties and the objects behaviors from the client.

Resources

The author didn't make all this stuff up himself. Okay, maybe a little, but not much. Most of the material for this session comes from books, magazines, newsgroup and forum conversations, user group meetings and other sources. The most important thing a developer can be doing is learning some new material every day. The toughest developers to work with are the ones who have mastered the trade. They know it all, need no books, magazines or conferences.

The business we are in is constantly changing. New techniques and new applications are constantly being invented. It is often this exciting change which attracted us to the field in the first place. However, the pace of change always seems to be increasing. It is a challenge for all of us to keep up. One well-known consultant states that he does not believe that it is practical for a consultant to log more than one thousand hours of billable time per year - only an average of 20 hours a week! What does he do in the rest of his time? Well, yes, there's the occasional game of Solitaire, and infrequent noodling on the web in search of… well, whatever, maintenance of the machines, marketing, accounting and legal procedures, but much of the time is consumed with learning.

Conferences

I don't have to preach to the choir here on the benefits of attending conferences. I have never failed to have a conference pay me back - the costs of lost billable time, airfare, hotel and conference fees - in the new items I learn, the new contacts I make, at a conference. Look for conferences both within your field and those of a more general nature - Comdex and TechEd, for example. I also like to catch two or three of the one-day local shows put on by the major to vendors, just so I can hear what they are telling the decision makers I try to sell my services to.

Magazines

I have written for both of these magazines and feel you are missing out on a lot of good material if you don't subscribe to both. One good time-saving tip should pay for a year's subscription.

FoxPro Advisor - glossy, lots of pictures, lots of advertising - a great source of third party info, usually over 64 pages (http://www.advisor.com)

FoxTalk - plain paper, fewer graphics, sometimes more code, usually 32 pages. (http://www.pinpub.com)

User Groups

A user group meeting should serve a number of purposes. First, the main presentation often has something of value. Most presenters at this conference practiced on the home town crowd, so you often get to see some top-notch performances from knowledgeable speakers. Second, the user group is a great opportunity to network with other developers and learn of new jobs, projects to join, or potential employees to hire. Lastly, the user group meeting can be the start of many informal relationships which can lead to further business down the road.

Professional Associations

Like user groups, these organizations often have local meetings with the opportunities to network and schmooze. However, they also have the advantage of being backed up by a national organization, typically with some top-notch publications.

Association of Computing Machinery (ACM) http://www.acm.org one of the oldest computer organizations, has some very good periodicals, and local chapters.

Independent Computer Consultants Association (ICCA) http://www.icca.org has some excellent resources for a new consultant just starting out or a seasoned professional needing some guidance. Also a good networking opportunity.

Web sites

Microsoft's Visual FoxPro web site (http://www.microsoft.com/vfoxpro.htm) - this site has been a veritable hotbed of activity since the last FoxPro DevCon. Changes are made every week. Updates for a replacement GENMENU and Setup Wizard are among many gems available at this site. Check it out weekly, at least.

Transformation - http://www.transformation.com - site of the FoxPro Yellow Pages and the Universal Thread, a promising use of Web technology to create a FoxPro developer community.

Developer's Studio Apartment (http://www.hentzenwerke.com) - besides being a shrine to the host of GLGDW, this site has a number of useful features, including the online equivalent of TUFR (The Ultimate FoxPro Reference) with a top-notch search engine to locate that article you knew you had a while ago.

Bibliography

This is not a comprehensive list of books on the topic, but rather a place to get started. Keep an eye out for these authors, as nearly anything written by them is worth getting. Several of these books, especially the recent titles, come with comprehensive bibliographies well worth the additional research.

Brooks, Frederick P., Jr., The Mythical Man Month, 1975, Addison-Wesley, 0-201-00650-2

DeMarco, Tom and Lister, Tim, Peopleware, Dorset House, 0-932633-14-5

Hentzen, Whil, The 1997 Developer's Guide, Hentzenwerke Corporation, 1996, 0-9655093-1-1

Humphrey, Watts S., Managing the Software Process, Addison-Wesley, 1996, 0-201-18095-2

Humphrey, Watts S., A Discipline for Software Engineering, Addison-Wesley, 1995, 0-201-54610-8

Humphrey, Watts S., Introduction to the Personal Software Process^SM, Addison-Wesley, 1997, 0-201-54809-7

James, Geoffrey, The Tao of Programming, InfoBooks, 1987, 0-931137-07-1

Jones, Capers, Applied Software Measurement, McGraw-Hill, 0-07-032613-7

McConnell, Steve, Code Complete, Microsoft Press, 1993, 1-55615-484-4

McConnell, Steve, Rapid Development, Microsoft Press, 1996,1-55615-900-5

Plauger, P.J., Programming on Purpose, Prentice Hall, 1993, 0-13-721374-3

Weinberg, Gerald M., Understanding the Professional Programmer, Dorset House, 1988, 0-932633-09-9

Yourdan, Edward, Decline and Fall of the American Programmer, Prentice Hall, 1993, 0-13-191958-X