^{Session E-MAINT}

^{Maintenance in FoxPro}

Ted Roche
Blackstone Incorporated

Contents

Maintaining FoxPro applications may not have all of the excitement of creating new Visual FoxPro applications from scratch, using all of the latest technologies. But maintenance can be as challenging, and a valid basis for a business, or a sideline to an existing business. In this session, we'll review the business arrangements needed to maintain your own, or someone else's, application. We'll discuss auditing an existing application, and extracting metrics indicating an application's maintainability. We'll talk about the tools to maintain an older application, issues unique to older code, and techniques to ensure a successful maintenance engagement.

Business Issues

Nearly all of us are developing and maintaining FoxPro applications as part of a business, and not as a not-for-profit hobby motivation. Therefore, we must first and foremost, make the business case for the projects we undertake. In the past few years, I have taken the opportunity to work on a number of existing projects that were in need of work. The range of work varied from minor cosmetic enhancements to the decision to kill a project that was not feasible to continue. Most of these projects were developed by other organizations, and in most cases the original developers were not available to explain their work. These projects had a number of challenges. While each project varied from the others, there were common issues I saw repeat themselves. As I have not seen this topic discussed at conferences before, I thought this might make an interesting discussion. I welcome your feedback, both during the session and afterwards. Please send your comments to me at mailto:troche@bstone.com. I hope you enjoy the session.

Cost-benefit, ROI, business case

The primary goal of commercial software development and maintenance is to solve a business problem. In the rush of everyday life, it can be easy to lose sight of this goal. Primarily, the work you are doing should solve a problem the business is having with their application. Either the behavior of their business has changed, or the application fails to correctly model their business. The principals of the business must be able to clearly identify where the application is lacking, what needs to be done and, most importantly, how much that change is worth to the business.

Without an understanding of what the value of the change is, there can be no rational decision on whether the value you deliver is worthwhile.

Good example: "We get 10 phone calls a month at Tech Support to explain that dialog box. At $75 a call, it is costing us $9,000 to answer the same question. We have a specification on how we want the dialog box to behave. Can you help us?"

Bad example: "My DOS code base does what the clients want, but I am hearing from the clients that they want a Windows app. I'd like to be able to pull data from their various client-server systems, and I'm thinking my app should be moving towards being a web-based application. How much will that cost?"

In the first case, the client has clearly identified the problem and its costs. If they plan on maintaining this application for several years, a sizable budget can be justified in terms of averted costs. In the second case, the client is not yet at the point ofmitting resources, but needs to be led through the process of identifying the costs involved in:

- losing customers to competitors with Windows applications

- what the implications of building generic data gathering capabilities might be, versus targeting a few specific platforms

- what the target architecture of their application (standalone, client-server, n-tier, component-based) should be

Until such discussions take place, there is no basis for deciding on the value of your services.

Should you choose to continue to work with the second client, bear in mind that your expertise at developing business models, performing cost-benefit analyses, and calculating ROI are just as valuable as your ability to tell a BROWSE from a grid. Clients often want a price quote at no charge, but they should be paying for your expertise when it is required.

Structuring a maintenance contract

Structuring your contracts, of course, involves a great deal of the requirements of your local jurisdictions, and you should consult with a legal authority familiar with local law and your industry. Please take the advise here as general in nature and insure that it fits your situation. In addition, the entire structuring of contracts is too large to cover in this document. Please check out some of the resources listed at the end of this paper.

The key items you want to ensure appear in your contract are an agreement of the work to be performed, the criteria that determine the work has been completed successfully, the schedule, including any key delivery dates, what resources are too be supplied by you (your expertise, the fact that you will have your own software and hardware and work offsite, for example) and by the company (that they have personnel available to answer questions, test the software, etc.)

If you are engaging with a client for maintenance on an application with which you are unfamiliar, it makes sense to start the engagement carefully, as you or the client may discover early in the process that this is not a suitable task to be engaged in. As a first step, I like to perform a short audit of the application – a review of the documentation (if any!), the code, the project structure, and the data to get an idea of the level of effort required. See Auditing, below for more details. An audit is usually a short-term, fixed-price, fixed-schedule engagement that can pretty much be summarized as "let me see what I can figure out in 40 hours." The deliverable for an audit is the audit report, also covered below.

The next step is to review the changes requested. Depending on the extent of the changes and your estimated difficulty in performing them, this may be as short as writing up an estimate, or an additional period of developing formal specifications.

Finally, we get to the point of contracting to actually perform the maintenance. Depending on how you like to run your business, the inclination of the client, and the nature of the work, several forms of contract may be acceptable. If the requirements of the work are crystal-clear to you, a fixed bid is the option most welcome to the client, as this is the easiest for them to sell to upper management, easiest to fit in the budget. However, realize that with this arrangement, you are taking most of the risk, and your estimate should reflect that.

If the requirements are less clear, or if the changes required are large, or if you are uncertain if the changes can be performed without perturbations to other portions of the system, a time-and-materials contract is usually desirable. This allows you to better share the risk with the client, who gets better control of the final cost. In most cases, a client has a budget ceiling, whether self-imposed or external, and you can work with the client to prioritize their list of changes, and get them to make the hard decisions on what they can afford to have done.

Working closely with the client through the actual maintenance, you can each evaluate the costs of each change as they unfold. In many cases, a client will see that a particular task is going to take a larger effort than was first expected, and work with you to consider a more moderate change, one that will meet their business needs without requiring as great a programming effort. Keeping the client abreast of the changes as they happen, and setting the clients expectations so that they understand the process as it unfolds is the best way to arrive at a successfully and mutually agreeable conclusion.

Structuring a support contract

A support contract is a different business altogether from that of developing software, and you must be careful that you and the client clearly understand each party's responsibilities under the contract. If you are promising to respond to a report of problems within a certain period, you must ensure that you understand how to fulfill that obligation (or not have that obligation) at night, during weekends, holidays and vacations. If the client really, truly needs 24-hour response time, you must figure your costs to include lugging a laptop with you everywhere, or maintaining a staff to be that responsive. The client, in turn, must understand what their contract covers. A fellow consultant had a support arrangement where he was responsible for some routine maintenance of a client's database. When their system crashed, they demanded that he appear on site, at once, as that is what they thought they were paying him for.

Slices of the pie

Alan Schwartz, formerly of MicroMega, pointed out some years ago that each time you are reserving some future hours to perform a task, you are taking a slice from that pie of all the future hours you have available. If maintenance and support of each application you create takes 8 percent of your next year's billings, when you have completed your twelfth assignment, there are no more hours left to sell, nor any time left for developing new applications. While some consultants would consider this an ideal arrangement – there is no need to wonder what you will be doing next month – there are many liabilities to this. First, it is the nature of the software development life-cycle that, at some point, an application will have outlived its time and be put aside. That time becomes available again, and you need to plan for it. On the other side of the coin, if you have done nothing but maintain a 5-year-old application, you will find that your skills are now 5 years old and far less marketable.

A balance must be struck between taking on new work and supporting and maintaining older applications. You need to budget time to develop new skills, learn new technologies, and market those skills. Plan to structure your engagements so that you can achieve this balance.

Auditing an Application

So, you're interested in taking on a maintenance assignment for a client. If you are considering maintenance on a project you have not worked on before, the first task is to get oriented by determining where all of the source code and documentation are located, how much documentation is available, the quality of each, so that you can start to form an estimate on the difficulties of the task. Again, like developing a cost-benefit analysis, this is a task that requires expertise, and should be compensated.

Review documentation

This bullet point is often one of the shortest tasks on the list, as the client looks back at you blank-faced and says "What documentation?" Hopefully, that is not the case. In the best of all possible worlds, you will have the following documentation to review:

- ERD

- User Manuals

- Developer manuals, standards, notes

Locate all source code

This, too, ought to be a short-lived task, but often takes far longer than expected. In many cases, I have seen source code spread over multiple network servers, referenced by mapped drive letters that no longer exist, or buried deep within a developers CONFIG.FP path statements. Ideally, all the code is located in one place. The best test to confirm you have all of the source code is to assemble it on a "clean" machine, bringing in only those files you believe are part of the project, verify that you can build the project, and then run the application through its paces. Many developers use the trick of indirect referencing of code (using structures like DO ("programname") or DO &lcProgram that might not be found in the build process.

Sort “live” code from old/temp code

Following up on the idea above, a "clean" build should only include the minimum code needed In many cases, developers wrote "quick and dirty" code for a one-time data repair, import or structure change, and these can take up useless megabytes of disk space. Similarly, many tables were just used for one-time data extracts or conversions and can also be discarded.

The goal of these two steps is to minimize the amount of code you will need to examine and evaluate. An additional benefit is that you will have a fairly clean baseline of code to work from. If you can build and run the application to the satisfaction of the client, you can be confident that if the behaviors change later in the project, that is caused by something you have done, rather than incorrect code you started out with.

Preserve this code with backups and source code control as your baseline.

Process Project File or Directory

A second check on the ability to identify code and related files is to attempt to build the project by creating an empty project, adding in main program, and using the "rebuild project" option to let the project manager add all needed files into the system. Compare this version with the version of the project you are supplied. A text search of the code for the names of the missing files should reveal if indirection should bring the files into the project (in which case you should add matching EXTERNAL statements to the main program).

Again, build the project into an executable. Compare its gross size with the client's executable to identify if you are missing any included tables, graphics or other large files.

Count #lines of code, comments, files

One key metric I have found of use when assessing the maintainability of a project's code is the number of lines of code per line of comment. On the conference CD, you will find a quick-and-dirty program I use to calculate this figure. (This code harks back to its 1993 origins: while it uses older and less efficient methods of reading files and processing projects, it still (last time I checked) ran in all versions of FoxPro after 2.0). The ProjScan application asks for a project file to read, then returns a text display listing the number of files of each type, the number of lines of code and comments in each one, the ratio of code to comments, and grand totals for the entire project. A sample result is shown in figure 1. The result of 2.56 lines of code per line of comments is very good. With this metric, and the projects I've worked on, my experience has been that when there are more than 7 lines of code per comment (on a project of any significant size), there is a problem with under-documentation.

Figure 1: The results of analyzing a project for code/comments ratio.

Review code

The ratio of code to comments is only one aspect of the review, and often a misleading one. In one case, I discovered that every comment contained within an application was not truly a comment, but rather a remarked-out line of code! You need to check the results you see in the metric against the real code. Using the checklists, included as an appendix at the end of this paper, a random sampling of code, including at least one each of the programs, forms, reports, class libraries and menus, should be reviewed for adherence to good coding practices. Remember, the focus of this evaluation is to determine how maintainable the code the code will be. Bad capitalization can be overlooked, and indenting cured by Beautify, but poor variable scoping will haunt you through the life of the application.

In many cases, you will find code that has been moved unchanged through many versions of FoxPro. Clipper add-on functions and template-generated FoxBASE code is not unheard of. While "if it works, don't mess with it" is a good rule of thumb, code written before the use of local variables and data sessions can be code in need of maintenance.

Review other programmer's code is a difficult process. It is essential to approach it in an impersonal manner. Too many programmers glance at another developers code and scoff, "This is garbage! You need to throw it all out and start again!" While this is true in some cases, in many other situations that is not the answer in the best interest of the clients.

Project structure, framework

Review the overall project structure. Are files included within the project that do not appear to be used? Are comments attached to each of the files?

Is the framework a home-grown framework? Is there a framework, separate from the working programs, or is it all merged together? Is the framework a commercial or public framework, and is it documented and supported?

Code structure

In reviewing each individual code element, look for an overall pattern of good software engineering, or clear indications of sloppiness. Are variables declared and scoped? Are inputs and outputs of routines clearly identified? Are good programming practices, like minimizing the number of exit points from a structure?

Commenting

My company’s coding standards state "Uncommented code is incomplete code." In the best routines, you should be able to read nothing but the comments, and follow the flow of logic directly to the point you wish to examine. Steve McConnell, in Code Complete, suggests that an excellent way to write a routine is to first write out the flow of code, in pseudo-code and comments. If this practice was followed in the code you are maintaining, the maintenance will be much easier.

Header comments with the name, date, author, purpose, parameters, return values and change log are ideal. If these are not present, consider adding at least the minimum structure using a tool like the Cob Editor Enhancements (CEE) available for free from http://www.cobsystem.com.

At a minimum, code should be present to explain when the "magic happens" – those portions of code that have the complex and difficult-to-decode algorithms.

Audit Report

Finally, after you have taken the steps above, you can conclude the audit phase of application maintenance by assembling your findings in an audit report, using the forms included at the end of this document. In a few pages, you can give the client an overview of their application, from the grand picture of how the application was engineered to the details of how individual portions were executed. With this document, you are in a much better position to justify your assertion that the code will be difficult to manage.

Tools to Maintain Application

Provided that you and the client are able to reach an agreement on what needs to be accomplished and what it is going to cost, this is the phase where you finally get to work on some code. Use the following tools to make the process easier.

Existing documentation, if any

Studying the existing documentation can give you an understanding of the mindset of the original programmers, and allow you to understand what were the intentions, occasionally misguided, of the developers.

Use the FoxPro tools to understand and explore the application

FoxDoc / Documenting Wizard and Beautify: Cleaning up the code to make it consistent and more readable is a favor you do for yourself as well as a good maintenance practice. In addition, the cross-reference tables and logic flow created by the documenting wizard can point out larger logic structures that you may not have picked up in the audit phase.

Filer and HexEdit: In many cases, you need to resort to brute force searching to locate all references to a file or variable. Using "File Finder" a.k.a. Filer (located in Tools/Filer in VFP 6.0 and later) to search .SCT and .VCT files can tell you which forms or classes reference the searched item. The hexadecimal editor, HexEdit.App, located in Home()+"Tools\HexEdit\" allows you to examine the structure of binary files.

Class Browser: Use the Class Browser to explore and document the object model.

Hacker’s Guide / Help File: In several applications I have work on recently, I have run across commands I have not seen used since the FoxBASE days, such as JOIN. Keep your favorite help file handy to decode the obscure stuff.

Issues in Maintenance

It can be a real challenge to recreate the environment in which the application is running, and this is point that should not be underestimated in the development process. Since you will want to incrementally test your changes, you need to reproduce the client's environment as faithfully as possible. Of course, in parallel with this, you will want to encourage your client to consider upgrading their existing hardware and software, ensure that they have the latest service packs and drivers on their systems, but often, for political, budgetary or technical reasons, the client must maintain older systems in their current state. There is nothing more embarrassing that having to tell a client "Gee, it didn't do that back at my office."

Be aware that there are subtle differences in relatively minor versions. There are clients still running FoxPro for Windows 2.5b because a number of things got broken in version 2.6. Similarly, any other change to the underlying operating system, add-ons such as ODBC, MDAC, ADO or OLE can cause incompatibilities. Be sure to retest the application carefully after making such changes.

Clients are not always aware of such subtleties. Upgrading to Office 2000, for example, installs new ODBC drivers that can kill applications that depended on the "FoxPro Files" DSN supported in all previous ODBC versions. A client of mine was recently surprised when he could not get his FoxPro DOS application to work with his new printer – an USB-connected model! Another client was really surprised that an "upgrade" to Windows Millennium Edition broke any DOS applications that required a FILES setting greater than 30 (see Microsoft Knowledge Base Q269030 for details on the work-around).

WTF Code

“Two wrongs don’t make a right, but three rights make a left.”

Sometimes you run into code that is so obscure, so strangely configured, that you can't help exclaiming "What's This For?" We all have tales of code we have seen that is just so amazingly wrong – and more amazing still because it works.

A light touch is the right touch. While it is always tempting to rip out all of the offending code and write it right from scratch, always consider the client's perspective. If a minor change can be accomplished without disturbing the rat's nest of code you have uncovered, try to do that. Of course, document what you are finding, and make note of code that you would like to fix, should the client chose to fund an overhaul of the application once the current changes are completed, successfully and under budget.

Summary

Application maintenance is not all that different a task from developing applications. The tasks of working with a client to establish expectations, develop a realistic set of specifications and work out a budget to get them accomplished, are all done in a similar manner to original development. A new wrinkle is introduced in the task of auditing and evaluating the existing application. This is both an impartial review of the state of the application, and an opinion on how easily your skills can be adapted to making the required changes. Finally, using the skills and tools at hand, the task of making the changes and testing them tests your skills as a "forensic debugger." If you anticipate the challenge, and take each application as a learning experience, the job of maintaining applications can be a rewarding one.

References

Hentzen, Whil, The 1999 Software Developer's Guide, Hentzenwerke Publishing, 1998, 0-96550-932-X. Thanks to Whil for permission to adapt and include his checklists.

McConnell, Steve, Software Project Survival Guide, Microsoft Press, 1997, ISBN 1-57231-6217

Appendix A: Sample Audit forms
System Overview

Reviewer______________________________________________________

Date______________________________________________________

Client______________________________________________________

System name_____________________________________________________________ Module

Purpose of System

Application Development Software Microsoft Visual FoxPro Version

Number of users

Distribution (multiple locations, stand-alone, networks)

Number of tiers___________________________________________________________

Distributed architecture description (WAN, transaction processors, application processors, etc.)

System Documentation

q Design Documents?

q Entity-relationship diagrams?

q Dataflow or workflow diagrams?

q OOP diagrams (sequence, structure, composition, messaging)? UML?

q Programmer documentation?

q User manuals?

System Framework

Is system based on a framework? No If so, which version?

q Devigus Visual Extend

q F1 Technologies' FoxExpress

q Flash Codebook

q Metamor MaxFrame

q Microsoft FFC

q Visual ProMatrix

q Other

Comments:

Project Checklist

Reviewer ______________________________________________________

Date______________________________________________________

Client______________________________________________________

System name______________________________________________________

Project Name______________________________________________________

These items can be examined without even opening up a code window:

q Does the system compile without errors?

q Does code compile with StrictDate set to 2?

q Do all of the functions work (has anything been stubbed out)?

q Are all of the files (programs, screens, reports) and procedures used, or does the project contain things that aren’t used anymore?

q Are files commented within the Project Manager?

q If files are called indirectly (example: DO ("Acme.PRG"), are they included in the project with EXTERNAL statements?

File Type	Count	Code Lines	Comment Lines	Code/Comment Ratio
Class Library
Icon
Library
Menu
Other
PJX Header
Program
Report
VFP Form
Totals
Count: Number of physical files Code-lines: executable lines, not counting white space. Executable lines continued on multiple lines are considered one line of code. Comments: non-executable lines starting with an asterisk or NOTE command. Automatically generated comments, such as those from the Screen or Menu Generator, are not counted. Executable code commented with the FoxPro standard comment *!* are not counted as comments.

Code Checklist

Reviewer ______________________________________________________

Date______________________________________________________

Client______________________________________________________

System name_____________________________________________________________ Module

Heading

q Does each routine have a proper header?

q Is the header updated to reflect major changes in the routine?

q Are parameters described in the header?

q Are return values specified?

Comments & Formatting

q Is the code formatted with indenting and white space?

q Are there comments in the code? (A good rule of thumb for knowing when there are "enough" comments in the code is to remove the code and just read the comments. Is it still clear what the routine is doing?)

q Do the comments explain why? (The comments should not simply answer "what" the routine is doing – but why.) If there are episodes of "magic" throughout the code, are they explained in the comments?

q Are long or complex calculations commented?

q Are there comments to indicate where, when and by whom the code has been modified (This is good, by the way.)?

q Have chunks of code been commented out with no explanation of why the code was left in place?

q Does the code contain "magic numbers" or are they (1) explained, or (2) DEFINED as appropriate for the language?

Variables

q Are variables named consistently and according to a standard?

q Are variables scoped?

q Are variables initialized?

Logic flow

q Are required parameters checked immediately upon entering the routine, both for their presence and their data type?

q Are variables declared, with a comment as to their purpose, and released properly?

q Does each logic structure check for the "in all other cases" condition? For example, does each IF have an ELSE and each DO CASE have an OTHERWISE?

q Are there complex nestings many levels deep or structures that look very elaborate?

q How many points of exit does the routine have? Do they do the same things?

q Is a return value always returned?

q Are all possible return values of the same data type?

q Are similar but not identical functions used as if they were interchangeable?

q Are there blocks of repetitious code?

Coding

q Are string comparisons handled properly - both with respect to exactness and case?

q Does arithmetic performed on dates handle the turn of the millennium gracefully?

q Are divisors tested for zero?

q Are file locations hard-coded or is the application portable?

q Are NULL values possible? If so, are they checked?

q Are macros used appropriately?

Data Review

Reviewer ______________________________________________________

Date______________________________________________________

Client______________________________________________________

System name_____________________________________________________________ Database Name

q Is an Entity-Relation Diagram (ERD) supplied?

q Is the ERD available in machine-readable format – ERwin, xCase, S-Designor or Other?

q Does the actual data structure appear to match the ERD?

q Is there a consistent table and field naming convention?

q Are reserved words such as DATE and MEMO used as field or table names?

q Are all entities contained within a single database container? (Utility tables, output tables and other tables may have to be free tables)

q Are comments included for tables and fields?

q Do all tables contain a primary key?

q What are the datatype(s) of primary keys?

q Are primary keys data-bearing or non-data-bearing?

q How are PKs generated?

q Are persistent relationships defined within the database?

q Is referential integrity code defined within the database?

q Are connections defined within the database or created manually?

q If connections are defined, how is login security maintained?

q Are connection settings the default values or have they been adjusted?

q Are views set to share connections?

q Are default values specified?

q How is null handling performed?

q Are views used?

q Do view definitions include comments?

q Are view definitions created with the view designer, by hand, or with a third-party tool (SDT, E-View)?

q Are parameters for views listed?

q Is DBCx or other metadata repository used?